{"thread":{"id":"40640","subject":"Why are submodules not automatically handled by default or at least configurable to do so?","startedAt":"2015-10-25T23:10:07Z","lastAt":"2015-10-28T07:36:11Z","messageCount":11,"participants":["John Smith","Chris Packham","Nazri Ramliy","Stefan Beller","Jens Lehmann","Junio C Hamano","Nick","Davide Fiorentino","Konstantin Khomoutov"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"272211","messageId":"D4E5E890658.000004DCjohsmi9933@inbox.com","threadId":"40640","inReplyTo":null,"subject":"Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"John Smith","fromEmail":"johsmi9933@inbox.com","sentAt":"2015-10-25T23:10:07Z","receivedAt":"2015-10-25T23:10:07Z","isPatch":false,"sender":{"key":"johsmi9933@inbox.com","avatar":null},"body":"I found that I use submodules much, much more often in my git projects than I used externals\nin Subversion and the reason is that git encourages/forces to organize large projects into\nsmaller repositories, one reason for this being that subversion allows to check out parts of \na repository while git does not.\n\nBut when I clone a git repository with subprojects, I (and everyone else) has to remember to\nadd the --recursive option. When switching between branches with different versions/commits of the \nsubmodules everyone has to remember to update the submodules. When updating a submodule \neveryone has to remember to recurse there too. \n\nBasically, everything with submodules has to be done manually every time and there seems \nto be no way to change that default.\n\nWhy is that? Basically all the time I use submodules I would want automatic handling of \nsubmodules to happen and I cannot remember having had a single situation where I would \nnot have wanted it to happen. So why does git default to doing nothing? \nWhy does it not provide a way to enable automatic\npulling/updating of submodules e.g. when cloning or switching branches?\nWhen would people routinely check out a branch and want to stay with the submodules as \nthe have been checked out for the old branch?\n\nI honestly do not understand it. \n\nJohn\n\n____________________________________________________________\nCan't remember your password? Do you need a strong and secure password?\nUse Password manager! It stores your passwords & protects your account.\nCheck it out at http://mysecurelogon.com/manager\n"},{"id":"272213","messageId":"CAFOYHZAKvN8xMKePCNFgo_ySHr0dc0+ASY0ux7j0p8UF1fuWCQ@mail.gmail.com","threadId":"40640","inReplyTo":"D4E5E890658.000004DCjohsmi9933@inbox.com","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Chris Packham","fromEmail":"judge.packham@gmail.com","sentAt":"2015-10-26T00:56:52Z","receivedAt":"2015-10-26T00:56:52Z","isPatch":false,"sender":{"key":"judge.packham@gmail.com","avatar":"https://avatars.githubusercontent.com/u/155667?v=4"},"body":"On Mon, Oct 26, 2015 at 12:10 PM, John Smith wrote:\n> I found that I use submodules much, much more often in my git projects than I used externals\n> in Subversion and the reason is that git encourages/forces to organize large projects into\n> smaller repositories, one reason for this being that subversion allows to check out parts of\n> a repository while git does not.\n>\n> But when I clone a git repository with subprojects, I (and everyone else) has to remember to\n> add the --recursive option. When switching between branches with different versions/commits of the\n> submodules everyone has to remember to update the submodules. When updating a submodule\n> everyone has to remember to recurse there too.\n\nThe config option fetch.recurseSubmodules exists. It's not quite the\nsame as what git clone --recurse-submodules does but it's a start.\n\n>\n> Basically, everything with submodules has to be done manually every time and there seems\n> to be no way to change that default.\n>\n> Why is that? Basically all the time I use submodules I would want automatic handling of\n> submodules to happen and I cannot remember having had a single situation where I would\n> not have wanted it to happen. So why does git default to doing nothing?\n\nIt's hard to pick a default that suits every workflow that submodules\nsupport. Also with submodules there is a chicken-and-egg scenario.\nWhile you can put things in ~/.gitconfig most of what you'd want to\nconfigure when using submodules would be in super/.git/config but that\ndoesn't exist until you've cloned super.git.\n\n> Why does it not provide a way to enable automatic\n> pulling/updating of submodules e.g. when cloning or switching branches?\n\nI believe Jens and Stefan (Cc'd) have been doing some great work in\nthis area. Jens even posted his todo list a few days ago\n(https://github.com/jlehmann/git-submod-enhancements/wiki).\n\n> When would people routinely check out a branch and want to stay with the submodules as\n> the have been checked out for the old branch?\n>\n> I honestly do not understand it.\n>\n> John\n>\n> ____________________________________________________________\n> Can't remember your password? Do you need a strong and secure password?\n> Use Password manager! It stores your passwords & protects your account.\n> Check it out at http://mysecurelogon.com/manager\n>\n>\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"272216","messageId":"CAEY4ZpPduXXqgW3rWn9rzkpHrTvY8QfPX=YcBZ_DpyVwnsZ6jw@mail.gmail.com","threadId":"40640","inReplyTo":"D4E5E890658.000004DCjohsmi9933@inbox.com","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Nazri Ramliy","fromEmail":"ayiehere@gmail.com","sentAt":"2015-10-26T04:48:48Z","receivedAt":"2015-10-26T04:48:48Z","isPatch":false,"sender":{"key":"ayiehere@gmail.com","avatar":"https://avatars.githubusercontent.com/u/164756?v=4"},"body":"On Mon, Oct 26, 2015 at 7:10 AM, John Smith wrote:\n> When would people routinely check out a branch and want to stay with the submodules as\n> the have been checked out for the old branch?\n\nI do this a lot. At my $dayjob we have a super project with bunch of\nsub projects.\nEach subproject has its corresponding rpm spec file in the\nsuperproject - it's quite\noften that I work on a \"git-merge-base--octopus\" branch that updates only the\nspec files and nothing else - so when changing between branches I\ndon't care what\nstates the submodules are in. When the fixes to the spec files are ready I just\ncheckout to the respective branches and merge in the changes - I don't actively\ndo \"git submodule update\" when switching to different branches.\n\nnazri\n"},{"id":"272243","messageId":"CAGZ79kb2kStk0+MqUXREH3g+rqbsXoNiTGj=4SxJ4vOR8TqEcA@mail.gmail.com","threadId":"40640","inReplyTo":"CAFOYHZAKvN8xMKePCNFgo_ySHr0dc0+ASY0ux7j0p8UF1fuWCQ@mail.gmail.com","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-26T16:28:11Z","receivedAt":"2015-10-26T16:28:11Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Sun, Oct 25, 2015 at 5:56 PM, Chris Packham wrote:\n> On Mon, Oct 26, 2015 at 12:10 PM, John Smith wrote:\n>> I found that I use submodules much, much more often in my git projects than I used externals\n>> in Subversion and the reason is that git encourages/forces to organize large projects into\n>> smaller repositories, one reason for this being that subversion allows to check out parts of\n>> a repository while git does not.\n>>\n>> But when I clone a git repository with subprojects, I (and everyone else) has to remember to\n>> add the --recursive option. When switching between branches with different versions/commits of the\n>> submodules everyone has to remember to update the submodules. When updating a submodule\n>> everyone has to remember to recurse there too.\n>\n> The config option fetch.recurseSubmodules exists. It's not quite the\n> same as what git clone --recurse-submodules does but it's a start.\n>\n>>\n>> Basically, everything with submodules has to be done manually every time and there seems\n>> to be no way to change that default.\n>>\n>> Why is that? Basically all the time I use submodules I would want automatic handling of\n>> submodules to happen and I cannot remember having had a single situation where I would\n>> not have wanted it to happen. So why does git default to doing nothing?\n\nIIUC at the time submodules were invented, there was need for lots of\ncode to be written.\nEach command needed new code to deal with submodules. As there was not\nenough people/time\nto do it properly, the \"do nothing\" was the safest action which could\nbe added fast.\n\n>\n> It's hard to pick a default that suits every workflow that submodules\n> support. Also with submodules there is a chicken-and-egg scenario.\n> While you can put things in ~/.gitconfig most of what you'd want to\n> configure when using submodules would be in super/.git/config but that\n> doesn't exist until you've cloned super.git.\n>\n>> Why does it not provide a way to enable automatic\n>> pulling/updating of submodules e.g. when cloning or switching branches?\n>\n> I believe Jens and Stefan (Cc'd) have been doing some great work in\n> this area. Jens even posted his todo list a few days ago\n> (https://github.com/jlehmann/git-submod-enhancements/wiki).\n\nYeah I would also point at Jens' wiki today.\n\nAll I did up to now was rewriting parts of the submodule code in C\n(git submodule update specifically), while the code/patches you find at Jens'\ncopy of Git includes already lots of useful stuff such as `git\ncheckout --recurse-submodules`\n(IIRC you don't need to type --recurse-submodules if you configured\nthat to be the default)\n\n>\n>> When would people routinely check out a branch and want to stay with the submodules as\n>> the have been checked out for the old branch?\n\nAs said above, it was a sane choice which could be implemented fast, IIUC.\n\nI mean what would happen if you had commits made in the submodule, or\njust a dirty working tree?\n\n>>\n>> I honestly do not understand it.\n>>\n>> John\n>>\n\nStefan\n"},{"id":"272246","messageId":"562E5B41.9090801@web.de","threadId":"40640","inReplyTo":"CAEY4ZpPduXXqgW3rWn9rzkpHrTvY8QfPX=YcBZ_DpyVwnsZ6jw@mail.gmail.com","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2015-10-26T16:56:33Z","receivedAt":"2015-10-26T16:56:33Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 26.10.2015 um 05:48 schrieb Nazri Ramliy:\n> On Mon, Oct 26, 2015 at 7:10 AM, John Smith wrote:\n>> When would people routinely check out a branch and want to stay with the submodules as\n>> the have been checked out for the old branch?\n>\n> I do this a lot. At my $dayjob we have a super project with bunch of\n> sub projects.\n> Each subproject has its corresponding rpm spec file in the\n> superproject - it's quite\n> often that I work on a \"git-merge-base--octopus\" branch that updates only the\n> spec files and nothing else - so when changing between branches I\n> don't care what\n> states the submodules are in. When the fixes to the spec files are ready I just\n> checkout to the respective branches and merge in the changes - I don't actively\n> do \"git submodule update\" when switching to different branches.\n\nWhich seems a bit error prone as you could forget to update the submodules\nand build incorrect rpms from them, or am I missing something?\n\nI understand why you don't need to update the submodules every time, but\nwould it hurt your workflow if they did (but don't get me wrong, that will\nalways be configurable).\n"},{"id":"272258","messageId":"xmqqoaflh0ef.fsf@gitster.mtv.corp.google.com","threadId":"40640","inReplyTo":"CAGZ79kb2kStk0+MqUXREH3g+rqbsXoNiTGj=4SxJ4vOR8TqEcA@mail.gmail.com","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-26T19:53:28Z","receivedAt":"2015-10-26T19:53:28Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> IIUC at the time submodules were invented, there was need for lots of\n> code to be written.\n> Each command needed new code to deal with submodules. As there was not\n> enough people/time\n> to do it properly, the \"do nothing\" was the safest action which could\n> be added fast.\n\nThat is quite different from how I remember. Soon after Linus and I\nadded the Gitlink in early Apr 20007, an early subproject/gitlink\n(thought) experiment was started with help from folks like Steven\nGrimm, Jan Hudec, Petr Baudis, Alex Riesen etc. The first principle\nof the design throughout that era was \"we admit that we do not know\nall the use cases, so let's start small and solid and make sure that\nsmall-and-solid thing can later be enhanced as people discover the\nway how they want to work\" (you can see me expressing that sentiment\nin $gmane/48287, for example).\n\nSo it wasn't \"not enough people to do it properly\" at all. It was\n\"we admit that we do not know what is proper, so we defer to actual\nusers to define what is proper for them\".\n"},{"id":"272303","messageId":"562F5704.5070405@letterboxes.org","threadId":"40640","inReplyTo":"D4E5E890658.000004DCjohsmi9933@inbox.com","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Nick","fromEmail":"oinksocket@letterboxes.org","sentAt":"2015-10-27T10:50:44Z","receivedAt":"2015-10-27T10:50:44Z","isPatch":false,"sender":{"key":"oinksocket@letterboxes.org","avatar":null},"body":"I too am interested in finding ways to automate working with submodules, \nas it's a particular pain point with my colleagues. They frequently \nshoot themselves in the foot trying to branch and merge a project with \nsubmodules, resulting in a broken build and grumpy comments about git \n(or possibly about me, as the \"Git Advocate\"). And they're right, it is \nawkward.\n\nWhether or not any sensible default configuration exists, users do need \na way to avoid excessively complicated workflows, or people are just \ngoing to avoid using submodules, or perhaps Git. Hand-rolled solutions \nmight be better than nothing, but I would expect that this is a common \nissue which would benefit from a built-in solution. For instance, so \nthat one can branch and merge the whole project without duplicating the \nwork for each submodule.\n\nAm I correct in thinking there isn't anything which does these kind of \nthings yet?\n\nThanks!\n\nNick\n"},{"id":"272304","messageId":"EC0D15E1-82B3-4A4C-96DE-8922AB870E2B@gmail.com","threadId":"40640","inReplyTo":"562F5704.5070405@letterboxes.org","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Davide Fiorentino","fromEmail":"davide.fiorentino@gmail.com","sentAt":"2015-10-27T10:56:32Z","receivedAt":"2015-10-27T10:56:32Z","isPatch":false,"sender":{"key":"davide.fiorentino@gmail.com","avatar":"https://gravatar.com/avatar/e314e7bc9824cf26157df764c7f19351dfb2747341b045aa4a0b280c0c9f0444?d=mp&s=160"},"body":"Why not set alias(es) for that?\n\nBest,\nDavide\n> On 27 Oct 2015, at 10:50, Nick wrote:\n> \n> I too am interested in finding ways to automate working with submodules, as it's a particular pain point with my colleagues. They frequently shoot themselves in the foot trying to branch and merge a project with submodules, resulting in a broken build and grumpy comments about git (or possibly about me, as the \"Git Advocate\"). And they're right, it is awkward.\n> \n> Whether or not any sensible default configuration exists, users do need a way to avoid excessively complicated workflows, or people are just going to avoid using submodules, or perhaps Git. Hand-rolled solutions might be better than nothing, but I would expect that this is a common issue which would benefit from a built-in solution. For instance, so that one can branch and merge the whole project without duplicating the work for each submodule.\n> \n> Am I correct in thinking there isn't anything which does these kind of things yet?\n> \n> Thanks!\n> \n> Nick\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"272305","messageId":"562F629F.7040206@letterboxes.org","threadId":"40640","inReplyTo":"EC0D15E1-82B3-4A4C-96DE-8922AB870E2B@gmail.com","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Nick","fromEmail":"oinksocket@letterboxes.org","sentAt":"2015-10-27T11:40:15Z","receivedAt":"2015-10-27T11:40:15Z","isPatch":false,"sender":{"key":"oinksocket@letterboxes.org","avatar":null},"body":"On 27/10/15 10:56, Davide Fiorentino wrote:\n> Why not set alias(es) for that?\n\nThat counts as a hand-rolled (i.e. ad-hoc) solution. So not out of the \nquestion, but I'd rather point my colleagues at something tried and \ntested, rather than simply re-invent wheels, possibly badly.\n\nI'd be interested if there are some out there I could adopt?\n\nBut oh yes, there is another difficulty with aliases. Eclipse users on \nWindows: they don't tend to love it if you tell them to install Cygwin, \nopen a shell and type things into it. The Eclipse experience is \nsomewhat fraught already (e.g. EGit running out of memory cloning \nmoderate-sized repos, and other UI difficulties). I'd be surprised if \ndefining aliases is going to help their user experience. I'm \nconsidering suggesting they switch to to IntelliJ, but that's also \nasking quite a lot of people who may be reluctant to relearn their whole \nworkflow again, and I'd need to do the research to ensure this doesn't \njust make things more confusing.\n\nCheers,\n\nN\n"},{"id":"272307","messageId":"20151027151637.d3ed7f0dd8c72c78d373c520@domain007.com","threadId":"40640","inReplyTo":"562F629F.7040206@letterboxes.org","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Konstantin Khomoutov","fromEmail":"kostix+git@007spb.ru","sentAt":"2015-10-27T12:16:37Z","receivedAt":"2015-10-27T12:16:37Z","isPatch":false,"sender":{"key":"kostix+git@007spb.ru","avatar":null},"body":"On Tue, 27 Oct 2015 11:40:15 +0000\nNick wrote:\n\n> > Why not set alias(es) for that?\n[...]\n> But oh yes, there is another difficulty with aliases. Eclipse users\n> on Windows:\n[...]\n\nNot to counter your actual argument, but AFAIK EGit uses JGit which is\na Java implementation which does not call out to the \"real\" Git binary.\nThis basically means that if something gets implemented in the stock\nGit, this won't affect JGit and EGit untill their respective\nmaintainers implement the same feature.\n\n> they don't tend to love it if you tell them to install\n> Cygwin, open a shell and type things into it.\n\nOn Windows, you typically want them to use Git for Windows, not\nCygwin. Various GUI front-ends to Git working on Windows (such as Git\nExtentions and TortoiseGit) rely on GfW to work as well.\n"},{"id":"272398","messageId":"CAEY4ZpNrvF3-aLYpoN1+Qi-ZcFUqh_t8QrVpWqqgnEHqF4_cOQ@mail.gmail.com","threadId":"40640","inReplyTo":"562E5B41.9090801@web.de","subject":"Re: Why are submodules not automatically handled by default or at least configurable to do so?","fromName":"Nazri Ramliy","fromEmail":"ayiehere@gmail.com","sentAt":"2015-10-28T07:36:11Z","receivedAt":"2015-10-28T07:36:11Z","isPatch":false,"sender":{"key":"ayiehere@gmail.com","avatar":"https://avatars.githubusercontent.com/u/164756?v=4"},"body":"On Tue, Oct 27, 2015 at 12:56 AM, Jens Lehmann wrote:\n> Which seems a bit error prone as you could forget to update the submodules\n> and build incorrect rpms from them, or am I missing something?\n\nFor my case I'm not building the rpms directly after merging in the fixes\ndone in the octopus branch, so I don't care much about the state of the\nsubmodules after the merge since I know that the octopus branch does\nnot modify any submodules. The rpms are built later on, possibly on\nanother machine where the submodules are updated w.r.t to each branch\nin a release commit.\n\n> I understand why you don't need to update the submodules every time, but\n> would it hurt your workflow if they did (but don't get me wrong, that will\n> always be configurable).\n\nI'd say it depends - for the times when all I want to do is work on plain files\nin the superproject (on an octopus branch for example) I don't want git to\nautomatically update the submodules everytime I switch branches - it would\nhurt my productivity as there will be more disk activities due to files being\nchecked out unnecessarily for updating the submodules.\n\nThe submodule states are not committed that often into the superproject,\nthey are done normally when we're cutting a new release. During day-to-day\ndevelopment each developer runs a script that pulls in the latest commits\nfor \"hot\" submodules.\n\ngit not updating the submodule state automatically is actually a convenient\nfor my particular use case here.\n\nnazri\n"}]}