{"thread":{"id":"24036","subject":"RFC: Making submodules \"track\" branches","startedAt":"2010-06-07T23:29:07Z","lastAt":"2012-11-20T12:04:37Z","messageCount":21,"participants":["Ævar Arnfjörð Bjarmason","Johan Herland","Marc Branchaud","Jens Lehmann","Junio C Hamano","Steven Michalske","nottrobin","W. Trevor King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"143195","messageId":"AANLkTilBQPHgkCLJ7ppNo5TwC9Bdmqo-OMRpaDFwbQPd@mail.gmail.com","threadId":"24036","inReplyTo":null,"subject":"RFC: Making submodules \"track\" branches","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-07T23:29:07Z","receivedAt":"2010-06-07T23:29:07Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Fri, May 21, 2010 at 16:10, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:\n> Add a $toplevel variable accessible to `git submodule foreach`, it\n> contains the absolute path of the top level directory (where\n> .gitmodules is).\n>\n> This makes it possible to e.g. read data in .gitmodules from within\n> foreach commands. I'm using this to configure the branch names I want\n> to track for each submodule:\n>\n>    git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'\n>\n> For a little history: This patch is borne out of my continuing fight\n> of trying to have Git track the branches of submodules, not just their\n> commits.\n>\n> Obviously that's not how they work (they only track commits), but I'm\n> just interested in being able to do:\n>\n>    git submodule foreach 'git pull'\n>\n> Of course that won't work because the submodule is in a disconnected\n> head, so I first have to connect it, but connect it *to what*.\n>\n> For a while I was happy with this because as fate had it, it just so\n> happened to do what I meant:\n>\n>    git submodule foreach 'git checkout $(git describe --all --always) && git pull'\n>\n> But then that broke down, if there's a tag and a branch the tag will\n> win out, and I can't git pull a branch:\n>\n>    $ git branch -a\n>    * master\n>      remotes/origin/HEAD -> origin/master\n>      remotes/origin/master\n>    $ git tag -l\n>    release-0.0.6\n>    $ git describe --always --all\n>    release-0.0.6\n>\n> So I figured that I might as well start tracking the branches I want\n> in .gitmodules itself:\n>\n>    [submodule \"yaml-mode\"]\n>        path = yaml-mode\n>        url = git://github.com/yoshiki/yaml-mode.git\n>        branch = master\n>\n> So now I can just do (as stated above):\n>\n>    git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'\n>\n> Maybe there's a less painful way to do *that* (I'd love to hear about\n> it). But regardless of that I think it's a good idea to be able to\n> know what the top-level is from git submodule foreach.\n\nThis patch is getting merged to next as per the June 2 What's cooking\nin Git post.\n\nBut I wonder how evil it would be to expand this this idea to allow\nthe porcelain to track branches instead of commits at the porcelain\nlevel.\n\nThat /could/ work like this. The tree format would be exactly the\nsame, i.e. bound to a specific commit:\n\n    $ git ls-tree HEAD | grep subthing\n    160000 commit 37469ca3fae264e790e4daac0fa8f2ddf8039c93  subthing\n\n*But*, the user could add some new submodule.*.* config key/values\nthat specify what branch the module should track and whether 'git\npull' on the master project should also pull new changes (from the\n'newstuff' branch) into the submodule:\n\n    [submodule \"subthing\"]\n        path = subthing\n        url = git://github.com/avar/subthing.git\n        branch = newstuff\n        update-on-pull = true\n\nCoupled with .gitignore this would allow for SVN-like externals that\nalways track the latest version of upstream, but it'd all be done on\nthe porcelain side.\n\nThe checked out copy wouldn't match the commit in the tree, but the\nuser could still git add && git commit it to record the new commit in\nthe master repository history.\n\nThe lack of this ability seems to be a fairly common complaint about\nsubmodules in Git, that you always have to do something in the parent\nproject to update the submodules, even if you don't care about\nspecific revisions, or the ability to roll back.\n\nI couldn't find a prior discussion of this on the list, maybe this has\nbeen beaten to death already.\n"},{"id":"143208","messageId":"201006080912.31448.johan@herland.net","threadId":"24036","inReplyTo":"AANLkTilBQPHgkCLJ7ppNo5TwC9Bdmqo-OMRpaDFwbQPd@mail.gmail.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2010-06-08T07:12:31Z","receivedAt":"2010-06-08T07:12:31Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Tuesday 08 June 2010, Ævar Arnfjörð Bjarmason wrote:\n> On Fri, May 21, 2010 at 16:10, Ævar Arnfjörð Bjarmason <avarab@gmail.com> \nwrote:\n> > Add a $toplevel variable accessible to `git submodule foreach`, it\n> > contains the absolute path of the top level directory (where\n> > .gitmodules is).\n> > \n> > This makes it possible to e.g. read data in .gitmodules from within\n> > foreach commands. I'm using this to configure the branch names I want\n> > to track for each submodule:\n> > \n> >    git submodule foreach 'git checkout $(git config --file\n> > $toplevel/.gitmodules submodule.$name.branch) && git pull'\n> > \n> > For a little history: This patch is borne out of my continuing fight\n> > of trying to have Git track the branches of submodules, not just their\n> > commits.\n> > \n> > Obviously that's not how they work (they only track commits), but I'm\n> > just interested in being able to do:\n> > \n> >    git submodule foreach 'git pull'\n> > \n> > Of course that won't work because the submodule is in a disconnected\n> > head, so I first have to connect it, but connect it *to what*.\n> > \n> > For a while I was happy with this because as fate had it, it just so\n> > happened to do what I meant:\n> > \n> >    git submodule foreach 'git checkout $(git describe --all --always)\n> > && git pull'\n> > \n> > But then that broke down, if there's a tag and a branch the tag will\n> > win out, and I can't git pull a branch:\n> > \n> >    $ git branch -a\n> >    * master\n> >      remotes/origin/HEAD -> origin/master\n> >      remotes/origin/master\n> >    $ git tag -l\n> >    release-0.0.6\n> >    $ git describe --always --all\n> >    release-0.0.6\n> > \n> > So I figured that I might as well start tracking the branches I want\n> > in .gitmodules itself:\n> > \n> >    [submodule \"yaml-mode\"]\n> >        path = yaml-mode\n> >        url = git://github.com/yoshiki/yaml-mode.git\n> >        branch = master\n> > \n> > So now I can just do (as stated above):\n> > \n> >    git submodule foreach 'git checkout $(git config --file\n> > $toplevel/.gitmodules submodule.$name.branch) && git pull'\n> > \n> > Maybe there's a less painful way to do *that* (I'd love to hear about\n> > it). But regardless of that I think it's a good idea to be able to\n> > know what the top-level is from git submodule foreach.\n> \n> This patch is getting merged to next as per the June 2 What's cooking\n> in Git post.\n> \n> But I wonder how evil it would be to expand this this idea to allow\n> the porcelain to track branches instead of commits at the porcelain\n> level.\n> \n> That /could/ work like this. The tree format would be exactly the\n> same, i.e. bound to a specific commit:\n> \n>     $ git ls-tree HEAD | grep subthing\n>     160000 commit 37469ca3fae264e790e4daac0fa8f2ddf8039c93  subthing\n> \n> *But*, the user could add some new submodule.*.* config key/values\n> that specify what branch the module should track and whether 'git\n> pull' on the master project should also pull new changes (from the\n> 'newstuff' branch) into the submodule:\n> \n>     [submodule \"subthing\"]\n>         path = subthing\n>         url = git://github.com/avar/subthing.git\n>         branch = newstuff\n>         update-on-pull = true\n\nI certainly like the idea, and so far this is the best way I've seen for \nassociating submodules to branches. I don't like the last \"update-on-pull\" \noption, though. It should probably be somewhat more general set of \noptions/triggers with a richer set of values than true/false. What about \nsomething like this?\n\n    [submodule \"subthing\"]\n        path = subthing\n        url = git://github.com/avar/subthing.git\n        branch = newstuff\n        on-pull = checkout,pull\n        on-checkout = checkout\n        on-commit = ignore (or commit?)\n        ...\n\nSee below for more discussion...\n\n> Coupled with .gitignore this would allow for SVN-like externals that\n> always track the latest version of upstream, but it'd all be done on\n> the porcelain side.\n> \n> The checked out copy wouldn't match the commit in the tree, but the\n> user could still git add && git commit it to record the new commit in\n> the master repository history.\n> \n> The lack of this ability seems to be a fairly common complaint about\n> submodules in Git, that you always have to do something in the parent\n> project to update the submodules, even if you don't care about\n> specific revisions, or the ability to roll back.\n> \n> I couldn't find a prior discussion of this on the list, maybe this has\n> been beaten to death already.\n\nThere are a lot of non-trivial challenges when you want to aggregate several \nsubmodule operations into a single \"toplevel\" command. Here are some off the \ntop of my head:\n\n- When submodule pulls result in conflicts, these must be presented to the \nuser in a way that's simple and straightforward for the user to resolve.\n\n- When switching branches in the superrepo, you sometimes also want to \nswitch branches in the submodule. This is signalled by changing the \nsubmodules.subthing.branch variable in .gitmodules between the two branches. \nHowever, it means that the submodule's update/pull operation must also be \ndone on 'checkout' in the superrepo.\n\n- How to handle local/uncommitted (staged or unstaged) modifications in a \nsubmodule when pulling or switching branches in the superrepo? The right \nanswer here is probably to do the same as in the no-submodule case, i.e. to \nrefuse if it would clobber/conflict with the local modifications.\n\n- When you track submodule branches instead of commits, the actual commit \nreferenced in the superrepo is no longer as important (provided it's part of \nthe ancestry of the submodule branch you're tracking). However, diff/status \nwill still list the submodule as changed because you checked out a different \ncommit from what Git has recorded. This raises two concerns: (1) What \n_should_ be considered \"changed\" from the diff/status perspective when \ntracking submodule branches? and (2) When do you update the commit reference \nin the submodule? \"never\" would work (since you're checking out a different \ncommit anyway), \"always\" would also work (for the same reason), but would \nlitter the superrepo history with submodule updates. There may be a better \nalternative somewhere in between.\n\n- If you want to give the illusion of \"one big repo\" then maybe it should \nalso be possible to trigger submodule commits from a superrepo commit? (i.e. \nhaving a single toplevel \"git commit\" also trigger commits in submodules). \nSome users will want to specify the commit message for each submodule \nseparately (IMHO the better approach), while some will want to give only one \ncommit message that is reused in every submodule commit.\n\n- As always with submodules, keep the case of nested submodules in mind.\n\nThere are probably more issues that escape me now...\n\nThanks for resurrecting the discussion.\n\n\nHave fun! :)\n\n...Johan\n\n-- \nJohan Herland, <johan@herland.net>\nwww.herland.net\n"},{"id":"143242","messageId":"4C0E630A.7020803@xiplink.com","threadId":"24036","inReplyTo":"201006080912.31448.johan@herland.net","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2010-06-08T15:34:34Z","receivedAt":"2010-06-08T15:34:34Z","isPatch":false,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 10-06-08 03:12 AM, Johan Herland wrote:\n> \n> There are probably more issues that escape me now...\n\nThanks for bringing this up!  I'm also very interested in this topic.\n\nThe main issue I see is that I don't always want my submodules to track (or\nnot track) a branch.  What I want changes depending on the circumstances.\n\nOne aspect I really like about submodules is the ease of tagging.  I can tag\nthe super-repo, and know that whenever I checkout that tag I'll always get\nthe corresponding versions of the submodules as they were when the tag was\nmade.  It would actually be disastrous, at least in my case, if the\nsubmodules were at the latest HEAD of some branch instead.\n\nThis goes for almost any commit in the super-repo's history, not just the\ntagged ones:  Whenever I checkout a historical committish, I want to get the\nsubmodules as they were when that commit was made.  Even if I'm working at\nthe HEAD of some branch, often that branch is based on a historical commit\nand I want to use the submodules as they were when that historical commit was\nmade.\n\nAll that said, I do think submodule branch tracking is useful.  Quite often a\ndevelopment topic will change the super-repo and one or more submodules.  It\nwould be extremely helpful to do that work in a branch that spans the\nsuper-repo and (a subset of) the submodules.  (In my mind this capability is\none of the main benefits Google's \"repo\" tool has over submodules.)\n\nSo, back to the issue at hand: Sometimes I want static (non-tracking)\nsubmodules, and sometimes I want dynamic (tracking) submodules.  IMO, this\nmakes Ævar's proposed configuration-based approach impractical.  (Of course,\nI'm not looking to replicate svn's externals...)\n\nI'm not sure what the right approach is, but I have some thoughts:\n\n - Maybe \"git branch\" should be able to create submodule-spanning branches.\n\n - If so, then checkout, merge, pull and other branch-related commands should\nhonor submodule-spanning branches.  \"checkout\" in particular needs to\ndistinguish between when it's checking out an actual branch vs. some other\ncommittish, and if the branch being checked out is submodule-spanning it\nshould checkout the latest HEAD of that branch for the submodules as well.\n\n - It *may* be good enough to assume that matching branch names in the\nsuper-repo and the submodules are in fact submodule-spanning branches.\n\n - Automating all this is tricky.  In my super-repo I almost never want to\ncheckout the master HEAD of all my submodules.  In fact, many of my\nsubmodules are big (e.g. they're different Linux kernels), and are only\nneeded when building particular things, so that checking all of them out at\nonce is almost always a huge waste of time.\n\nAll this is probably not the kind of feedback you were hoping for!  :)\n\n\t\tM.\n"},{"id":"143244","messageId":"4C0E6A8A.70608@web.de","threadId":"24036","inReplyTo":"201006080912.31448.johan@herland.net","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-06-08T16:06:34Z","receivedAt":"2010-06-08T16:06:34Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 08.06.2010 09:12, schrieb Johan Herland:\n> - When switching branches in the superrepo, you sometimes also want to \n> switch branches in the submodule. This is signalled by changing the \n> submodules.subthing.branch variable in .gitmodules between the two branches. \n> However, it means that the submodule's update/pull operation must also be \n> done on 'checkout' in the superrepo.\n\nHm, I always want the submodules to switch branches along with the super-\nproject (I posted a RFC patch for that), but i can see other people don't\nwant that at all or just for some submodules. But am I wrong assuming that\nit's either \"switch branches in submodules too every time\" or \"never do\nthat\" for a single submodule?\n\n\n> - How to handle local/uncommitted (staged or unstaged) modifications in a \n> submodule when pulling or switching branches in the superrepo? The right \n> answer here is probably to do the same as in the no-submodule case, i.e. to \n> refuse if it would clobber/conflict with the local modifications.\n\nYup. I thing one goal for submodules is that they should blend in with\nthe superprojects as far as possible (unless configured to not to).\n\n\n> - When you track submodule branches instead of commits, the actual commit \n> referenced in the superrepo is no longer as important (provided it's part of \n> the ancestry of the submodule branch you're tracking). However, diff/status \n> will still list the submodule as changed because you checked out a different \n> commit from what Git has recorded. This raises two concerns: (1) What \n> _should_ be considered \"changed\" from the diff/status perspective when \n> tracking submodule branches? and (2) When do you update the commit reference \n> in the submodule? \"never\" would work (since you're checking out a different \n> commit anyway), \"always\" would also work (for the same reason), but would \n> litter the superrepo history with submodule updates. There may be a better \n> alternative somewhere in between.\n\nDon't record a commit in the first place, following a branch is not bound\nto a special commit, so pretending to do that might do more harm than good.\nJust putting the 0-hash there might be the solution.\n\n\n> - If you want to give the illusion of \"one big repo\" then maybe it should \n> also be possible to trigger submodule commits from a superrepo commit? (i.e. \n> having a single toplevel \"git commit\" also trigger commits in submodules). \n> Some users will want to specify the commit message for each submodule \n> separately (IMHO the better approach), while some will want to give only one \n> commit message that is reused in every submodule commit.\n\nHm, personally I am fine with first committing in the submodules and then\nin the superproject.\n"},{"id":"143245","messageId":"AANLkTimtWrp1yimeooJ-ptAaDoxwpUc5KOP9HJUxx0X2@mail.gmail.com","threadId":"24036","inReplyTo":"4C0E630A.7020803@xiplink.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-08T16:09:20Z","receivedAt":"2010-06-08T16:09:20Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, Jun 8, 2010 at 15:34, Marc Branchaud <marcnarc@xiplink.com> wrote:\n> On 10-06-08 03:12 AM, Johan Herland wrote:\n>>\n>> There are probably more issues that escape me now...\n>\n> Thanks for bringing this up!  I'm also very interested in this topic.\n>\n> The main issue I see is that I don't always want my submodules to track (or\n> not track) a branch.  What I want changes depending on the circumstances.\n>\n> One aspect I really like about submodules is the ease of tagging.  I can tag\n> the super-repo, and know that whenever I checkout that tag I'll always get\n> the corresponding versions of the submodules as they were when the tag was\n> made.  It would actually be disastrous, at least in my case, if the\n> submodules were at the latest HEAD of some branch instead.\n>\n> This goes for almost any commit in the super-repo's history, not just the\n> tagged ones:  Whenever I checkout a historical committish, I want to get the\n> submodules as they were when that commit was made.  Even if I'm working at\n> the HEAD of some branch, often that branch is based on a historical commit\n> and I want to use the submodules as they were when that historical commit was\n> made.\n>\n> All that said, I do think submodule branch tracking is useful.  Quite often a\n> development topic will change the super-repo and one or more submodules.  It\n> would be extremely helpful to do that work in a branch that spans the\n> super-repo and (a subset of) the submodules.  (In my mind this capability is\n> one of the main benefits Google's \"repo\" tool has over submodules.)\n>\n> So, back to the issue at hand: Sometimes I want static (non-tracking)\n> submodules, and sometimes I want dynamic (tracking) submodules.  IMO, this\n> makes Ævar's proposed configuration-based approach impractical.  (Of course,\n> I'm not looking to replicate svn's externals...)\n\nI'm proposing that you be able to configure how you want to handle\nsubmodules on a per-submodule basis.\n\nThe exact semantics that I proposed may be impractical for some\nreason, but the idea is that it'd be opt in. We'd perhaps have\nmultiple approaches (via config) to submodules, instead of the current\nmonolithic scheme.\n\nSo if you didn't want a svn:externals like \"always track trunk\"\nrepository you'd just not set your superproject up to treat the\nsubmodule like that.\n\n> I'm not sure what the right approach is, but I have some thoughts:\n>\n>  - Maybe \"git branch\" should be able to create submodule-spanning branches.\n>\n>  - If so, then checkout, merge, pull and other branch-related commands should\n> honor submodule-spanning branches.  \"checkout\" in particular needs to\n> distinguish between when it's checking out an actual branch vs. some other\n> committish, and if the branch being checked out is submodule-spanning it\n> should checkout the latest HEAD of that branch for the submodules as well.\n>\n>  - It *may* be good enough to assume that matching branch names in the\n> super-repo and the submodules are in fact submodule-spanning branches.\n\nThat won't work for submodules that you don't control. I have a\nrepository that includes a lot of foreign code, they have a lot of\ndifferent names for their \"main branch\" between them. So it needs to\nbe configurable in the superproject.\n\n>  - Automating all this is tricky.  In my super-repo I almost never want to\n> checkout the master HEAD of all my submodules.  In fact, many of my\n> submodules are big (e.g. they're different Linux kernels), and are only\n> needed when building particular things, so that checking all of them out at\n> once is almost always a huge waste of time.\n>\n> All this is probably not the kind of feedback you were hoping for!  :)\n"},{"id":"143266","messageId":"4C0E9AC7.7080802@xiplink.com","threadId":"24036","inReplyTo":"AANLkTimtWrp1yimeooJ-ptAaDoxwpUc5KOP9HJUxx0X2@mail.gmail.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2010-06-08T19:32:23Z","receivedAt":"2010-06-08T19:32:23Z","isPatch":false,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 10-06-08 12:09 PM, Ævar Arnfjörð Bjarmason wrote:\n> On Tue, Jun 8, 2010 at 15:34, Marc Branchaud <marcnarc@xiplink.com> wrote:\n>>\n>> So, back to the issue at hand: Sometimes I want static (non-tracking)\n>> submodules, and sometimes I want dynamic (tracking) submodules.  IMO, this\n>> makes Ævar's proposed configuration-based approach impractical.  (Of course,\n>> I'm not looking to replicate svn's externals...)\n> \n> I'm proposing that you be able to configure how you want to handle\n> submodules on a per-submodule basis.\n\nYes, and that's precisely the problem.  For a given submodule, sometimes it\nshould track a branch and sometimes it shouldn't.  Having to edit a\nconfiguration to change that is impractical.\n\n> The exact semantics that I proposed may be impractical for some\n> reason, but the idea is that it'd be opt in. We'd perhaps have\n> multiple approaches (via config) to submodules, instead of the current\n> monolithic scheme.\n\nOpting in or out can't just be a monolithic setting for each submodule.  A\nsubmodule's branch tracking has to be on or off depending on the circumstances.\n\n> So if you didn't want a svn:externals like \"always track trunk\"\n> repository you'd just not set your superproject up to treat the\n> submodule like that.\n\nYes, of course.\n\nI guess what I'm saying is that duplicating svn's externals doesn't seem all\nthat useful to me and I'd rather see git do better.  I've no objection if\nfolks want to have such a feature, but to me it's not what \"submodules\ntracking branches\" should be about.\n\n>>  - It *may* be good enough to assume that matching branch names in the\n>> super-repo and the submodules are in fact submodule-spanning branches.\n> \n> That won't work for submodules that you don't control. I have a\n> repository that includes a lot of foreign code, they have a lot of\n> different names for their \"main branch\" between them. So it needs to\n> be configurable in the superproject.\n\nGood point.  I agree.\n\n\t\tM.\n"},{"id":"143270","messageId":"AANLkTilYHfDrtCAcPPxB1AZnzch2ELTEiIFTW3N5LBEc@mail.gmail.com","threadId":"24036","inReplyTo":"4C0E9AC7.7080802@xiplink.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-08T20:23:49Z","receivedAt":"2010-06-08T20:23:49Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, Jun 8, 2010 at 19:32, Marc Branchaud <marcnarc@xiplink.com> wrote:\n> On 10-06-08 12:09 PM, Ævar Arnfjörð Bjarmason wrote:\n>> On Tue, Jun 8, 2010 at 15:34, Marc Branchaud <marcnarc@xiplink.com> wrote:\n>>>\n>>> So, back to the issue at hand: Sometimes I want static (non-tracking)\n>>> submodules, and sometimes I want dynamic (tracking) submodules.  IMO, this\n>>> makes Ævar's proposed configuration-based approach impractical.  (Of course,\n>>> I'm not looking to replicate svn's externals...)\n>>\n>> I'm proposing that you be able to configure how you want to handle\n>> submodules on a per-submodule basis.\n>\n> Yes, and that's precisely the problem.  For a given submodule, sometimes it\n> should track a branch and sometimes it shouldn't.  Having to edit a\n> configuration to change that is impractical.\n\nSee below.\n\n>> The exact semantics that I proposed may be impractical for some\n>> reason, but the idea is that it'd be opt in. We'd perhaps have\n>> multiple approaches (via config) to submodules, instead of the current\n>> monolithic scheme.\n>\n> Opting in or out can't just be a monolithic setting for each submodule.  A\n> submodule's branch tracking has to be on or off depending on the circumstances.\n\nI don't really get what the objection is exactly. How should \"branch\ntracking\" be achieved do you think?\n\nAnyway, right now what we track is set in a monolithic fashion by a\ncombination of a commit pointer in a tree and what's being versioned\nin .gitmodules. If you want to change anything that's where you have\nto do it.\n\nWhy should it be any different when the submodule isn't tracking a\nspecific commit? I.e. when it's \"track the latest version of $thingy,\nwhatever that is\", instead of \"track version $version of $thingy\".\n\n>> So if you didn't want a svn:externals like \"always track trunk\"\n>> repository you'd just not set your superproject up to treat the\n>> submodule like that.\n>\n> Yes, of course.\n>\n> I guess what I'm saying is that duplicating svn's externals doesn't seem all\n> that useful to me and I'd rather see git do better.  I've no objection if\n> folks want to have such a feature, but to me it's not what \"submodules\n> tracking branches\" should be about.\n\nObviously I have no objection to doing better, but how specifically\nshould that be done? If the semantics you want are \"give me the latest\nversion of $URL, whatever that is\" then the SVN semantics are pretty\ngood.\n"},{"id":"143287","messageId":"201006082352.38136.johan@herland.net","threadId":"24036","inReplyTo":"4C0E6A8A.70608@web.de","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2010-06-08T21:52:37Z","receivedAt":"2010-06-08T21:52:37Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Tuesday 08 June 2010, Jens Lehmann wrote:\n> Am 08.06.2010 09:12, schrieb Johan Herland:\n> > - When switching branches in the superrepo, you sometimes also want to\n> > switch branches in the submodule. This is signalled by changing the\n> > submodules.subthing.branch variable in .gitmodules between the two\n> > branches. However, it means that the submodule's update/pull operation\n> > must also be done on 'checkout' in the superrepo.\n> \n> Hm, I always want the submodules to switch branches along with the super-\n> project (I posted a RFC patch for that), but i can see other people don't\n> want that at all or just for some submodules. But am I wrong assuming\n> that it's either \"switch branches in submodules too every time\" or\n> \"never do that\" for a single submodule?\n\nWell, the good thing is that keeping this config/info in .gitmodules (which \nis versioned-controlled along with the rest of the project) enables you to \nchoose one or the other, or anything in between. For example, given a \nsubmodule \"foo\", say I want to keep it on its \"master\" branch when the \nsuper-repo is on _its_ master branch. The .gitmodules file on the super-\nrepo's \"master\" branch would contain:\n\n    [submodule \"foo\"]\n        path = foo\n        url = git://url/to/foo/upstream\n        branch = master\n\nNow, if I create a new branch \"topic\" in the super-repo, the submodule would \nby default keep on tracking its \"master\" branch. If I want to track another \nbranch, called \"subtopic\", inside \"foo\", I change .gitmodules on my super-\nrepo's \"topic\" branch to say:\n\n    [submodule \"foo\"]\n        path = foo\n        url = git://url/to/foo/upstream\n        branch = subtopic\n\nFinally, if I have a branch \"release\" in the super-repo, in which I want to \npin submodule \"foo\" to a specific commit, I change .gitmodules on the super-\nrepo's \"release\" branch to say:\n\n    [submodule \"foo\"]\n        path = foo\n        url = git://url/to/foo/upstream\n\n(i.e. no branch tracking), and then record the appropriate submodule commit \nthe \"old-fashioned\" way.\n\nThe good thing with Ævar's approach is that this is all configurable per \nbranch (indeed, per commit[1]) by editing your .gitmodules file.\n\n> > - When you track submodule branches instead of commits, the actual\n> > commit referenced in the superrepo is no longer as important (provided\n> > it's part of the ancestry of the submodule branch you're tracking).\n> > However, diff/status will still list the submodule as changed because\n> > you checked out a different commit from what Git has recorded. This\n> > raises two concerns: (1) What _should_ be considered \"changed\" from\n> > the diff/status perspective when tracking submodule branches? and (2)\n> > When do you update the commit reference in the submodule? \"never\"\n> > would work (since you're checking out a different commit anyway),\n> > \"always\" would also work (for the same reason), but would litter the\n> > superrepo history with submodule updates. There may be a better\n> > alternative somewhere in between.\n> \n> Don't record a commit in the first place, following a branch is not bound\n> to a special commit, so pretending to do that might do more harm than\n> good. Just putting the 0-hash there might be the solution.\n\nInteresting. Will the object parsing machinery handle that without hiccups? \nWhat if an older Git version tries to checkout/update a submodule with a 0-\nhash?\n\n> > - If you want to give the illusion of \"one big repo\" then maybe it\n> > should also be possible to trigger submodule commits from a superrepo\n> > commit? (i.e. having a single toplevel \"git commit\" also trigger\n> > commits in submodules). Some users will want to specify the commit\n> > message for each submodule separately (IMHO the better approach),\n> > while some will want to give only one commit message that is reused in\n> > every submodule commit.\n> \n> Hm, personally I am fine with first committing in the submodules and then\n> in the superproject.\n\nMe too, but I suspect that if you draw the \"one big repo\" approach to its \nlogical conclusion, there will be some demand for recursive commits.\n\n\nHave fun! :)\n\n...Johan\n\n\n[1]: Say your submodule usually tracks a branch, but you're creating some \ntag in the super-repo, and you want that tag to uniquely identify the \nsubmodule. You achieve this by making sure the tagged commit removes the \nrelevant \"branch = whatever\" line from .gitmodules, and records the \nappropriate submodule version in the super-repo tree. Then, you can revert \nthe .gitmodules change on the next commit to resume tracking the submodule \nbranch.\n\nNow, whenever you checkout the tag, you will always get the exact same \nversion of the submodule, although the submodule otherwise tracks some \nbranch.\n\n\n-- \nJohan Herland, <johan@herland.net>\nwww.herland.net\n"},{"id":"143293","messageId":"7vbpblruj8.fsf@alter.siamese.dyndns.org","threadId":"24036","inReplyTo":"4C0E6A8A.70608@web.de","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-06-08T23:09:31Z","receivedAt":"2010-06-08T23:09:31Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann <Jens.Lehmann@web.de> writes:\n\n> Don't record a commit in the first place, following a branch is not bound\n> to a special commit, so pretending to do that might do more harm than good.\n> Just putting the 0-hash there might be the solution.\n\nUgh.  Even though I understand that in some scenarios you would want to\nsay \"I don't care what commit is used for this submodule---just use the\ntip of the branch 'fred'\", I don't think you want to use 0{40} in the\nsuperproject.  I think it would be Ok to add such a note to .gitmodules in\nthe superproject, but I also think we should still record which _exact_\ncommit was used to test and validate such a commit in the superproject\nwhen it was made.\n\nIf you clone a superproject that contains such a submodule from an\nupstream, keeping them up-to-date while working on your own change, it is\nperfectly fine to choose to use whatever random commit that happens to be\nat the tip of 'fred' branch in a submodule (and needless to say, that\ncommit might be your own commit that nobody else has, if you have been\nactively working in that submodule, that you haven't published), that is\ndifferent from what the person who created the commit in the superproject\nhad.  But at least you would need to be able to tell that the result of a\nbuild from such a state is different from what the superproject had.\nRecording 0{40} would make the information contained in the superproject\ntree meaningless.\n\nWouldn't it be enough to say --ignore-submodules for your day-to-day work,\nwithout lying in the gitlink entry in the superproject tree?  An entry\n\"submodule.foo.branch = fred\" in your .gitmodules will still tell your\nlocal git to update the submodule worktree to work on 'fred' branch.  At\nleast, an arrangement like that would allow the build infrastructure to\nuse --no-ignore-submodules when running its equivalent of GIT-VERSION-GEN\nto notice that what you are building is using something different from\nwhat the superproject specified to use in the submodule, while not bugging\nyou with differences you do not care about (or you already know about and\nare irrelevant to the change you are working on).\n"},{"id":"143294","messageId":"AANLkTimApr6P0sQ0FQiNkzhFuftOu1e4VefQxUCXpA53@mail.gmail.com","threadId":"24036","inReplyTo":"7vbpblruj8.fsf@alter.siamese.dyndns.org","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-08T23:19:39Z","receivedAt":"2010-06-08T23:19:39Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, Jun 8, 2010 at 23:09, Junio C Hamano <gitster@pobox.com> wrote:\n> Wouldn't it be enough to say --ignore-submodules for your day-to-day work,\n> without lying in the gitlink entry in the superproject tree?  An entry\n> \"submodule.foo.branch = fred\" in your .gitmodules will still tell your\n> local git to update the submodule worktree to work on 'fred' branch.  At\n> least, an arrangement like that would allow the build infrastructure to\n> use --no-ignore-submodules when running its equivalent of GIT-VERSION-GEN\n> to notice that what you are building is using something different from\n> what the superproject specified to use in the submodule, while not bugging\n> you with differences you do not care about (or you already know about and\n> are irrelevant to the change you are working on).\n\nYes I think that's even better, to have no entry in the superproject's\ntree at all, and just a repo/branch pair in .gitmodules.\n\nLess confusion and the same features.\n"},{"id":"143308","messageId":"4C0F3E3C.3090007@web.de","threadId":"24036","inReplyTo":"AANLkTimApr6P0sQ0FQiNkzhFuftOu1e4VefQxUCXpA53@mail.gmail.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-06-09T07:09:48Z","receivedAt":"2010-06-09T07:09:48Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 09.06.2010 01:19, schrieb Ævar Arnfjörð Bjarmason:\n> On Tue, Jun 8, 2010 at 23:09, Junio C Hamano <gitster@pobox.com> wrote:\n>> Wouldn't it be enough to say --ignore-submodules for your day-to-day work,\n>> without lying in the gitlink entry in the superproject tree?  An entry\n>> \"submodule.foo.branch = fred\" in your .gitmodules will still tell your\n>> local git to update the submodule worktree to work on 'fred' branch.  At\n>> least, an arrangement like that would allow the build infrastructure to\n>> use --no-ignore-submodules when running its equivalent of GIT-VERSION-GEN\n>> to notice that what you are building is using something different from\n>> what the superproject specified to use in the submodule, while not bugging\n>> you with differences you do not care about (or you already know about and\n>> are irrelevant to the change you are working on).\n> \n> Yes I think that's even better, to have no entry in the superproject's\n> tree at all, and just a repo/branch pair in .gitmodules.\n\nThats not how I understood Junio proposal, but an alternative to using\n0{40} could be to just drop the submodule entry from the tree. You get the\nsame result, but maybe less problems with older versions of git.\n\n\n> Less confusion and the same features.\n\nNot knowing the version of a submodule looks to me like a very powerful\nsource of confusion, but maybe thats just me ;-)\n"},{"id":"143309","messageId":"4C0F3FA9.7000800@web.de","threadId":"24036","inReplyTo":"7vbpblruj8.fsf@alter.siamese.dyndns.org","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-06-09T07:15:53Z","receivedAt":"2010-06-09T07:15:53Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 09.06.2010 01:09, schrieb Junio C Hamano:\n> Jens Lehmann <Jens.Lehmann@web.de> writes:\n> \n>> Don't record a commit in the first place, following a branch is not bound\n>> to a special commit, so pretending to do that might do more harm than good.\n>> Just putting the 0-hash there might be the solution.\n> \n> Ugh.  Even though I understand that in some scenarios you would want to\n> say \"I don't care what commit is used for this submodule---just use the\n> tip of the branch 'fred'\", I don't think you want to use 0{40} in the\n> superproject.  I think it would be Ok to add such a note to .gitmodules in\n> the superproject, but I also think we should still record which _exact_\n> commit was used to test and validate such a commit in the superproject\n> when it was made.\n\nI think we are in violent agreement here. But I as far as understood the\nalways-tip mode (and I might be wrong here as I never used something like\nSVN Externals) it is intended to not be able to tell which exact version\nof the submodules branch was used. Otherwise you could just update the\nbranch in the submodule and commit that in the superproject, which is what\npeople do not seem to want (please correct me if I am wrong).\n\nUnder this assumption it seems to me that it doesn't make sense to record\nanything but 0{40}, as this tells people \"this submodule was somewhere at\nthe tip of <branch>, but we can't say where exactly\"). Or maybe don't add\nthe submodule to the tree at all, like Ævar proposed. Same outcome, maybe\neven easier to do.\n\nBut I always have the feeling there is something I don't get when people\ntalk about the always-tip mode, so maybe the potential users of such a\nfeature should speak up now.\n"},{"id":"143311","messageId":"4C0F4185.2000207@web.de","threadId":"24036","inReplyTo":"201006082352.38136.johan@herland.net","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-06-09T07:23:49Z","receivedAt":"2010-06-09T07:23:49Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 08.06.2010 23:52, schrieb Johan Herland:\n> The good thing with Ævar's approach is that this is all configurable per \n> branch (indeed, per commit[1]) by editing your .gitmodules file.\n\nYep, I think this is the sane way to do that.\n\n\n> Interesting. Will the object parsing machinery handle that without hiccups? \n> What if an older Git version tries to checkout/update a submodule with a 0-\n> hash?\n\nMaybe Ævar's idea of dropping such a submodule from the tree is better.\n\n\n> Me too, but I suspect that if you draw the \"one big repo\" approach to its \n> logical conclusion, there will be some demand for recursive commits.\n\nYou may be right here. But as submodules often have a detached HEAD, this\nmight get interesting ;-)\n\n\n> [1]: Say your submodule usually tracks a branch, but you're creating some \n> tag in the super-repo, and you want that tag to uniquely identify the \n> submodule. You achieve this by making sure the tagged commit removes the \n> relevant \"branch = whatever\" line from .gitmodules, and records the \n> appropriate submodule version in the super-repo tree. Then, you can revert \n> the .gitmodules change on the next commit to resume tracking the submodule \n> branch.\n> \n> Now, whenever you checkout the tag, you will always get the exact same \n> version of the submodule, although the submodule otherwise tracks some \n> branch.\n\nWon't work anymore when we would use 0{40} or drop it from the tree.\nAFAICS always-tip and referencing a certain commit don't mix well.\n"},{"id":"143315","messageId":"201006091022.18896.johan@herland.net","threadId":"24036","inReplyTo":"4C0F4185.2000207@web.de","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2010-06-09T08:22:18Z","receivedAt":"2010-06-09T08:22:18Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Wednesday 09 June 2010, Jens Lehmann wrote:\n> Am 08.06.2010 23:52, schrieb Johan Herland:\n> > The good thing with Ævar's approach is that this is all configurable\n> > per branch (indeed, per commit[1]) by editing your .gitmodules file.\n> \n> Yep, I think this is the sane way to do that.\n> \n> > Interesting. Will the object parsing machinery handle that without\n> > hiccups? What if an older Git version tries to checkout/update a\n> > submodule with a 0- hash?\n> \n> Maybe Ævar's idea of dropping such a submodule from the tree is better.\n\nAgreed. That will of course cause older Git versions to skip the submodule \naltogether, which is probably the safest failure mode.\n\n> > Me too, but I suspect that if you draw the \"one big repo\" approach to\n> > its logical conclusion, there will be some demand for recursive\n> > commits.\n> \n> You may be right here. But as submodules often have a detached HEAD, this\n> might get interesting ;-)\n\nYes, trying to recursively commit across a submodule with detached HEAD \nshould obviously fail (at least by default). But as long as a local branch \nis checked out in the submodule (which is not necessarily the same as having \nthe submodule _track_ that branch), a recursive commit should be relatively \nstraightforward.\n\n> > [1]: Say your submodule usually tracks a branch, but you're creating\n> > some tag in the super-repo, and you want that tag to uniquely identify\n> > the submodule. You achieve this by making sure the tagged commit\n> > removes the relevant \"branch = whatever\" line from .gitmodules, and\n> > records the appropriate submodule version in the super-repo tree.\n> > Then, you can revert the .gitmodules change on the next commit to\n> > resume tracking the submodule branch.\n> > \n> > Now, whenever you checkout the tag, you will always get the exact same\n> > version of the submodule, although the submodule otherwise tracks some\n> > branch.\n> \n> Won't work anymore when we would use 0{40} or drop it from the tree.\n> AFAICS always-tip and referencing a certain commit don't mix well.\n\nAFAICS, it would still work as long as it exists in the tree for that \nspecific commit (but is missing/0{40} in other commits).\n\nWe're not mixing \"always-tip\" and \"exact-commit\" in the same commit. We use \n\"always-tip\" in regular commits, and then temporarily switch to \"exact-\ncommit\" in the commits where a certain submodule version is required.\n\n\n...Johan\n\n-- \nJohan Herland, <johan@herland.net>\nwww.herland.net\n"},{"id":"143327","messageId":"C0EA2469-DA5B-413E-9AB4-F79954DBE3AE@gmail.com","threadId":"24036","inReplyTo":"201006091022.18896.johan@herland.net","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Steven Michalske","fromEmail":"smichalske@gmail.com","sentAt":"2010-06-09T12:47:04Z","receivedAt":"2010-06-09T12:47:04Z","isPatch":false,"sender":{"key":"smichalske@gmail.com","avatar":"https://gravatar.com/avatar/721f27456adc9ac84f3bb235f021a70015abb9e09222ae8622fc5579c6a203c1?d=mp&s=160"},"body":"\nOn Jun 9, 2010, at 1:22 AM, Johan Herland wrote:\n\n> On Wednesday 09 June 2010, Jens Lehmann wrote:\n>> Am 08.06.2010 23:52, schrieb Johan Herland:\n>>> The good thing with Ævar's approach is that this is all configurable\n>>> per branch (indeed, per commit[1]) by editing your .gitmodules file.\n>>\n>> Yep, I think this is the sane way to do that.\n>>\n>>> Interesting. Will the object parsing machinery handle that without\n>>> hiccups? What if an older Git version tries to checkout/update a\n>>> submodule with a 0- hash?\n>>\n>> Maybe Ævar's idea of dropping such a submodule from the tree is  \n>> better.\n>\n> Agreed. That will of course cause older Git versions to skip the  \n> submodule\n> altogether, which is probably the safest failure mode.\n>\n>>> Me too, but I suspect that if you draw the \"one big repo\" approach  \n>>> to\n>>> its logical conclusion, there will be some demand for recursive\n>>> commits.\n>>\n>> You may be right here. But as submodules often have a detached  \n>> HEAD, this\n>> might get interesting ;-)\n>\n> Yes, trying to recursively commit across a submodule with detached  \n> HEAD\n> should obviously fail (at least by default). But as long as a local  \n> branch\n> is checked out in the submodule (which is not necessarily the same  \n> as having\n> the submodule _track_ that branch), a recursive commit should be  \n> relatively\n> straightforward.\n>\n>>> [1]: Say your submodule usually tracks a branch, but you're creating\n>>> some tag in the super-repo, and you want that tag to uniquely  \n>>> identify\n>>> the submodule. You achieve this by making sure the tagged commit\n>>> removes the relevant \"branch = whatever\" line from .gitmodules, and\n>>> records the appropriate submodule version in the super-repo tree.\n>>> Then, you can revert the .gitmodules change on the next commit to\n>>> resume tracking the submodule branch.\n>>>\n>>> Now, whenever you checkout the tag, you will always get the exact  \n>>> same\n>>> version of the submodule, although the submodule otherwise tracks  \n>>> some\n>>> branch.\n>>\n>> Won't work anymore when we would use 0{40} or drop it from the tree.\n>> AFAICS always-tip and referencing a certain commit don't mix well.\n>\n> AFAICS, it would still work as long as it exists in the tree for that\n> specific commit (but is missing/0{40} in other commits).\n>\n> We're not mixing \"always-tip\" and \"exact-commit\" in the same commit.  \n> We use\n> \"always-tip\" in regular commits, and then temporarily switch to  \n> \"exact-\n> commit\" in the commits where a certain submodule version is required.\n>\nWhen making a tag, could the notes system be used for marking what  \ncommit was exactly on the submodule, perhaps include the closest  \nremote commits as well?\n\n\nSomething like\n\nSubmodule Status:\n\t[\"foo\"]\n\t\tbranch = subtopic:SHA\n\nThis assumes that git notes are shared/cloned......\n\n\nOther thoughts.\n\n\nThings that should still work with tracking submodules.\n\t- bisect   - Must be able to identify that a submodule change  \nintroduced the bug.\n\t- archive  - Should it use the version from the commit, or the latest?\n\t- rebase   - update all of the submodule commits?\n\t- checkout - tip vs commit\n\t- reset --hard - Good question... not sure.... probably depend on tip  \nvs commit like checkout.\n\t- More????\n\n\nI would rather the submodule entree in the tree be always updated to  \nwhat is in the submodule on the commit, so that the history is always  \nthere.  Then actions updating the repository from remotes  \nautomatically pull the latest version.  I feel that the submodule if  \nautomatically be pulled, merged, etc, than the submodule should get a  \ncommit, with the message about this being an automatic update of the  \nsubmodule.  Checking out is a different story.... checking out a  \nbranch tip of the super gets the latest tip from the submodule.  When  \nyou commit, the submodule gets it's auto commit, then a second commit  \nfor the code changes.  checking out a previous revision should  \nprobably put the sub module to the state it was in at that point in  \ntime.  Creating a branch and adding new content would update according  \nto the rules.  but show the change of the subproject as from the  \nsuper's at the branch point, not the tip.\n\nThis way older gits have a submodule to work with and newer gits will  \ndo the right thing.\n\nExample:\n\ns-y-y-z\nA-B-C-D\n\\\n  \\F-G\ns-z-z\n\nF is branched when the latest sub module is at z  but shows the change  \nfrom s not z because A the parent of F was created with the submodule  \nat s\n\nSituational Example:\n\nI am developing away and as I progress in development I get a  \nregression bug, so I run git bisect from the last stable release with  \nout this bug, and it starts bisecting away.\n\nIn the mode where we don't store the state of the project I can't  \nbisect the changes against the subproject, where my bug might have  \nbeen introduced from.\n\nSo that issue should be probably handled in the git bisect code, that  \nis \"Make git bisect submodule aware\"  in more verbose terms, when  \nbisecting a super project the sub modules should undergo bisection as  \nwell.  This is a permutation that will expand rapidly, but some  \nthoughts on how to dig into the bisection issues.\nThis is another email ;-)\n\n\nRebase:\n\tWith the auto commit of submodule scheme, a rebase would change the  \ntracking branches to the latest of the tracked version. and auto merge  \nand record the previous submodule revision in the commit message of  \nthe submodule auto commit.\n\nCheckout with nonexistant submodule sha:\n\tThis is the case where the submodules ref was not pushed publicly,  \nso, the contents are not available.  You get a nice warning and the  \ntip of the submodules branch gets checked out for that submodule.\n\nSteve\n"},{"id":"143342","messageId":"4C0FA6F2.8060308@xiplink.com","threadId":"24036","inReplyTo":"AANLkTilYHfDrtCAcPPxB1AZnzch2ELTEiIFTW3N5LBEc@mail.gmail.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2010-06-09T14:36:34Z","receivedAt":"2010-06-09T14:36:34Z","isPatch":false,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 10-06-08 04:23 PM, Ævar Arnfjörð Bjarmason wrote:\n> On Tue, Jun 8, 2010 at 19:32, Marc Branchaud <marcnarc@xiplink.com> wrote:\n>>\n>> Opting in or out can't just be a monolithic setting for each submodule.  A\n>> submodule's branch tracking has to be on or off depending on the circumstances.\n> \n> I don't really get what the objection is exactly. How should \"branch\n> tracking\" be achieved do you think?\n\nWell, I outlined some ideas in my first message in this thread...\n\n>> I guess what I'm saying is that duplicating svn's externals doesn't seem all\n>> that useful to me and I'd rather see git do better.  I've no objection if\n>> folks want to have such a feature, but to me it's not what \"submodules\n>> tracking branches\" should be about.\n> \n> Obviously I have no objection to doing better, but how specifically\n> should that be done? If the semantics you want are \"give me the latest\n> version of $URL, whatever that is\" then the SVN semantics are pretty\n> good.\n\nThe nuance is that the semantics aren't \"*always* give me the latest version\nof $URL\" but rather \"*sometimes* give me the latest version of $URL.\"\n\nAnyway, others have raised issues that touch on this, and I'm happy to just\nsee where those discussions go.\n\n\t\tM.\n"},{"id":"143343","messageId":"201006091637.43726.johan@herland.net","threadId":"24036","inReplyTo":"C0EA2469-DA5B-413E-9AB4-F79954DBE3AE@gmail.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Johan Herland","fromEmail":"johan@herland.net","sentAt":"2010-06-09T14:37:43Z","receivedAt":"2010-06-09T14:37:43Z","isPatch":false,"sender":{"key":"johan@herland.net","avatar":"https://avatars.githubusercontent.com/u/547031?v=4"},"body":"On Wednesday 09 June 2010, Steven Michalske wrote:\n> On Jun 9, 2010, at 1:22 AM, Johan Herland wrote:\n> > On Wednesday 09 June 2010, Jens Lehmann wrote:\n> >> Am 08.06.2010 23:52, schrieb Johan Herland:\n> >>> [1]: Say your submodule usually tracks a branch, but you're\n> >>> creating some tag in the super-repo, and you want that tag to\n> >>> uniquely identify\n> >>> the submodule. You achieve this by making sure the tagged commit\n> >>> removes the relevant \"branch = whatever\" line from .gitmodules,\n> >>> and records the appropriate submodule version in the super-repo\n> >>> tree. Then, you can revert the .gitmodules change on the next\n> >>> commit to resume tracking the submodule branch.\n> >>>\n> >>> Now, whenever you checkout the tag, you will always get the exact\n> >>> same\n> >>> version of the submodule, although the submodule otherwise tracks\n> >>> some\n> >>> branch.\n> >>\n> >> Won't work anymore when we would use 0{40} or drop it from the\n> >> tree. AFAICS always-tip and referencing a certain commit don't mix\n> >> well.\n> >\n> > AFAICS, it would still work as long as it exists in the tree for\n> > that specific commit (but is missing/0{40} in other commits).\n> >\n> > We're not mixing \"always-tip\" and \"exact-commit\" in the same\n> > commit. We use \"always-tip\" in regular commits, and then temporarily\n> > switch to \"exact-commit\" in the commits where a certain submodule\n> > version is required.\n>\n> When making a tag, could the notes system be used for marking what\n> commit was exactly on the submodule, perhaps include the closest\n> remote commits as well?\n>\n>\n> Something like\n>\n> Submodule Status:\n> \t[\"foo\"]\n> \t\tbranch = subtopic:SHA\n>\n> This assumes that git notes are shared/cloned......\n\nI don't think this is a good use of notes. We already have an \ninfrastructure for recording the exact submodule version used (the \nsubmodule entry in the superproject tree), and I see no reason why we \nshould not use that in this case.\n\n> Other thoughts.\n>\n>\n> Things that should still work with tracking submodules.\n> \t- bisect   - Must be able to identify that a submodule change\n> introduced the bug.\n\nWhen tracking submodule commits (the default), this is just a matter of \nrecursively applying the bisect operation to the good/bad submodule \ncommits identified by the superproject bisect.\n\nWhen tracking submodule branches, you can no longer use bisect in the \nsuperproject to find bugs in the submodule, since it would always check \nout the same (i.e. latest) version of the submodule (I'm assuming that \nwe're not switching submodule branches in .gitmodules here...).\n\nIn the tag scenario I present which is quoted above, we're switching \nbetween tracking submodule commits (in the commits that are tagged), \nand tracking submodule branches (in the other commits), so bisect would \nbe next to useless (unless you limit the bisect to only look at the \ntagged commits).\n\nAlternatively, we could reconsider the question I asked Ævar initially \nin this thread:\n\n<quote>\nWhen do you update the commit reference in the submodule? \"never\" would \nwork (since you're checking out a different commit anyway), \"always\" \nwould also work (for the same reason), but would litter the superrepo \nhistory with submodule updates. There may be a better alternative \nsomewhere in between.\n</quote>\n\nSo if we _always_ (or often) update the submodule commit reference in \nthe superproject, then we could disregard the branch tracking while \nbisecting and checkout the commit referenced from the super-repo. That \nwould hopefully be as useful as if we'd tracked submodule commits \nexplicitly.\n\nBut again, the more often we update the submodule commit reference \n(while still primarily tracking a _branch_ in the submodule), the more \nwe \"litter\" the superproject history with these updates.\n\n> \t- archive  - Should it use the version from the commit, or the\n> latest?\n\nGood question. In principle, since you've explicitly asked to track a \nbranch, the only assumption Git can make is that you really want the \n_latest_ version of that subdmodule/branch.\n\nAgain we're back to the question of if/how often we record update \nsubmodule commits in the superproject, when the submodule primarily \ntracks a branch: If we're religious about recording our submodule \ncommit references in the superproject, then we can temporarily \ndisregard the branch tracking in order to get a somewhat realistic \nsubmodule update history.\n\n>       - rebase   - update all of the submodule commits? \n\nAgain, depends. If you really want branch-tracking, rebase will not \nchange the submodule (unless you change which branch is tracked \nin .gitmodules).\n\nHere, I don't see a good rationale for updating submodules though. If \nyou're tracking submodule branches, then a superproject rebase won't \naffect what's at the tip of a submodule branch.\n\n...unless you're talking about rebasing \"foo\" onto \"bar in the \nsuperproject causing a corresponding rebase of \"subfoo\" onto \"subbar\" \nin the submodule, which is a whole 'nother can of worms...\n\n> \t- checkout - tip vs commit\n\nNo question. If you've specified branch-tracking in .gitmodules, you get \ntip, otherwise you get commit.\n\n> \t- reset --hard - Good question... not sure.... probably depend on\n> tip vs commit like checkout.\n\nAs above, a reset --hard in the superproject does not affect what's at \nthe tip of some submodule branch, so if you've chosen to track \nsubmodule branches, a reset --hard will not touch the submodule (unless \nthe reset changes .gitmodules, obviously)\n\n> \t- More????\n>\n> I would rather the submodule entree in the tree be always updated to\n> what is in the submodule on the commit, so that the history is always\n> there.\n\nI see your point, as this would enable you to temporarily disable \nbranch-tracking after the fact (typically for debugging purposes). But \nwe still have to weigh its usefulness (in practice) against the cost of \nadding these extra commit references.\n\nAfter all, even if Git does not do this automatically, you could still \nfairly easily add a pre-commit hook in the superproject that stages all \nsubmodule references.\n\n> Then actions updating the repository from remotes \n> automatically pull the latest version.  I feel that the submodule if\n> automatically be pulled, merged, etc, than the submodule should get a\n> commit, with the message about this being an automatic update of the\n> submodule.  Checking out is a different story.... checking out a\n> branch tip of the super gets the latest tip from the submodule.  When\n> you commit, the submodule gets it's auto commit, then a second commit\n> for the code changes.  checking out a previous revision should\n> probably put the sub module to the state it was in at that point in\n> time.  Creating a branch and adding new content would update\n> according to the rules.  but show the change of the subproject as\n> from the super's at the branch point, not the tip.\n>\n> This way older gits have a submodule to work with and newer gits will\n> do the right thing.\n>\n> Example:\n>\n> s-y-y-z\n> A-B-C-D\n> \\\n>   \\F-G\n> s-z-z\n>\n> F is branched when the latest sub module is at z  but shows the\n> change from s not z because A the parent of F was created with the\n> submodule at s\n>\n> Situational Example:\n>\n> I am developing away and as I progress in development I get a\n> regression bug, so I run git bisect from the last stable release with\n> out this bug, and it starts bisecting away.\n>\n> In the mode where we don't store the state of the project I can't\n> bisect the changes against the subproject, where my bug might have\n> been introduced from.\n\nNo, you would have to run a separate bisect in the subproject. What's \nwrong about that?\n\n> So that issue should be probably handled in the git bisect code, that\n> is \"Make git bisect submodule aware\"  in more verbose terms, when\n> bisecting a super project the sub modules should undergo bisection as\n> well.  This is a permutation that will expand rapidly, but some\n> thoughts on how to dig into the bisection issues.\n> This is another email ;-)\n>\n>\n> Rebase:\n> \tWith the auto commit of submodule scheme, a rebase would change the\n> tracking branches to the latest of the tracked version. and auto\n> merge and record the previous submodule revision in the commit\n> message of the submodule auto commit.\n\nNot sure I understand what you're getting at here. Say you're tracking \nsubmodule branches (in the superproject's .gitmodules). Now, you do a \nrebase in the superproject. If the rebase does not change .gitmodules \n(i.e. which branch is tracked in the submodule), then there is nothing \nto be done in the submodule (it has already checked out the tip of that \nbranch). If the rebase _does_ change .gitmodules - let's say there's a \nconflict in which submodule branch to track - then you first resolve \nthat conflict in the superproject, to track the appropriate submodule \nbranch. Then the tip of that branch is simply checked out in the \nsubmodule. You may want to record this update in the superproject, \neither by recording a separate commit, or by amending into the rebased \ncommit.\n\nNow, it might be that the correct resolution of the superproject's \nrebase conflict requires a nested rebase in the submodule, but this is \ncertainly not a conclusion that Git can reach independently, so you \nwill have to do the nested rebase manually, and then record the rebased \nbranch in the superproject's .gitmodules (and optionally commit the \nupdated submodule reference).\n\n> Checkout with nonexistant submodule sha:\n> \tThis is the case where the submodules ref was not pushed publicly,\n> so, the contents are not available.  You get a nice warning and the\n> tip of the submodules branch gets checked out for that submodule.\n\nIf the .gitmodules specifies branch \"foo\" to be checked out in the \nsubmodule, and branch \"foo\" does not exist in the submodule, then \nthat's the same type of error as if the superproject specifies a commit \nSHA1 for the submodule, and that commit does not exist in the \nsubmodule.\n\nGranted, in the branch case, we can make it more network/remote-aware, \nby checking for both \"foo\" and \"origin/foo\" before giving up.\n\n\n...Johan\n\n-- \nJohan Herland, <johan@herland.net>\nwww.herland.net\n"},{"id":"143351","messageId":"4C0FB50F.3020403@xiplink.com","threadId":"24036","inReplyTo":"4C0F3FA9.7000800@web.de","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2010-06-09T15:36:47Z","receivedAt":"2010-06-09T15:36:47Z","isPatch":false,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 10-06-09 03:15 AM, Jens Lehmann wrote:\n> Am 09.06.2010 01:09, schrieb Junio C Hamano:\n>> Jens Lehmann <Jens.Lehmann@web.de> writes:\n>>\n>>> Don't record a commit in the first place, following a branch is not bound\n>>> to a special commit, so pretending to do that might do more harm than good.\n>>> Just putting the 0-hash there might be the solution.\n>>\n>> Ugh.  Even though I understand that in some scenarios you would want to\n>> say \"I don't care what commit is used for this submodule---just use the\n>> tip of the branch 'fred'\", I don't think you want to use 0{40} in the\n>> superproject.  I think it would be Ok to add such a note to .gitmodules in\n>> the superproject, but I also think we should still record which _exact_\n>> commit was used to test and validate such a commit in the superproject\n>> when it was made.\n> \n> I think we are in violent agreement here.\n\nI too am in this camp.\n\nIf a submodule is tracking the tip of a branch, I think it's vital that\nchecking out the superproject's HEAD@{3 months ago} gives you the submodule\nas it was in the superproject 3 months ago.  Back then, it may have been\ntracking a different branch.  It may not have been tracking a branch at all.\n It may have been using a completely different repository altogether.\n\nIt's hard for me to see the utility of having the submodule reflect the\ntip-of-some-branch-as-of-today when I'm looking at 3-month-old code in the\nsuperproject.\n\nAFAICT, Ævar's original proposal does the right thing here, because a\nsubmodule tracking a branch would look dirty in the superproject if the\nbranch's HEAD doesn't match the commit ID recorded in the superproject.  So\n\"submodule update\" would restore the submodule's state to what the\nsuperproject says it should be.\n\nI don't think I mind dirty branch-tracking submodules, but folks seem to find\nit distasteful.  However, I believe all the proposals made so far to address\nit break what I call the superproject's \"historical consistency.\"\n\nI wish I could come up with some way to reconcile clean branch-tracking\nsubmodules with historical consistency, but alas my imagination is so far too\nlimited.  :(\n\n\t\tM.\n"},{"id":"143366","messageId":"AANLkTiknGcteTjrHWM02H2KOMMDPZKHY1w0ZOIswddFn@mail.gmail.com","threadId":"24036","inReplyTo":"4C0FB50F.3020403@xiplink.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-09T18:54:34Z","receivedAt":"2010-06-09T18:54:34Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Wed, Jun 9, 2010 at 15:36, Marc Branchaud <marcnarc@xiplink.com> wrote:\n> I wish I could come up with some way to reconcile clean branch-tracking\n> submodules with historical consistency, but alas my imagination is so far too\n> limited.  :(\n\nI think the two concepts are fundimentally at odds with each other,\nand that that's completely fine.\n\nSometimes you're promiscuous enough with your history that you don't\ncare about being able to go back in time, beyond checking out both\ntrees as they were at some given time that is. As Johan and others\npoint out above you could get around that with tags if you wanted\nsnapshots.\n\nI think we might actually have several different modes of operation:\n\n  * Disconnected head + commit sha1 in the superproject's tree: This\n   is what we have now.\n\n  * The same, but make it branch aware. I've scripted this locally\n    with the $toplevel patch to git-submodule that started this\n    thread. But it could be expanded.\n\n    It would be really neat for example to do:\n\n        # Or some shorter way of doing this, perhaps even with\n        # git-pull\n        git submodule foreach 'git fetch'\n\n        # Tells you that \"submodule xyz which you've pinned to SHA1SUM\n        # on the FOOBAR branch is 20 commits behind the upstream\n        # FOOBAR branch\"\n        git status --submodules\n\n    You'd still have to take action to update the module and move the\n    SHA1SUM in the parent project, but something like this would make\n    cases where you've e.g. included a lot of plugins in your project,\n    and would like Git to tell you if they get new updates.\n\n  * Branch-only: What I proposed in this thread. It's certainly not\n    for everyone, but there's a lot of cases where you just want a\n    quick meta-repository but aren't very interested in 100%\n    historical consistency.\n\n  * More? Actually if we're doing multiple strategies I see no reason\n    not to e.g. include a foreign scm interface. That would be really\n    useful to some projects that are in a SVN -> Git transition:\n\n       [submodule \"svn-lib\"]\n           ;; type defaults to git\n           type = svn\n           path = src/svn-lib\n           url = svn://example.net/path/to/include\n           ;; driver-specific attributes\n           svn:revision = r54238\n"},{"id":"203550","messageId":"1353410195055-7571610.post@n2.nabble.com","threadId":"24036","inReplyTo":"AANLkTiknGcteTjrHWM02H2KOMMDPZKHY1w0ZOIswddFn@mail.gmail.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"nottrobin","fromEmail":"robin@robinwinslow.co.uk","sentAt":"2012-11-20T11:16:35Z","receivedAt":"2012-11-20T11:16:35Z","isPatch":false,"sender":{"key":"robin@robinwinslow.co.uk","avatar":null},"body":"Did any of this ever find its way into the submodule core? I'd like to have a\nsubmodule that tracks a branch.\n\n\n\n--\nView this message in context: http://git.661346.n2.nabble.com/RFC-Making-submodules-track-branches-tp5151566p7571610.html\nSent from the git mailing list archive at Nabble.com.\n"},{"id":"203555","messageId":"20121120120437.GB7096@odin.tremily.us","threadId":"24036","inReplyTo":"1353410195055-7571610.post@n2.nabble.com","subject":"Re: RFC: Making submodules \"track\" branches","fromName":"W. Trevor King","fromEmail":"wking@tremily.us","sentAt":"2012-11-20T12:04:37Z","receivedAt":"2012-11-20T12:04:37Z","isPatch":false,"sender":{"key":"wking@tremily.us","avatar":"https://avatars.githubusercontent.com/u/209920?v=4"},"body":"On Tue, Nov 20, 2012 at 03:16:35AM -0800, nottrobin wrote:\n> Did any of this ever find its way into the submodule core? I'd like\n> to have a submodule that tracks a branch.\n\nIn progress.  See:\n\n  http://thread.gmane.org/gmane.comp.version-control.git/208254\n\nCheers,\nTrevor\n\n-- \nThis email may be signed or encrypted with GnuPG (http://www.gnupg.org).\nFor more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\n"}]}