{"thread":{"id":"43929","subject":"Feature Request: Branch-Aware Submodules","startedAt":"2016-08-25T09:18:12Z","lastAt":"2016-08-25T21:31:34Z","messageCount":6,"participants":["Hedges Alexander","Stefan Beller","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"300135","messageId":"7B5AD16B-9729-453E-839C-2A2FDB07F8E9@student.ethz.ch","threadId":"43929","inReplyTo":null,"subject":"Feature Request: Branch-Aware Submodules","fromName":"Hedges Alexander","fromEmail":"ahedges@student.ethz.ch","sentAt":"2016-08-25T09:00:47Z","receivedAt":"2016-08-25T09:18:12Z","isPatch":false,"sender":{"key":"ahedges@student.ethz.ch","avatar":null},"body":"Dear Git Developers,\n\nFirst of all, thanks for this great project, it has made my life a lot easier\nas a developer!\n\nI'm writing this email to suggest some improvements to git submodules. In my\neyes how git handles submodules could be improved to be more intuitive to a\nnovice and require less manual management.\n\n\nRight now updating a submodule in a topic branch and merging it into master\nwill not change the submodule index in master leading to at least two commit\nfor the same change (one in any active branch). This happened to me quite a few\ntimes. To a newcomer this behavior is confusing and it leads to unnecessary\ncommits.\n\n\nThe proposed change would be to have a submodule either ignored or tracked by\nthe .gitmodules file.\nIf it is ignored, as for instance after a clone of the superproject, git simply\nignores all files in the submodule directory. The content of the gitmodules\nfile is then also not updated by git.\nIf it is not ignored, the .gitmodules is updated every time a commit happens in\nthe submodule. On branch switches the revision shown in the gitmodules from\nthat branch is checked out.\nThis change would have submodules conceptually behave more like files to the\nsuperproject.\n\n\nLike current behavior, git status would display whether the submodule has\nuncommitted changes or is at a new commit. A repository is in a dirty state if\nthere are changes to the gitmodules file or any tracked submodule is in a dirty\nstate. Every time a commit happens in a submodule, the parents gitmodules is\nupdated. Uncommitted changes are not reflected in the parent's gitmodules file.\n\nWhen the user manually edits the .gitmodules, git switches to that revision\nafter commit. But the user would have to stash or commit all uncommitted\nchanges in the submodule first.\n\nWhen checking out a commit in a submodule, if there is currently a branch\npointing to that commit, HEAD could point to that branch instead (Is there a\ncase where that doesn't make sense? What about multiple branches pointing to\nthe commit?). It could also support branch names as references where the branch\n(or tag) would be checked out instead.\n\nWith git submodule init you could have the submodule tracked. Using deinit\nwould put the submodule into the ignored state.\n\nAnd while we're at it, it is quite some work to completely delete a submodule.\nYou have to manually remove all the associated files in the git repository\n(StackOverflow lists 7 steps). Obviously it's not encouraged, as everything\nthat removes data without recovery method, but it should be possible.\ngit submodule rm --force could remove the repository and the associated nested\n.git tree. git submodule rm could keep the .git directory but move it to another\nlocation.\n\nThe behavior of git submodule sync and git submodule update would stay the same.\n\n\nMigrating existing repositories to the new behavior should be quite straight\nforward. Submodules that are not init'ed yet would be ignored. All others\nbehave accordingly to the new rules. Maybe a message with a note about the\nchanges could be displayed by the appropriate git-submodule commands or even by\ngit status.\n\n\nAn alternative considered was to have submodules decoupled stronger from the\nsuperproject. That would mean having the .gitmodules only tracked by master and\nleaving the other behaviors unchanged. For consistency one could do the same\nthing for the .gitignore.\n\nThe drawback of this option are obviously no per branch submodules, if you want\nto experiment with external libraries, topic branches would not be the place to\ngo. Also there would be a lot of intricacies that would have to be worked out.\n\n\nI couldn't find any discussions on the initial implementation of git-submodule\nor any previous proposals related to this in nature due to gmane being down\nright now and the mailing list archives on the other sites are not great for\nsearching. So please excuse me if I'm bringing up already discussed stuff.\n\nUntil now I only worked on projects with few submodules. I expect the\nproposed changes to have a larger effect on projects containing lots of\nsubmodules. So it would be nice if maybe somebody with experience working on\nprojects with lots of submodules could weigh into the discussion.\n\n\nBest Regards,\nAlexander Hedges\n\n\n\n"},{"id":"300176","messageId":"CAGZ79kbHfofmowHnwzFrumc1XJHE=ZMQEwNSeOSyF7YDG+aSjw@mail.gmail.com","threadId":"43929","inReplyTo":"7B5AD16B-9729-453E-839C-2A2FDB07F8E9@student.ethz.ch","subject":"Re: Feature Request: Branch-Aware Submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-08-25T17:45:59Z","receivedAt":"2016-08-25T17:47:30Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"+cc Jacob and Lars who work with submodules as well.\n\nOn Thu, Aug 25, 2016 at 2:00 AM, Hedges  Alexander\n<ahedges@student.ethz.ch> wrote:\n>\n> Right now updating a submodule in a topic branch and merging it into master\n> will not change the submodule index in master leading to at least two commit\n> for the same change (one in any active branch). This happened to me quite a few\n> times. To a newcomer this behavior is confusing and it leads to unnecessary\n> commits.\n\nSo you roughly do\n\n    git checkout -b new-topic\n    # change the submodule to point at the latest upstream version:\n    git submodule update --remote <submodule-path>\n    git commit -a -m \"update submodule\"\n    git checkout master\n    git merge new-topic\n    # here seems to be your point of critic?\n    # now the submodule pointer would still point to the latest\nupstream version?\n\n>\n>\n> The proposed change would be to have a submodule either ignored or tracked by\n> the .gitmodules file.\n> If it is ignored, as for instance after a clone of the superproject, git simply\n> ignores all files in the submodule directory. The content of the gitmodules\n> file is then also not updated by git.\n> If it is not ignored, the .gitmodules is updated every time a commit happens in\n> the submodule.\n\nSo\n\n    git -C <submodule-path> commit\n\nshould trigger a commit in the superproject as well, that changes the gitmodules\nfile? What do you record in the git modules file that needs updating?\nAs the version is tracked via the gitlink entry, I do not see the\ninformation that\nneeds tracking here?\n\n> On branch switches the revision shown in the gitmodules from\n> that branch is checked out.\n\nSo you are proposing to put the revision into the gitmodules file?\nThat would be redundant with the actual gitlink entry in your tree.\n(as shown via `git submodule status`)\nWhat would happen if the recorded revision in the gitmodules file and the\ngitlink are out of sync?\n\nOh, are you just proposing to actually make `git checkout` aware of the\nsubmodules? See[1]. I would welcome such a change and be happy th\n\n[1] https://github.com/jlehmann/git-submod-enhancements\nwhich has some attempts for checkout including the submodules.\nI also tried writing some patches which integrate checking out submodules\nvia checkout as well. A quicker `solution` would be a config option that\njust runs `git submodule update` after each checkout/pull etc.\n\n\n> This change would have submodules conceptually behave more like files to the\n> superproject.\n>\n>\n> Like current behavior, git status would display whether the submodule has\n> uncommitted changes or is at a new commit.\n\nSee config options diff.submodule and status.submoduleSummary.\n\n\n>\n> I couldn't find any discussions on the initial implementation of git-submodule\n> or any previous proposals related to this in nature due to gmane being down\n> right now and the mailing list archives on the other sites are not great for\n> searching. So please excuse me if I'm bringing up already discussed stuff.\n\nhttps://public-inbox.org/git for reading on the web, or\n\n    git clone https://public-inbox.org/git\n\nfor reading offline.\n\n>\n> Until now I only worked on projects with few submodules. I expect the\n> proposed changes to have a larger effect on projects containing lots of\n> submodules. So it would be nice if maybe somebody with experience working on\n> projects with lots of submodules could weigh into the discussion.\n>\n"},{"id":"300177","messageId":"xmqqvayon3md.fsf@gitster.mtv.corp.google.com","threadId":"43929","inReplyTo":"7B5AD16B-9729-453E-839C-2A2FDB07F8E9@student.ethz.ch","subject":"Re: Feature Request: Branch-Aware Submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-08-25T17:46:18Z","receivedAt":"2016-08-25T17:49:55Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Hedges  Alexander\" <ahedges@student.ethz.ch> writes:\n\n> Right now updating a submodule in a topic branch and merging it into master\n> will not change the submodule index in master leading to at least two commit\n> for the same change (one in any active branch).\n\nI stopped reading here because I am not getting this.\n\nI guess I am confused because I do not understand what you mean by\n\"the submodule index in master\".  The concept of \"index\" does not\nbelong to each branch (or even a commit), so by \"index\" you are\ntrying to point at something else, but I cannot guess what it is.\n\nYou have a top-level superproject that has another project as its\nsubmodule.  The superproject has topic and master branches (or it\nmay only have master).  The project that is used as its submodule\nalso has topic and master branches (it may have more).  You do your\ndevelopment in the submodule, e.g.\n\n    cd submoduledir\n    git checkout topic\n    hack hack hack\n    git commit\n    git checkout master\n    git merge topic\n\nand merge the topic branch into its master when the topic is\npolished enough.\n\nAnd then?  The 'master' in the submodule is good enough, so you'd\ngo back to the top-level superproject and bind that merged result \nin its place?  e.g.\n\n    cd ..\n    git add submoduledir\n    git commit -m \"Updated submoduledir with the topic\"\n\nThat is only one commit each in the superproject and the submodule\nproject.\n\n"},{"id":"300198","messageId":"xmqqlgzklgiy.fsf@gitster.mtv.corp.google.com","threadId":"43929","inReplyTo":"CAGZ79kbHfofmowHnwzFrumc1XJHE=ZMQEwNSeOSyF7YDG+aSjw@mail.gmail.com","subject":"Re: Feature Request: Branch-Aware Submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-08-25T20:50:29Z","receivedAt":"2016-08-25T20:51:35Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> +cc Jacob and Lars who work with submodules as well.\n>\n> On Thu, Aug 25, 2016 at 2:00 AM, Hedges  Alexander\n> <ahedges@student.ethz.ch> wrote:\n>>\n>> Right now updating a submodule in a topic branch and merging it into master\n>> will not change the submodule index in master leading to at least two commit\n>> for the same change (one in any active branch). This happened to me quite a few\n>> times. To a newcomer this behavior is confusing and it leads to unnecessary\n>> commits.\n>\n> So you roughly do\n>\n>     git checkout -b new-topic\n>     # change the submodule to point at the latest upstream version:\n>     git submodule update --remote <submodule-path>\n>     git commit -a -m \"update submodule\"\n>     git checkout master\n>     git merge new-topic\n>     # here seems to be your point of critic?\n>     # now the submodule pointer would still point to the latest\n> upstream version?\n\nIsn't <submodule-path> subject to the usual 3-way merge when the\nlast step (i.e. a merge of new-topic branch into master in the\nsuperproject) is made?  If 'master' hasn't changed <submodule-path>\nsince 'new-topic' forked from it, because 'new-topic' updated the\ncommit bound at <submodule-path>, doesn't \"git merge new-topic\" just\ntake that change as the normal \"One side updated, the other did not\ntouch; take the update\" merge?\n"},{"id":"300199","messageId":"CAGZ79ka60e2Y87k6oKtTN9e0ZexHx2GfJ7yfhvyQon_VgbUgNA@mail.gmail.com","threadId":"43929","inReplyTo":"xmqqlgzklgiy.fsf@gitster.mtv.corp.google.com","subject":"Re: Feature Request: Branch-Aware Submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-08-25T20:55:59Z","receivedAt":"2016-08-25T21:04:19Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Aug 25, 2016 at 1:50 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> +cc Jacob and Lars who work with submodules as well.\n>>\n>> On Thu, Aug 25, 2016 at 2:00 AM, Hedges  Alexander\n>> <ahedges@student.ethz.ch> wrote:\n>>>\n>>> Right now updating a submodule in a topic branch and merging it into master\n>>> will not change the submodule index in master leading to at least two commit\n>>> for the same change (one in any active branch). This happened to me quite a few\n>>> times. To a newcomer this behavior is confusing and it leads to unnecessary\n>>> commits.\n>>\n>> So you roughly do\n>>\n>>     git checkout -b new-topic\n>>     # change the submodule to point at the latest upstream version:\n>>     git submodule update --remote <submodule-path>\n>>     git commit -a -m \"update submodule\"\n>>     git checkout master\n>>     git merge new-topic\n>>     # here seems to be your point of critic?\n>>     # now the submodule pointer would still point to the latest\n>> upstream version?\n>\n> Isn't <submodule-path> subject to the usual 3-way merge when the\n> last step (i.e. a merge of new-topic branch into master in the\n> superproject) is made?  If 'master' hasn't changed <submodule-path>\n> since 'new-topic' forked from it, because 'new-topic' updated the\n> commit bound at <submodule-path>, doesn't \"git merge new-topic\" just\n> take that change as the normal \"One side updated, the other did not\n> touch; take the update\" merge?\n\nYes. I was unclear here.\nBy \"latest upstream version\" I meant the version you pulled in in the new-topic\nbranch via the \"submodule update --remote\" and that is preserved as is.\n"},{"id":"300201","messageId":"xmqqd1kwlewj.fsf@gitster.mtv.corp.google.com","threadId":"43929","inReplyTo":"CAGZ79ka60e2Y87k6oKtTN9e0ZexHx2GfJ7yfhvyQon_VgbUgNA@mail.gmail.com","subject":"Re: Feature Request: Branch-Aware Submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-08-25T21:25:32Z","receivedAt":"2016-08-25T21:31:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n>>> So you roughly do\n>>>\n>>>     git checkout -b new-topic\n>>>     # change the submodule to point at the latest upstream version:\n>>>     git submodule update --remote <submodule-path>\n>>>     git commit -a -m \"update submodule\"\n>>>     git checkout master\n>>>     git merge new-topic\n>>>     # here seems to be your point of critic?\n>>>     # now the submodule pointer would still point to the latest\n>>> upstream version?\n>>\n>> Isn't <submodule-path> subject to the usual 3-way merge when the\n>> last step (i.e. a merge of new-topic branch into master in the\n>> superproject) is made?  If 'master' hasn't changed <submodule-path>\n>> since 'new-topic' forked from it, because 'new-topic' updated the\n>> commit bound at <submodule-path>, doesn't \"git merge new-topic\" just\n>> take that change as the normal \"One side updated, the other did not\n>> touch; take the update\" merge?\n>\n> Yes. I was unclear here.\n> By \"latest upstream version\" I meant the version you pulled in in the new-topic\n> branch via the \"submodule update --remote\" and that is preserved as is.\n\nI do not think you were unclear at all.\n\nWhat else is desired?  \"git merge new-topic\" leaves a result that is\nnot a merge of the changes made on that new-topic branch, by leaving\na stale <submodule-path> that was in 'master' as-is?\n\nAfter all, the new-topic branch committed that \"update submodule\",\nshowing its desire that the latest-from-upstream commit it just\nobtained must be at <submodule-path> from then on in the top-level\nproject.  If that change is not propagated (or at least \"taken into\naccount\") when merging it to 'master', the result is not a proper\n\"merge\".  If new-topic didn't want the updated commit from the\nsubmodule, it shouldn't have recorded that in its commit in the\nfirst place.\n\n"}]}