{"thread":{"id":"39545","subject":"Submodules as first class citizens (was Re: Moving to subtrees for plugins?)","startedAt":"2015-06-06T17:49:14Z","lastAt":"2015-06-15T09:03:54Z","messageCount":7,"participants":["Phil Hord","Luca Milanesio","Stefan Beller","Jens Lehmann","Heiko Voigt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"263119","messageId":"CABURp0og9i9S3_ZWf5Ce9LT785QJo4H-TVtFaKUTXr2N7FB+ew@mail.gmail.com","threadId":"39545","inReplyTo":null,"subject":"Submodules as first class citizens (was Re: Moving to subtrees for plugins?)","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2015-06-06T17:49:14Z","receivedAt":"2015-06-06T17:49:14Z","isPatch":false,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Fri, Jun 5, 2015, 2:58 AM lucamilanesio wrote:\n>>\n>> Some devs of my Team complained that with submodules it is\n>> difficult to see the “full picture” of the difference\n>> between two SHA1 on the root project, as the submodules\n>> would just show as different SHA1s. When you Google\n>> “subtree submodules” you find other opinions as well:\n>>\n>> Just to mention a few:\n>> -\n>> https://codingkilledthecat.wordpress.com/2012/04/28/why-y\n>> our-company-shouldnt-use-git-submodules/ -\n>> http://blogs.atlassian.com/2013/05/alternatives-to-git-su\n>> bmodule-git-subtree/\n>>\n>> To be honest with you, I am absolutely fine with\n>> submodules as I can easily leave with the “extra pain” of\n>> diffing by hand recursively on submodules. But it is true\n>> that it may happen to either forget to do a git submodule\n>> update or otherwise forget you are in a detached branch\n>> and start committing “on the air” without a branch.\n\n...\n\n> Ideally, as a \"git clone --recursive\" already exists, I would like to\n> see a \"git diff --recursive\" that goes through the submodules as well :-)\n>\n> Something possibly to propose to the Git mailing list?\n\n\nI've worked on git diff --recursive a bit myself, along with some\nsimpler use cases (git ls-tree --recursive) as POCs. I think some of\nthe needs there begin to have ui implications which could be\nhigh-friction. I really want to finish it someday, but I've been too\nbusy lately at $job, and now my experiments are all rather stale.\n\nIt would be a good discussion to have over at the git list (copied).\nHeiko and Jens have laid some new groundwork in this area and it may\nbe a good time to revisit it. Or maybe they've even moved deeper than\nthat; I have been distracted for well over a year now.\n\nPhil\n\n-- \n-- \nTo unsubscribe, email repo-discuss+unsubscribe@googlegroups.com\nMore info at http://groups.google.com/group/repo-discuss?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Repo and Gerrit Discussion\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"263121","messageId":"D2BB8369-E552-4AC3-967E-8F963206E03C@gmail.com","threadId":"39545","inReplyTo":"CABURp0og9i9S3_ZWf5Ce9LT785QJo4H-TVtFaKUTXr2N7FB+ew@mail.gmail.com","subject":"Re: Submodules as first class citizens (was Re: Moving to subtrees for plugins?)","fromName":"Luca Milanesio","fromEmail":"luca.milanesio@gmail.com","sentAt":"2015-06-06T19:53:37Z","receivedAt":"2015-06-06T19:53:37Z","isPatch":false,"sender":{"key":"luca.milanesio@gmail.com","avatar":"https://gravatar.com/avatar/64e45570bb8baaba9a566592150e6c7341a300e314501a944cb95b0bc1380f49?d=mp&s=160"},"body":"Thank you Phil, you anticipated me :-)\n\nLuca.\n\n> On 6 Jun 2015, at 18:49, Phil Hord wrote:\n> \n> On Fri, Jun 5, 2015, 2:58 AM lucamilanesio wrote:\n>>> \n>>> Some devs of my Team complained that with submodules it is\n>>> difficult to see the “full picture” of the difference\n>>> between two SHA1 on the root project, as the submodules\n>>> would just show as different SHA1s. When you Google\n>>> “subtree submodules” you find other opinions as well:\n>>> \n>>> Just to mention a few:\n>>> -\n>>> https://codingkilledthecat.wordpress.com/2012/04/28/why-y\n>>> our-company-shouldnt-use-git-submodules/ -\n>>> http://blogs.atlassian.com/2013/05/alternatives-to-git-su\n>>> bmodule-git-subtree/\n>>> \n>>> To be honest with you, I am absolutely fine with\n>>> submodules as I can easily leave with the “extra pain” of\n>>> diffing by hand recursively on submodules. But it is true\n>>> that it may happen to either forget to do a git submodule\n>>> update or otherwise forget you are in a detached branch\n>>> and start committing “on the air” without a branch.\n> \n> ...\n> \n>> Ideally, as a \"git clone --recursive\" already exists, I would like to\n>> see a \"git diff --recursive\" that goes through the submodules as well :-)\n>> \n>> Something possibly to propose to the Git mailing list?\n> \n> \n> I've worked on git diff --recursive a bit myself, along with some\n> simpler use cases (git ls-tree --recursive) as POCs. I think some of\n> the needs there begin to have ui implications which could be\n> high-friction. I really want to finish it someday, but I've been too\n> busy lately at $job, and now my experiments are all rather stale.\n> \n> It would be a good discussion to have over at the git list (copied).\n> Heiko and Jens have laid some new groundwork in this area and it may\n> be a good time to revisit it. Or maybe they've even moved deeper than\n> that; I have been distracted for well over a year now.\n> \n> Phil\n"},{"id":"263138","messageId":"5573E40A.3020502@gmail.com","threadId":"39545","inReplyTo":"D2BB8369-E552-4AC3-967E-8F963206E03C@gmail.com","subject":"Re: Submodules as first class citizens (was Re: Moving to subtrees for plugins?)","fromName":"Stefan Beller","fromEmail":"stefanbeller@gmail.com","sentAt":"2015-06-07T06:26:18Z","receivedAt":"2015-06-07T06:26:18Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On 06.06.2015 12:53, Luca Milanesio wrote:\n> Thank you Phil, you anticipated me :-)\n> \n> Luca.\n> \n>> On 6 Jun 2015, at 18:49, Phil Hord wrote:\n>>\n>> On Fri, Jun 5, 2015, 2:58 AM lucamilanesio wrote:\n>>>>\n>>>> Some devs of my Team complained that with submodules it is\n>>>> difficult to see the “full picture” of the difference\n>>>> between two SHA1 on the root project, as the submodules\n>>>> would just show as different SHA1s. When you Google\n>>>> “subtree submodules” you find other opinions as well:\n>>>>\n>>>> Just to mention a few:\n>>>> -\n>>>> https://codingkilledthecat.wordpress.com/2012/04/28/why-y\n>>>> our-company-shouldnt-use-git-submodules/ -\n>>>> http://blogs.atlassian.com/2013/05/alternatives-to-git-su\n>>>> bmodule-git-subtree/\n>>>>\n>>>> To be honest with you, I am absolutely fine with\n>>>> submodules as I can easily leave with the “extra pain” of\n>>>> diffing by hand recursively on submodules. But it is true\n>>>> that it may happen to either forget to do a git submodule\n>>>> update or otherwise forget you are in a detached branch\n>>>> and start committing “on the air” without a branch.\n>>\n>> ...\n>>\n>>> Ideally, as a \"git clone --recursive\" already exists, I would like to\n>>> see a \"git diff --recursive\" that goes through the submodules as well :-)\n>>>\n>>> Something possibly to propose to the Git mailing list?\n>>\n>>\n>> I've worked on git diff --recursive a bit myself, along with some\n>> simpler use cases (git ls-tree --recursive) as POCs. I think some of\n>> the needs there begin to have ui implications which could be\n>> high-friction. I really want to finish it someday, but I've been too\n>> busy lately at $job, and now my experiments are all rather stale.\n>>\n>> It would be a good discussion to have over at the git list (copied).\n>> Heiko and Jens have laid some new groundwork in this area and it may\n>> be a good time to revisit it. Or maybe they've even moved deeper than\n>> that; I have been distracted for well over a year now.\n>>\n\nGlad you're working (or planning to) working on submodulues. This is\nalso on my todo list for the next months as well.\n\nI'd review stuff in that area if you're looking for reviewers.\n\nStefan\n\n>> Phil\n> \n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n> \n"},{"id":"263399","messageId":"5577330E.3060803@web.de","threadId":"39545","inReplyTo":"5573E40A.3020502@gmail.com","subject":"Re: Submodules as first class citizens (was Re: Moving to subtrees for plugins?)","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2015-06-09T18:40:14Z","receivedAt":"2015-06-09T18:40:14Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 07.06.2015 um 08:26 schrieb Stefan Beller:\n> On 06.06.2015 12:53, Luca Milanesio wrote:\n>>> On 6 Jun 2015, at 18:49, Phil Hord wrote:\n>>> On Fri, Jun 5, 2015, 2:58 AM lucamilanesio wrote:\n>>>> Ideally, as a \"git clone --recursive\" already exists, I would like to\n>>>> see a \"git diff --recursive\" that goes through the submodules as well :-)\n>>>>\n>>>> Something possibly to propose to the Git mailing list?\n\nSuch an option makes lots of sense to me (though \"--recurse-submodules\"\nshould be its name for consistency reasons). This could be an alias for\n\"--submodule=full\", as the \"--submodule\" option controls the format of\nsubmodule diffs.\n\n>>> I've worked on git diff --recursive a bit myself, along with some\n>>> simpler use cases (git ls-tree --recursive) as POCs. I think some of\n>>> the needs there begin to have ui implications which could be\n>>> high-friction. I really want to finish it someday, but I've been too\n>>> busy lately at $job, and now my experiments are all rather stale.\n>>>\n>>> It would be a good discussion to have over at the git list (copied).\n>>> Heiko and Jens have laid some new groundwork in this area and it may\n>>> be a good time to revisit it. Or maybe they've even moved deeper than\n>>> that; I have been distracted for well over a year now.\n>>>\n>\n> Glad you're working (or planning to) working on submodulues. This is\n> also on my todo list for the next months as well.\n\nMore hands are always welcome!\n\n> I'd review stuff in that area if you're looking for reviewers.\n\nI'll be happy help too.\n"},{"id":"263611","messageId":"CABURp0qf3TCB5ofKG4=MHz1VP4_g8Es8=s9aefW4Sr2b6ZCz_A@mail.gmail.com","threadId":"39545","inReplyTo":"5577330E.3060803@web.de","subject":"Re: Submodules as first class citizens (was Re: Moving to subtrees for plugins?)","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2015-06-11T16:11:28Z","receivedAt":"2015-06-11T16:11:28Z","isPatch":false,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Tue, Jun 9, 2015 at 2:40 PM, Jens Lehmann wrote:\n> Am 07.06.2015 um 08:26 schrieb Stefan Beller:\n>>\n>> On 06.06.2015 12:53, Luca Milanesio wrote:\n>>>>\n>>>> On 6 Jun 2015, at 18:49, Phil Hord wrote:\n>>>> On Fri, Jun 5, 2015, 2:58 AM lucamilanesio \n>>>> wrote:\n>>>>>\n>>>>> Ideally, as a \"git clone --recursive\" already exists, I would like to\n>>>>> see a \"git diff --recursive\" that goes through the submodules as well\n>>>>> :-)\n>>>>>\n>>>>> Something possibly to propose to the Git mailing list?\n>\n>\n> Such an option makes lots of sense to me (though \"--recurse-submodules\"\n> should be its name for consistency reasons). This could be an alias for\n> \"--submodule=full\", as the \"--submodule\" option controls the format of\n> submodule diffs.\n\nTo me, --recurse-submodules means submodules are still not first-class\ncitizens. But let's put that aside for a moment; I don't care about\nthe switch name too much as long as I can configure\n'diff.recurse-submodules = true'.\n\n[The following is rather long. I'm sorry for that. Feel free to look\naway when it gets too vague.]\n\nLet me set up a submodule like so:\n\n $ git init /tmp/Super && cd /tmp/Super\n Super$ git submodule add https://github.com/gitster/git.git Foo\n\nI wish to be able to grep from Super and find matches in all my submodules.\n\n Super$ git grep --recurse-submodules base--int\n Foo/.gitignore:/git-rebase--interactive\n Foo/Makefile:SCRIPT_LIB += git-rebase--interactive\n\nBut I want this to work naturally across git-module boundaries, so I\nwant this also to work (grepping a super-project from within a\nsubmodule):\n\n Super$ cd Foo\n Foo$ git grep --recurse-submodules base--int ..\n .gitignore:/git-rebase--interactive\n Makefile:SCRIPT_LIB += git-rebase--interactive\n\nI expect some groans from the audience here, because I think if the\nsyntax above worked, then so would this:\n\n $ cd /tmp\n tmp$ git grep base--int /tmp/Super/Foo\n /tmp/Super/Foo/.gitignore:/git-rebase--interactive\n /tmp/Super/Foo/Makefile:SCRIPT_LIB += git-rebase--interactive\n\nThis usage has nothing to do with submodules, really, except that it\nallows git commands to reach into foreign git directories by virtue of\nthe path supplied as some argument instead of via $GITDIR, and in\ndoing so it helps solve some git submodules use cases of mine.\n\nBut if that did not turn your stomach, try this one:\n\n $ cd /tmp/Super\n Super$ printf \"Some submodule data\">Foo/data.txt\n Super$ git add Foo/data.txt\n fatal: Pathspec 'Foo/data.txt' is in submodule 'Foo'\n Super$ git add --recurse-submodules Foo/data.txt\n\nSome notes on this usage:\n\n1. --recurse-submodules seems like a reasonable name for this switch,\nespecially when you consider the 'git add --recurse-submodules .' use\ncase.\n\n2. This recursive 'git add' seems dangerous to me unless git-status\nalso shows all the changed/untracked files in submodules as well if\nthe --recurse-submodules switch is included. This would support the\nexpectation that 'git add .' is going to add the files shown by 'git\nstatus .'\n\n3. Configuring --recurse-submodules as the default mode for 'git add'\nbut not for 'git status' seems reckless enough that I think there\nshould not be separate options for these two commands. There are\nprobably many other \"cross-command\" scenarios with similar coupling.\n\nMoving on, as we have :/ to mean 'workdir root', I wonder how you\nwould spell \"super-project workdir root\". Maybe it would be ::/\n\nI realize the kinds of features I'm talking about require extensive\ncode changes in Git. For example, consider the meaning of this:\n\n Super$ git diff --recurse-submodules origin/next origin/master\n\nSince I created Super just a few minutes ago and it has no remote\nnamed 'origin', this command seems meaningless to me. But suppose\nthat origin/next and origin/master did exist in my Super project.\nThen, I would expect in my wishlist Git, that\n\nA. Super$ git diff --recurse-submodules origin/next origin/master\nThis would include differences in Foo between origin/master:Foo and\norigin/next:Foo; that is, the commits referenced from those gitlinks\nin Super.\n\nB. Super$ git diff --recurse-submodules origin/next HEAD\nThis would include differences in Foo between origin/master:Foo and\nHEAD:Foo; that is, the commits referenced from those gitlinks in\nSuper.\n\nC. Super$ git diff --recurse-submodules origin/next\nThis would include differences in Foo between origin/master:Foo and\nthe current Foo workdir.\n\nD. Super$ cd Foo && git diff origin/next\nThis would include differences in Foo between the Foo submodule's\norigin/master and the current Foo workdir.\n\nNow, C and D seem confusingly similar to me and technically very\ndifferent. I could understand the results, but I could easily be led\nastray, especially if I am writing a script. But I still think it is\nreasonable and correct.\n\nI think this could have dire consequences for some commands like 'git\napply'. But I think it is reasonable for git apply to reject such\ncross-project diffs, at least in the beginning. :-)\n\nWhile I am thinking about it, let me also mention these cases:\nE. Super$ git diff --recurse-submodules origin/next origin/master -- Foo\nI think 'origin/next' and 'origin/master' here are referring to\nSuper's refs, but I can imagine an implementer choosing to use Foo's\ninstead.\n\nF. Super$ cd Foo\n Foo$ git diff --recurse-submodules origin/next origin/master -- ..\nIf this worked, I would think 'origin/next' and 'origin/master' here\nmust refer to Super's refs even though I began in Foo. This one is so\nambiguous I think I would have to call this an error. More\nspecifically, I think it would have to be rewritten like this next one\n(G).\n\nG. Super$ cd Foo\n Foo$ git -C .. diff --recurse-submodules origin/next origin/master\nThat is, at least for 'git diff', the parameter at the end is\nonly used to filter the results; it is not used to find the git-dir.\n\nBut look at me speaking in the present tense. How silly. I live too\nmuch in my own imagination.\n\nPhil\n\n-- \n"},{"id":"263627","messageId":"5579D9FB.8050104@web.de","threadId":"39545","inReplyTo":"CABURp0qf3TCB5ofKG4=MHz1VP4_g8Es8=s9aefW4Sr2b6ZCz_A@mail.gmail.com","subject":"Re: Submodules as first class citizens (was Re: Moving to subtrees for plugins?)","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2015-06-11T18:56:59Z","receivedAt":"2015-06-11T18:56:59Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 11.06.2015 um 18:11 schrieb Phil Hord:\n> On Tue, Jun 9, 2015 at 2:40 PM, Jens Lehmann wrote:\n>> Am 07.06.2015 um 08:26 schrieb Stefan Beller:\n>>>\n>>> On 06.06.2015 12:53, Luca Milanesio wrote:\n>>>>>\n>>>>> On 6 Jun 2015, at 18:49, Phil Hord wrote:\n>>>>> On Fri, Jun 5, 2015, 2:58 AM lucamilanesio \n>>>>> wrote:\n>>>>>>\n>>>>>> Ideally, as a \"git clone --recursive\" already exists, I would like to\n>>>>>> see a \"git diff --recursive\" that goes through the submodules as well\n>>>>>> :-)\n>>>>>>\n>>>>>> Something possibly to propose to the Git mailing list?\n>>\n>>\n>> Such an option makes lots of sense to me (though \"--recurse-submodules\"\n>> should be its name for consistency reasons). This could be an alias for\n>> \"--submodule=full\", as the \"--submodule\" option controls the format of\n>> submodule diffs.\n>\n> To me, --recurse-submodules means submodules are still not first-class\n> citizens. But let's put that aside for a moment; I don't care about\n> the switch name too much as long as I can configure\n> 'diff.recurse-submodules = true'.\n\nAfter somebody implemented the 'full' mode for 'diff --submodule',\nsetting 'diff.submodule' to 'full' would make --recurse-submodules the\ndefault for diff (unless recursing into the submodules is overridden\nby either the global 'diff.ignoreSubmodules' or the per-submodule\n'submodule..ignore' setting of course).\n\n> [The following is rather long. I'm sorry for that. Feel free to look\n> away when it gets too vague.]\n\nSorry, that was too long for todays git time budget ;-)\n"},{"id":"263843","messageId":"20150615090354.GA8048@book.hvoigt.net","threadId":"39545","inReplyTo":"5577330E.3060803@web.de","subject":"Re: Submodules as first class citizens (was Re: Moving to subtrees for plugins?)","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2015-06-15T09:03:54Z","receivedAt":"2015-06-15T09:03:54Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"On Tue, Jun 09, 2015 at 08:40:14PM +0200, Jens Lehmann wrote:\n> Am 07.06.2015 um 08:26 schrieb Stefan Beller:\n> >On 06.06.2015 12:53, Luca Milanesio wrote:\n> >>>On 6 Jun 2015, at 18:49, Phil Hord wrote:\n> >>>On Fri, Jun 5, 2015, 2:58 AM lucamilanesio wrote:\n> >>>>Ideally, as a \"git clone --recursive\" already exists, I would like to\n> >>>>see a \"git diff --recursive\" that goes through the submodules as well :-)\n> >>>>\n> >>>>Something possibly to propose to the Git mailing list?\n> \n> Such an option makes lots of sense to me (though \"--recurse-submodules\"\n> should be its name for consistency reasons). This could be an alias for\n> \"--submodule=full\", as the \"--submodule\" option controls the format of\n> submodule diffs.\n\nBTW, for long running topics (or low hanging fruits) we collect/link\neverything in the wiki of Jens git fork on github. This is the central\npage:\n\nhttps://github.com/jlehmann/git-submod-enhancements/wiki\n\nMaybe everyone that has work in the queue can add his work there (the work that\ntakes more time) so we can avoid doubling any effort. Not everything there is\nup to date at the moment but I will look into it to remove outdated things.\n\n> >>>I've worked on git diff --recursive a bit myself, along with some\n> >>>simpler use cases (git ls-tree --recursive) as POCs. I think some of\n> >>>the needs there begin to have ui implications which could be\n> >>>high-friction. I really want to finish it someday, but I've been too\n> >>>busy lately at $job, and now my experiments are all rather stale.\n> >>>\n> >>>It would be a good discussion to have over at the git list (copied).\n> >>>Heiko and Jens have laid some new groundwork in this area and it may\n> >>>be a good time to revisit it. Or maybe they've even moved deeper than\n> >>>that; I have been distracted for well over a year now.\n> >>>\n> >\n> >Glad you're working (or planning to) working on submodulues. This is\n> >also on my todo list for the next months as well.\n> \n> More hands are always welcome!\n> \n> >I'd review stuff in that area if you're looking for reviewers.\n> \n> I'll be happy help too.\n\nMe too.\n\nCheers Heiko\n"}]}