{"thread":{"id":"13541","subject":"Why do git submodules require manual checkouts and commits?","startedAt":"2008-05-16T04:16:23Z","lastAt":"2008-05-19T04:38:16Z","messageCount":9,"participants":["skillzero@gmail.com","Johannes Schindelin","Avery Pennarun","Lars Hjemli"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"77097","messageId":"2729632a0805152116o3c998324xb401674207dd2e1e@mail.gmail.com","threadId":"13541","inReplyTo":null,"subject":"Why do git submodules require manual checkouts and commits?","fromName":"","fromEmail":"skillzero@gmail.com","sentAt":"2008-05-16T04:16:23Z","receivedAt":"2008-05-16T04:16:23Z","isPatch":false,"sender":{"key":"skillzero@gmail.com","avatar":null},"body":"Why do git submodules require manually committing the submodule itself\nto each super repository after something in the submodule repository\nchanges? Is there some reason the super repository can't just \"link\"\nto the submodules by branch name? It seems that if the .gitsubmodules\nalso specified the branch to use:\n\n[submodule \"libfoo\"]\n\tpath = libs/foo\n\turl = git://foo.com/git/libfoo.git\n\tbranch = master\n\n[submodule \"libbar\"]\n\tpath = libs/bar\n\turl = git://bar.com/git/libbar.git\n\tbranch = stable\n\nThen a git pull (or git clone) of the super repository could also pull\nin all submodules. A commit to a file in a submodule would then be\nautomatically reflected in the super repository (since the super\nrepository would always pull HEAD of that branch).\n\nIs this difficult (or somehow undesirable)?\n"},{"id":"77105","messageId":"alpine.DEB.1.00.0805161055540.30431@racer","threadId":"13541","inReplyTo":"2729632a0805152116o3c998324xb401674207dd2e1e@mail.gmail.com","subject":"Re: Why do git submodules require manual checkouts and commits?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-05-16T10:17:14Z","receivedAt":"2008-05-16T10:17:14Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 15 May 2008, skillzero@gmail.com wrote:\n\n> Why do git submodules require manually committing the submodule itself\n> to each super repository after something in the submodule repository\n> changes?\n\nSubmodules are special.\n\nYou cannot recreate the exact state from a commit in the superproject, for \none, and often not even from the commit itself, since the submodule can \ncontain more than just the tracked files.\n\nAlso, no submodule _has_ to be checked out. If you are working inside a \nsuperproject, chances are that you are uninterested in most of the \nsubmodules.\n\nSo no, there is nothing to change here, please move along.\n\nCiao,\nDscho\n"},{"id":"77129","messageId":"32541b130805160643y3bfe609et22b2d00627f98c04@mail.gmail.com","threadId":"13541","inReplyTo":"2729632a0805152116o3c998324xb401674207dd2e1e@mail.gmail.com","subject":"Re: Why do git submodules require manual checkouts and commits?","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-05-16T13:43:33Z","receivedAt":"2008-05-16T13:43:33Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 5/16/08, skillzero@gmail.com wrote:\n> Why do git submodules require manually committing the submodule itself\n> to each super repository after something in the submodule repository\n> changes? Is there some reason the super repository can't just \"link\"\n> to the submodules by branch name? It seems that if the .gitsubmodules\n> also specified the branch to use:\n\nThis is difficult but (contrary to what others might say :)) in my\nopinion it's worth fixing. Exactly how to fix it is an open question\nthough.\n\nThe main reason the simple approach you suggested (just link to a\nbranch instead of a particular commit) isn't good is that it doesn't\nguarantee you always get the same version every time. If the\nsupermodule links to submodule branch \"foo\" and makes supermodule\ncommit 23abc918c, and someone later pushes to submodule branch \"foo\",\nthen checking out commit 23abc918c in the supermodule would get the\nvery latest submodule \"foo\", not the one you had when 23abc918c was\ncreated. Thus all sorts of bad things could happen.\n\nSo I think it would be very bad if the supermodule automatically\nupdated to the latest version of the submodule whenever you commit in\nthe submodule. *However*, the other way around might be fine: if you\ncommit in the supermodule, maybe it should commit in the submodule at\nthe same time and link to that specific commit. I'm pretty sure that\nidea doesn't have any *fundamental* flaws, it's just got a lot of\nreally tricky details that need to be worked out. I've seen some\n\"recursive commit\" and \"recursive push\" patches floating around, so\npeople are actively working on this. One of the hardest things to\ndeal with is where to auto-push submodules to (which remote? which\nbranch?) when you push the supermodule.\n\nHave fun,\n\nAvery\n"},{"id":"77130","messageId":"alpine.DEB.1.00.0805161457250.30431@racer","threadId":"13541","inReplyTo":"32541b130805160643y3bfe609et22b2d00627f98c04@mail.gmail.com","subject":"Re: Why do git submodules require manual checkouts and commits?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-05-16T13:58:21Z","receivedAt":"2008-05-16T13:58:21Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 16 May 2008, Avery Pennarun wrote:\n\n> So I think it would be very bad if the supermodule automatically\n> updated to the latest version of the submodule whenever you commit in\n> the submodule. *However*, the other way around might be fine: if you\n> commit in the supermodule, maybe it should commit in the submodule at\n> the same time and link to that specific commit. I'm pretty sure that\n> idea doesn't have any *fundamental* flaws, it's just got a lot of\n> really tricky details that need to be worked out.\n\nJust the fundamental flaw that you might _not_ want to commit that, just \nas you can have a dirty Makefile _forever_.\n\nCiao,\nDscho\n"},{"id":"77134","messageId":"32541b130805160712m24f24c6aw59b54a0f0ace6269@mail.gmail.com","threadId":"13541","inReplyTo":"alpine.DEB.1.00.0805161457250.30431@racer","subject":"Re: Why do git submodules require manual checkouts and commits?","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-05-16T14:12:16Z","receivedAt":"2008-05-16T14:12:16Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 5/16/08, Johannes Schindelin wrote:\n> On Fri, 16 May 2008, Avery Pennarun wrote:\n> > So I think it would be very bad if the supermodule automatically\n> > updated to the latest version of the submodule whenever you commit in\n> > the submodule. *However*, the other way around might be fine: if you\n> > commit in the supermodule, maybe it should commit in the submodule at\n> > the same time and link to that specific commit. I'm pretty sure that\n> > idea doesn't have any *fundamental* flaws, it's just got a lot of\n> > really tricky details that need to be worked out.\n>\n> Just the fundamental flaw that you might _not_ want to commit that, just\n> as you can have a dirty Makefile _forever_.\n\nI consider that one of the annoying details rather than a fundamental\nflaw. I agree that it's hard to solve though.\n\nThink of it this way: I can commit, or not commit, my dirty Makefile\nat the same time as everything else (in a single project) with a\nsingle \"git commit\" line, depending on what I want to do. Things like\n\"git commit -a\" and \"git add -u\" speed up the common case where I just\nwant to commit everything. But with submodules, that common case\nlooks more like this:\n\n cd sub\n git checkout -b manual_branchname_because_there_was_no_default\n git commit -a\n git push etc.\n cd ..\n git commit -a\n git push etc.\n\nThat's *really* tedious, and the number of commands multiplies when\nyou have more than one submodule going at once.\n\nI think git's submodules are awesome because they *don't* have\nfundamental flaws. They just need an (optional) more automated\nworkflow for the common case. And I'll be sure to propose one when I\nfigure out what my common case actually is :)\n\nHave fun,\n\nAvery\n"},{"id":"77137","messageId":"alpine.DEB.1.00.0805161521510.30431@racer","threadId":"13541","inReplyTo":"32541b130805160712m24f24c6aw59b54a0f0ace6269@mail.gmail.com","subject":"Re: Why do git submodules require manual checkouts and commits?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-05-16T14:24:18Z","receivedAt":"2008-05-16T14:24:18Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 16 May 2008, Avery Pennarun wrote:\n\n> Think of it this way: I can commit, or not commit, my dirty Makefile at \n> the same time as everything else (in a single project) with a single \n> \"git commit\" line, depending on what I want to do. Things like \"git \n> commit -a\" and \"git add -u\" speed up the common case where I just want \n> to commit everything. But with submodules, that common case looks more \n> like this:\n> \n> cd sub\n> git checkout -b manual_branchname_because_there_was_no_default\n> git commit -a\n> git push etc.\n> cd ..\n> git commit -a\n> git push etc.\n\nFunny, for me it looks completely different:\n\n$ cd sub\n# work, work, work\n# from time to time commit\n# from time to time rebase -i to clean up some things\n# test, test, test\n# sometimes push\n\nAnd then, every once in a while, it is\n\n$ cd ..\n$ git add submodule\n$ git commit -s submodule\n$ git push\n\n> That's *really* tedious, and the number of commands multiplies when you \n> have more than one submodule going at once.\n\nBut hey, if you find that tedious, why did I not see a patch from you yet, \nimplementing \"git submodule commit-n-push\"?\n\nCiao,\nDscho\n"},{"id":"77141","messageId":"32541b130805160744u6afb2018y931993ea342cade6@mail.gmail.com","threadId":"13541","inReplyTo":"alpine.DEB.1.00.0805161521510.30431@racer","subject":"Re: Why do git submodules require manual checkouts and commits?","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-05-16T14:44:46Z","receivedAt":"2008-05-16T14:44:46Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 5/16/08, Johannes Schindelin wrote:\n> On Fri, 16 May 2008, Avery Pennarun wrote:\n> > But with submodules, that common case looks more\n> > like this:\n> > cd sub\n> > git checkout -b manual_branchname_because_there_was_no_default\n> > git commit -a\n> > git push etc.\n> > cd ..\n> > git commit -a\n> > git push etc.\n>\n> Funny, for me it looks completely different:\n>\n> $ cd sub\n> # work, work, work\n> # from time to time commit\n> # from time to time rebase -i to clean up some things\n> # test, test, test\n> # sometimes push\n>\n> And then, every once in a while, it is\n>\n> $ cd ..\n> $ git add submodule\n> $ git commit -s submodule\n> $ git push\n\nIndeed. I think you and I use submodules for very different purposes.\n The important case for me is that I often add a function to a library\n(submodule), then *immediately* move on to use that function in an\napplication (supermodule). When I commit the new submodule gitlink,\nthere are almost always other source code changes in the supermodule\nat the same time. And then I want to share the new application with\nmy co-workers shortly thereafter, often several times per day.\n\nIt looks like your own work on submodules tends to be quite isolated\nfrom the supermodule, which would make sense if you use submodules for\n(say) bundling a bunch of separate applications together.\n\n> > That's *really* tedious, and the number of commands multiplies when you\n> > have more than one submodule going at once.\n>\n> But hey, if you find that tedious, why did I not see a patch from you yet,\n> implementing \"git submodule commit-n-push\"?\n\nWell, I think someone already submitted some \"git submodule recursive\" stuff.\n\nI don't mean to complain. I think the existing git-submodule code\ngives all the raw materials necessary to solve all my\nsubmodule-related problems, whatever those problems may be. I'd much\nrather have tedious-but-flexible instead of quick-but-inflexible.\nHowever, having such powerful tools means it takes longer to figure\nout how to use them most effectively.\n\nDon't worry, I'll be the first to submit an all-singing all-dancing\nshell script as soon as I figure out what that looks like. Err,\nunless someone else submits it first. I guess that should cover all\nmy bases :)\n\n(And now... I'm going to disappear for at least a week for vacation. Bye!)\n\nHave fun,\n\nAvery\n"},{"id":"77143","messageId":"8c5c35580805160758n3b43c282i48009867b52106bf@mail.gmail.com","threadId":"13541","inReplyTo":"alpine.DEB.1.00.0805161521510.30431@racer","subject":"Re: Why do git submodules require manual checkouts and commits?","fromName":"Lars Hjemli","fromEmail":"lh@elementstorage.no","sentAt":"2008-05-16T14:58:44Z","receivedAt":"2008-05-16T14:58:44Z","isPatch":false,"sender":{"key":"lh@elementstorage.no","avatar":null},"body":"On Fri, May 16, 2008 at 4:24 PM, Johannes Schindelin\n wrote:\n> On Fri, 16 May 2008, Avery Pennarun wrote:\n>\n>\n> > Think of it this way: I can commit, or not commit, my dirty Makefile at\n> > the same time as everything else (in a single project) with a single\n> > \"git commit\" line, depending on what I want to do. Things like \"git\n> > commit -a\" and \"git add -u\" speed up the common case where I just want\n> > to commit everything. But with submodules, that common case looks more\n> > like this:\n> >\n> > cd sub\n> > git checkout -b manual_branchname_because_there_was_no_default\n> > git commit -a\n> > git push etc.\n> > cd ..\n> > git commit -a\n> > git push etc.\n>\n> Funny, for me it looks completely different:\n>\n> $ cd sub\n> # work, work, work\n> # from time to time commit\n> # from time to time rebase -i to clean up some things\n> # test, test, test\n> # sometimes push\n>\n> And then, every once in a while, it is\n>\n> $ cd ..\n> $ git add submodule\n> $ git commit -s submodule\n> $ git push\n\nJust to add to the picture, for me it's\n\n$ (cd submodule && git checkout tag)\n$ git add submodule\n$ git commit -s -m \"Use submodule-tag\"\n\nIf I want to work in/with the submodule, I usually do that by \"cd\n../submodule\", i.e. I've got another clone of the submodule\nrepository.\n\n--\nlh\n"},{"id":"77253","messageId":"2729632a0805182138s3e268cdbxd0d7c42bbcf01f84@mail.gmail.com","threadId":"13541","inReplyTo":"32541b130805160643y3bfe609et22b2d00627f98c04@mail.gmail.com","subject":"Re: Why do git submodules require manual checkouts and commits?","fromName":"","fromEmail":"skillzero@gmail.com","sentAt":"2008-05-19T04:38:16Z","receivedAt":"2008-05-19T04:38:16Z","isPatch":false,"sender":{"key":"skillzero@gmail.com","avatar":null},"body":"On Fri, May 16, 2008 at 6:43 AM, Avery Pennarun wrote:\n\n> The main reason the simple approach you suggested (just link to a\n> branch instead of a particular commit) isn't good is that it doesn't\n> guarantee you always get the same version every time. If the\n> supermodule links to submodule branch \"foo\" and makes supermodule\n> commit 23abc918c, and someone later pushes to submodule branch \"foo\",\n> then checking out commit 23abc918c in the supermodule would get the\n> very latest submodule \"foo\", not the one you had when 23abc918c was\n> created. Thus all sorts of bad things could happen.\n\nIt seems like this could be handled by changing the way a commit is\ntracked such that it also includes the commit hash of the submodules.\nSo for example, instead of refs/heads/master just being just a single\ncommit hash, it would be a list:\n\n[refs/heads/master]\n 1033bb1ed64d1dbac9f93360e69402195386d145\n libfoo = a13dcb7f26a160d85038385b3024b700dec208d9\n libbar = 123123123dcb7f26a160d85038385b3024b700de\n\nIf I commit a change to libfoo, it updates libfoo's commit hash to the latest.\nIf I do git pull on the supermodule, it recursively does git pull on\nall submodules and then updates refs/heads/master (or whatever branch\nyou happen to be on) with the new commit hashes of the submodules.\n\nIf you checkout supermodule commit 1033bb1, it would look at the\nsupermodule's refs/heads/master for 1033bb and see that libfoo was at\na13dcb and libbar was at 123123 and it would then check out those\ncommits, respectively.\n\nIf you branch the supermodule, it recursively branches the submodules\nwith the same name, but using a namespace. Maybe super/\nsimilar to how origin/branch is use for remote branches.\n\nIf you push the supermodule, it would also recursively push the\nsubmodules using the URL specified when it was cloned (i.e. the one\nspecified in the .gitsubmodules file).\n\nWhat I'm trying to do is use git to manage multiple, large\nrepositories that share a lot of code. I want to make things more\nsparse so independent pieces can be used standalone or included as\nsubmodules in other, larger projects.\n"}]}