{"thread":{"id":"26170","subject":"how to update a submodule?","startedAt":"2010-12-31T22:24:38Z","lastAt":"2011-01-02T17:57:26Z","messageCount":7,"participants":["Oliver Kullmann","Seth Robertson","Jens Lehmann","Jonathan Nieder"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"158796","messageId":"20101231222438.GA28199@cs-wsok.swansea.ac.uk","threadId":"26170","inReplyTo":null,"subject":"how to update a submodule?","fromName":"Oliver Kullmann","fromEmail":"o.kullmann@swansea.ac.uk","sentAt":"2010-12-31T22:24:38Z","receivedAt":"2010-12-31T22:24:38Z","isPatch":false,"sender":{"key":"o.kullmann@swansea.ac.uk","avatar":null},"body":"Hello,\n\nit would be very helpful if somebody could tell me the\nsupposed workflow how to update a submodule (I'm using version\n1.7.3.2; the man-page doesn't say much about it).\n\nI have on the same machine two versions of repository A,\nsay A1, A2, and both contain repository B as submodule, but are otherwise\nunrelated.\n\nIn A1 we have B up-to-date, but not in A2 (and with up-to-date I mean\nthe master branch in the same state as in A1).\n\ngit submodule update\n\nis of no help, since A1, A2 have no relation to each other, and \"update\"\nassumes some relation; the man-page doesn't speak of options for update,\nand since \"git submodule update\" runs through all submodules, it's hard\nto see how that could work.\n\nSo I have to go into B in A2, and do some pull (I suppose), but that\ndoesn't work so easily; the man-page seems to speak of B in A2 is a \"normal repository\"\nbut that doesn't seem to be the case in the sense that by default \nwe are on \"no branch\". So it seems one can't do anything there, since \nyou can't merge \"no branch\" into some other branch. The branch \"no branch\"\nis up-to-date after a pull, but this can't be communicated to the other\nbranches (so seems to be misleading, or a waste of time).\n\nSo apparently I have to checkout master in A2:B, and then pull from\nmaster in A1:B ?\n\nWould be good if somebody could shed light on this.\n\nIn general I find that \"pull\" is much under-documented in git.\nWhat I would like to do in A2 is just to say \"pull everything from A1\",\nwhich seems a logical thing to do, and which should include all corresponding\nbranches and all corresponding submodules, but such a \"pull all\" doesn't\nseem to exist?\n\nThanks for your help in any case!\n\nOliver\n\nP.S. What would be really needed for the Git documentation is to speak\nabout the fundamental ideas, about the \"mental images\".\n\nThe submodule documentation starts with a blurb-sentence which I would\nregard as misleading.\nThen comes a paragraph about what it is not.\nThen comes some hints regarding the implementation.\nThen the final paragraph seems to indicate that above I could just\nuse \"git pull A2\" for A1, and would get B in A2 from B in A1, but only not checked out?\nThis doesn't seem to be the case, since apparently then \"git submodule update\"\ntries to fetch from some foreign repository (not just checking out what was fetched before)?\n\nAdding here examples for a complete workflow (perhaps with various stages, but without\ncomplications(!)), from initialisation to updates from other repositories would be\nvery helpful.\n\nAs you can see, I don't know what to expect, and so I don't know what are errors in\nthe usage, and what is supposed to be. No mental image about \"submodule\" is offered,\nand I don't know what is its intention. A great thing would be using the computer-science\nconcept of an abstract data type, so to explain what the interface and its semantics is,\nnot what is the implementation.\n"},{"id":"158797","messageId":"201012312342.oBVNg1lx021930@no.baka.org","threadId":"26170","inReplyTo":"20101231222438.GA28199@cs-wsok.swansea.ac.uk","subject":"Re: how to update a submodule?","fromName":"Seth Robertson","fromEmail":"in-gitvger@baka.org","sentAt":"2010-12-31T23:42:01Z","receivedAt":"2010-12-31T23:42:01Z","isPatch":false,"sender":{"key":"in-gitvger@baka.org","avatar":null},"body":"\nIn message <20101231222438.GA28199@cs-wsok.swansea.ac.uk>, Oliver Kullmann writ\nes:\n\n    it would be very helpful if somebody could tell me the\n    supposed workflow how to update a submodule (I'm using version\n    1.7.3.2; the man-page doesn't say much about it).\n\nhttps://git.wiki.kernel.org/index.php/GitSubmoduleTutorial\n\n--------------------------------------------------\ncd submodule_path\ngit checkout <branchname>\ngit pull\ncd .. # or otherwise get to the superproject\n\ngit add submodule_path\ngit commit -m \"Updated submodule to latest <branchname>\"\ngit submodule update\n--------------------------------------------------\n\nThis (untested) code obviously only works for a single repository.  If\nyou want to do it for all repositories, you need something more like\nthe untested:\n\n--------------------------------------------------\ngit submodule foreach git checkout <branchname>\ngit submodule foreach git pull\ngit add -a -m \"Updated all submodules to latest <branchname>\"\ngit submodule update\n--------------------------------------------------\n\nIf you find yourself doing this continuously, and doing it for all of\nyour submodules, you may way to use gitslave instead of submodules\nwhich keeps the branches checked out all of the time so all you need to\ndo is `gits pull`.  This may be better, or worse, for you workflow.\n\ngitslave (http://gitslave.sf.net) is useful when you control and\ndevelop on the subprojects at more of less the same time as the\nsuperproject, and furthermore when you typically want to tag, branch,\npush, pull, etc all repositories at the same time.\n\ngit-submodule is better when you do not control the subprojects or\nmore specifically wish to fix the subproject at a specific revision\neven as the subproject changes\n\n\t\t\t\t\t-Seth Robertson\n"},{"id":"158802","messageId":"20110101203957.GC26920@cs-wsok.swansea.ac.uk","threadId":"26170","inReplyTo":"201012312342.oBVNg1lx021930@no.baka.org","subject":"Re: how to update a submodule?","fromName":"Oliver Kullmann","fromEmail":"o.kullmann@swansea.ac.uk","sentAt":"2011-01-01T20:39:57Z","receivedAt":"2011-01-01T20:39:57Z","isPatch":false,"sender":{"key":"o.kullmann@swansea.ac.uk","avatar":null},"body":"On Fri, Dec 31, 2010 at 06:42:01PM -0500, Seth Robertson wrote:\n> \n> In message <20101231222438.GA28199@cs-wsok.swansea.ac.uk>, Oliver Kullmann writ\n> es:\n> \n>     it would be very helpful if somebody could tell me the\n>     supposed workflow how to update a submodule (I'm using version\n>     1.7.3.2; the man-page doesn't say much about it).\n> \n> https://git.wiki.kernel.org/index.php/GitSubmoduleTutorial\n>\n\nthanks, that's definitely something.\n \n> --------------------------------------------------\n> cd submodule_path\n> git checkout <branchname>\n> git pull\n> cd .. # or otherwise get to the superproject\n> \n> git add submodule_path\n> git commit -m \"Updated submodule to latest <branchname>\"\n> git submodule update\n> --------------------------------------------------\n> \n> This (untested) code obviously only works for a single repository.  If\n> you want to do it for all repositories, you need something more like\n> the untested:\n> \n> --------------------------------------------------\n> git submodule foreach git checkout <branchname>\n> git submodule foreach git pull\n> git add -a -m \"Updated all submodules to latest <branchname>\"\n> git submodule update\n> --------------------------------------------------\n> \n\nAs far as I see that, this doesn't concern the problem how to I update\none repository with submodules from another repository with \"these\" submodules\n(as the same paths)?\n\nActually, even the simplest case of just cloning a repository with submodules\ndoesn't work:\nAfter cloning, \"git submodule status\" shows \"-\", okay so I do \"git submodule init\",\nwhich already shows a false understanding --- it shows the URL of the old repository,\nfrom which the original submodule originated, which is long gone and no longer relevant.\nThen of course \"git submodule update\" fails, and in the submodule there is just nothing.\n\nA clone should provide a relation to the mother-clone, not transitively to\nthe grandmother-clone.\n\nThe problem seems to be that the information about the place where to update a\nsubmodule is in .gitmodules, which git actually has under version control (different\nfrom other configuration information), and thus copies it verbatimly.\nOkay, then apparently after a clone .gitmodules has to be updated (by hand).\n\nSo .gitmodules concerns only \"git submodule update\", not \"git pull\" from within\nthe submodules? This would be good to know, to understand the role of the information\nin .gitmodules (where the task of \"git submodule init\" is apparently just to\ntransport this information to .git/configure ?).\n\nThis has the disadvantage, that one has to publish this private\ninformation about the places where by chance one is pulling from?\nPerhaps I should then put .gitmodules into .gitignore? Or would that have\nbad consequences??\n\nOr is the idea that .gitmodules normally is not under version control, and\nthus \"git init\", which actually hides the information of .gitmodules ?\n\n> If you find yourself doing this continuously, and doing it for all of\n> your submodules, you may way to use gitslave instead of submodules\n> which keeps the branches checked out all of the time so all you need to\n> do is `git pull`.  This may be better, or worse, for you workflow.\n> \n> gitslave (http://gitslave.sf.net) is useful when you control and\n> develop on the subprojects at more of less the same time as the\n> superproject, and furthermore when you typically want to tag, branch,\n> push, pull, etc all repositories at the same time.\n>\n\nOn the one hand, I always want to pull \"everything\" (inclusive the content of all submodules),\nhowever the submodules are completely independent (they are from Github, or collaborations on\npapers, and such things --- I just want to conveniently carry around all of that with me in\none go).\n \n> git-submodule is better when you do not control the subprojects or\n> more specifically wish to fix the subproject at a specific revision\n> even as the subproject changes\n>\n\nThis seems to be the case here, so it seemed to me that gitslave wouldn't be appropriate\nhere.\n\nThanks for your help!\n\nOliver\n \n"},{"id":"158815","messageId":"4D2061C7.5050405@web.de","threadId":"26170","inReplyTo":"20110101203957.GC26920@cs-wsok.swansea.ac.uk","subject":"Re: how to update a submodule?","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-01-02T11:30:15Z","receivedAt":"2011-01-02T11:30:15Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 01.01.2011 21:39, schrieb Oliver Kullmann:\n> On Fri, Dec 31, 2010 at 06:42:01PM -0500, Seth Robertson wrote:\n> As far as I see that, this doesn't concern the problem how to I update\n> one repository with submodules from another repository with \"these\" submodules\n> (as the same paths)?\n\nI'm not sure I completely understand your use case, but submodules are\nrepositories of their own, so they don't get updated by just pulling\na superproject into another containing the same submodule. The submodule\nchanges have to be pushed to its own parent repository and can then be\nfetched from there into another superproject's submodule.\n\n> Actually, even the simplest case of just cloning a repository with submodules\n> doesn't work:\n> After cloning, \"git submodule status\" shows \"-\", okay so I do \"git submodule init\",\n> which already shows a false understanding --- it shows the URL of the old repository,\n> from which the original submodule originated, which is long gone and no longer relevant.\n> Then of course \"git submodule update\" fails, and in the submodule there is just nothing.\n\nThen the URL of the submodule in the .gitmodules file is not up to\ndate. You can either fix it there or - if you only want to change it\nlocally - edit the .git/config after the \"git submodule init\" copied\nit there.\n\n> The problem seems to be that the information about the place where to update a\n> submodule is in .gitmodules, which git actually has under version control (different\n> from other configuration information), and thus copies it verbatimly.\n> Okay, then apparently after a clone .gitmodules has to be updated (by hand).\n\nNo, the problem seems to be that the remote URL in the submodule has\nbeen changed directly without updating it in the .gitmodules of the\nsuperproject (the recommended way to do that is to change it in the\n.gitmodules file and then use \"git submodule sync\" to activate it).\n\n> So .gitmodules concerns only \"git submodule update\", not \"git pull\" from within\n> the submodules? This would be good to know, to understand the role of the information\n> in .gitmodules (where the task of \"git submodule init\" is apparently just to\n> transport this information to .git/configure ?).\n> \n> This has the disadvantage, that one has to publish this private\n> information about the places where by chance one is pulling from?\n\nYes, the submodules URL must be available - at least for fetching -\nto everyone who wants to clone your superproject (and all changes\nmade in the submodule which are committed in the superproject must\nhave been pushed there to be able to check them out in the clone).\n\n> Perhaps I should then put .gitmodules into .gitignore? Or would that have\n> bad consequences??\n\nYes, because then git won't know where the it can clone the submodules\nfrom when you clone your superproject somewhere else.\n"},{"id":"158816","messageId":"20110102155514.GB32745@cs-wsok.swansea.ac.uk","threadId":"26170","inReplyTo":"4D2061C7.5050405@web.de","subject":"Re: how to update a submodule?","fromName":"Oliver Kullmann","fromEmail":"o.kullmann@swansea.ac.uk","sentAt":"2011-01-02T15:55:14Z","receivedAt":"2011-01-02T15:55:14Z","isPatch":false,"sender":{"key":"o.kullmann@swansea.ac.uk","avatar":null},"body":"On Sun, Jan 02, 2011 at 12:30:15PM +0100, Jens Lehmann wrote:\n> Am 01.01.2011 21:39, schrieb Oliver Kullmann:\n> > On Fri, Dec 31, 2010 at 06:42:01PM -0500, Seth Robertson wrote:\n> > As far as I see that, this doesn't concern the problem how to I update\n> > one repository with submodules from another repository with \"these\" submodules\n> > (as the same paths)?\n> \n> I'm not sure I completely understand your use case, but submodules are\n> repositories of their own, so they don't get updated by just pulling\n> a superproject into another containing the same submodule. The submodule\n> changes have to be pushed to its own parent repository and can then be\n> fetched from there into another superproject's submodule.\n>\n\nI know that --- but if there wouldn't be any savings possible (in terms of using it),\nthen submodules would be pointless, and so the question is *how* to use them.\n\nI see that my use-case is not understood so I try it:\n\nThe good thing with Git is that there are no central repositories.\nThat's exactly what I want to use, but again and again the automatic\nassumptions of \"central repositories\" are made, which should be actually alien to Git.\n\nMore concretely: For a collaboration on a scientific paper (where more and more\npeople have never seen each other, and this is also the case here) one doesn't\nuse version-control yet because of all this server-nonsense. Now with Git we can\njust toss tarballs of git-archives around! Sent via e-mail (please note that zero\nknowledge of git-remotes or whatever is required for that --- that's very important),\nthese archives float around, without any URL or central repository, but they\nare locally merged and whatever. I think that's exactly exploiting the Git philosophy.\n\nSo I have a couple of those. Then I have modules to teach, where I organise also various\nforms of collaborations with colleagues and students (various networks, various levels\nof security).\n\nThen there are software projects. Some private, some not.\n\nAll of that happens independently. I have at this time around 15 of such active projects\n(some smaller, some rather big).\n\nNow it happens that things are forgotten, updates aren't done, I just forget to copy\na repository from one place to another, etc.\nImportant that from my main working place at home to the main university working place\nthere is no network connection, but memory sticks do the job (which is fine and safe).\n\nSure, I can (and perhaps have to) start writing scripts, which just exploit basic git,\nand add some form of super-organisation on top of it.\n\nHowever I still hope that the task is so common that I can use tools.\n\nTo summarise: At some point in time on some machine, a super-repository A is updated,\nfrom various (permanently changing) places etc. No problem with pull itself.\nThe submodules each needs to be taken care of individually.\n\nOkay, but then I want to take that whole thing, the supermodule with the 15 (or more)\nsubmodules, copy it to the memory stick, and on my workplace untar it (decrypt it ...),\nand update a super-repository B *over there*, just from A, which has nothing to\ndo how I created A (on a different machine, completely different circumstances).\n\nIt seems to fully comply with the original Git philosophy, that I can make some\nform of \"super-pull\" from B, getting everything from A --- without the assumption\nof some central URL (repository), neither for the supermodule nor for the submodules.\nJust pulling everything over, as it happens with ordinary \"git pull\".\n\n \n> > Actually, even the simplest case of just cloning a repository with submodules\n> > doesn't work:\n> > After cloning, \"git submodule status\" shows \"-\", okay so I do \"git submodule init\",\n> > which already shows a false understanding --- it shows the URL of the old repository,\n> > from which the original submodule originated, which is long gone and no longer relevant.\n> > Then of course \"git submodule update\" fails, and in the submodule there is just nothing.\n> \n> Then the URL of the submodule in the .gitmodules file is not up to\n> date. You can either fix it there or - if you only want to change it\n> locally - edit the .git/config after the \"git submodule init\" copied\n> it there.\n>\n\nThat file .gitmodule seems to enforce the central structure, contradicting the original\n(and great!) Git insight of fully distributed version control.\n \n> > The problem seems to be that the information about the place where to update a\n> > submodule is in .gitmodules, which git actually has under version control (different\n> > from other configuration information), and thus copies it verbatimly.\n> > Okay, then apparently after a clone .gitmodules has to be updated (by hand).\n> \n> No, the problem seems to be that the remote URL in the submodule has\n> been changed directly without updating it in the .gitmodules of the\n> superproject (the recommended way to do that is to change it in the\n> .gitmodules file and then use \"git submodule sync\" to activate it).\n>\n\nAgain, I hope by now it is clear that in my cases it makes no sense\nto attribute an URL to the submodule.\n \n> > So .gitmodules concerns only \"git submodule update\", not \"git pull\" from within\n> > the submodules? This would be good to know, to understand the role of the information\n> > in .gitmodules (where the task of \"git submodule init\" is apparently just to\n> > transport this information to .git/configure ?).\n> > \n> > This has the disadvantage, that one has to publish this private\n> > information about the places where by chance one is pulling from?\n> \n> Yes, the submodules URL must be available - at least for fetching -\n> to everyone who wants to clone your superproject (and all changes\n> made in the submodule which are committed in the superproject must\n> have been pushed there to be able to check them out in the clone).\n>\n\nThen submodules would be a wrong concept, as it seems to me, contradicting\ndecentralised version control. Still I have the hope that actually\nsubmodule can be used, but apparently that file .gitmodules is in the way\n(false information is much worse than no information!).\n \n> > Perhaps I should then put .gitmodules into .gitignore? Or would that have\n> > bad consequences??\n> \n> Yes, because then git won't know where the it can clone the submodules\n> from when you clone your superproject somewhere else.\n\nBut that would be what is needed --- there is no \"somewhere else\", but\njust the given repository.\n\nPuuuh, I really really tried hard now to make my use-case clear :-|. Hopefully\nnow the picture emerges.\n\nThanks for your considerations! And I hope you don't mind me speaking of\n\"contradictions\" etc., but I feel there are fundamental misunderstandings.\n\nThanks!\n\nOliver\n"},{"id":"158818","messageId":"4D20B629.8000107@web.de","threadId":"26170","inReplyTo":"20110102155514.GB32745@cs-wsok.swansea.ac.uk","subject":"Re: how to update a submodule?","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-01-02T17:30:17Z","receivedAt":"2011-01-02T17:30:17Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 02.01.2011 16:55, schrieb Oliver Kullmann:\n> On Sun, Jan 02, 2011 at 12:30:15PM +0100, Jens Lehmann wrote:\n>> Am 01.01.2011 21:39, schrieb Oliver Kullmann:\n>>> On Fri, Dec 31, 2010 at 06:42:01PM -0500, Seth Robertson wrote:\n>>> As far as I see that, this doesn't concern the problem how to I update\n>>> one repository with submodules from another repository with \"these\" submodules\n>>> (as the same paths)?\n>>\n>> I'm not sure I completely understand your use case, but submodules are\n>> repositories of their own, so they don't get updated by just pulling\n>> a superproject into another containing the same submodule. The submodule\n>> changes have to be pushed to its own parent repository and can then be\n>> fetched from there into another superproject's submodule.\n> \n> I know that --- but if there wouldn't be any savings possible (in terms of using it),\n> then submodules would be pointless, and so the question is *how* to use them.\n\nNo, they aren't pointless at all. But if you want to collaborate using\nsubmodules they IMO work best if all your coworkers are able to access\nthe same submodule repos you are pushing to. Otherwise you'll have to\ntransport all submodule changes additionally to those of the superproject\n(which might be more of a hassle than not using submodules in the first\nplace). Then you might be better off pulling the modules into your repo\nusing \"git subtree\" or \"gitslave\".\n\nA possibility to put all submodule commits in the object directory of\nthe superproject has been discussed some time ago on this list [1] and\nat the last GitTogether. That might be just what you need, but I am not\naware of any work done in that direction yet.\n\n> The good thing with Git is that there are no central repositories.\n> That's exactly what I want to use, but again and again the automatic\n> assumptions of \"central repositories\" are made, which should be actually alien to Git.\n\nNo, Git works perfectly fine with central repositories too (and that is\na feature :-). But I think I understand where your impression comes from.\nSubmodules don't work very well when you change URLs (that can result\nin forcing your coworkers to do a \"git submodule sync\" in their repo\nevery time they switch to a commit with a changed URL). But while that\nsomehow works not being able to access a submodules repo doesn't work at\nall. So the constraint for submodules is to have a repo which is visible\nfor the people you work with.\n\nBut submodules don't really force you into the centralized model, as you\ncan modify the .gitmodules file e.g. in a downstream fork and let it point\nto your own forked version of the submodules repos where you can do your\nown development independent of the submodules upstream.\n\n> Puuuh, I really really tried hard now to make my use-case clear :-|. Hopefully\n> now the picture emerges.\n\nYep, thanks for sharing that!\n\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/151473/\n"},{"id":"158820","messageId":"20110102175725.GC13358@burratino","threadId":"26170","inReplyTo":"4D20B629.8000107@web.de","subject":"Re: how to update a submodule?","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2011-01-02T17:57:26Z","receivedAt":"2011-01-02T17:57:26Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jens Lehmann wrote:\n\n> Submodules don't work very well when you change URLs (that can result\n> in forcing your coworkers to do a \"git submodule sync\" in their repo\n> every time they switch to a commit with a changed URL).\n\nFor the future, it is probably most convenient to use URLs starting\nwith \"../\" or \"./\", which would rarely need to change.  That can even\nwork with passing around a tarball of bundles and a script to\nreconstitute them, I think.\n\nThanks for some food for thought.\nJonathan\n"}]}