{"thread":{"id":"14722","subject":"git submodules","startedAt":"2008-07-28T16:20:03Z","lastAt":"2008-08-18T00:46:19Z","messageCount":29,"participants":["Pierre Habouzit","Nigel Magnay","Avery Pennarun","Jakub Narebski","Junio C Hamano","Benjamin Collins","Shawn O. Pearce","Petr Baudis","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"85303","messageId":"20080728162003.GA4584@artemis.madism.org","threadId":"14722","inReplyTo":null,"subject":"git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-28T16:20:03Z","receivedAt":"2008-07-28T16:20:03Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"\nWhile trying to sum up some things I'd like submodules to do, and things\nlike that, I came to ask myself why the heck we were doing things the\nway we currently do wrt submodules.\n\nThis question is related to the `.git` directories of submodules. I\nwonder why we didn't chose to use a new reference namespace\n(refs/submodules/$path/$remote/$branch).\n\nThis would have the net benefit that most of the plumbing tasks would be\neasier if they have to deal with submodules, because they aren't in this\nuncomfortable situation where they have to recurse into another git\ndirectory to know what to do.\n\nIt also has the absolutely nice property to share objects, so that\nprojects that replaced a subdirectory with a submodule don't see their\ncheckouts grow too large.\n\nWe probably still want submodules to act like plain independant git\nrepositories, but one can still *fake* that this way: submodules have\nonly a .git/config file (also probably an index and a couple of things\nlike that, but that's almost a different issue for what I'm considering\nnow) that has the setting:\n\n [core]\n submodule = true\n\nThis could make all the builtins look for the real $GIT_DIR up, which in\nturn gives the submodule \"name\". Then, for this submodule, every\nreference, remote name, ... would be virtualized using the\n\"remote/$submodule_name\" prefix. IOW, in a submodule \"some/sub/module\"\nthe branch \"origin/my/topic/branch\" is under:\n refs/submodules/some/sub/module/origin/my/topic/branch\n <-- submod. --><-- submod. --><-- --><-- branch -->\n namespace \t path/name remote\nNote that this doesn't mean that we must rip out .gitmodules, because\nit's needed to help splitting the previous reference name properly, and\nfor bootstrapping purposes.\n\n\nHaving that, one can probably extend most of the porcelains in _very_\nstraightforward ways. For example, a local topic branch `topic` would be\nthe union of the supermodule `topic` branch, and all the\n`refs/submodules/$names/topic` ones.\n\nMost importantly, it would help implementing that tries to make your\nsubmodules stay _on branch_. One irritating problem with submodules, is\nthat when someone else commited, and that you git submodule update,\nyou're on a detached head. Absolutely horrible. If you see your current\nbranch (assume it's master), then when you do that, you would update\nyour `refs/submodules/$name/master` references instead and keep the\nsubmodule HEADs `on branch`. Of course we can _probably_ hack something\ntogether along those lines with the current setup, but it would be _so_\nmuch more convenient this way...\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85305","messageId":"20080728162331.GB4584@artemis.madism.org","threadId":"14722","inReplyTo":"20080728162003.GA4584@artemis.madism.org","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-28T16:23:31Z","receivedAt":"2008-07-28T16:23:31Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Mon, Jul 28, 2008 at 04:20:03PM +0000, Pierre Habouzit wrote:\n> It also has the absolutely nice property to share objects, so that\n> projects that replaced a subdirectory with a submodule don't see their\n> checkouts grow too large.\n\n Especially it \"fixes\" git-new-workdir, which becomes really\ninefficient (storage and typing-wise) when submodules are in use, since\nit doesn't share git_dir's for the submodules (this could also be hacked\ntogether, but again, it's so much more convenient if we have only _one_\ngit_dir per repository that ... oh well)\n\n\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85333","messageId":"320075ff0807281323l51bb6478j30e3e4c490974a70@mail.gmail.com","threadId":"14722","inReplyTo":"20080728162003.GA4584@artemis.madism.org","subject":"Re: git submodules","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-07-28T20:23:39Z","receivedAt":"2008-07-28T20:23:39Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":">\n> While trying to sum up some things I'd like submodules to do, and things\n> like that, I came to ask myself why the heck we were doing things the\n> way we currently do wrt submodules.\n>\n> This question is related to the `.git` directories of submodules. I\n> wonder why we didn't chose to use a new reference namespace\n> (refs/submodules/$path/$remote/$branch).\n>\nI'm maybe being a bit slow - what would be the contents of (say)\nrefs/submodules/moduleA/remotes/origin/master ? The ref\nthat's currently in moduleA/.git/refs/remotes/origin/master ?\n\n> This would have the net benefit that most of the plumbing tasks would be\n> easier if they have to deal with submodules, because they aren't in this\n> uncomfortable situation where they have to recurse into another git\n> directory to know what to do.\n>\n> It also has the absolutely nice property to share objects, so that\n> projects that replaced a subdirectory with a submodule don't see their\n> checkouts grow too large.\n>\n\nAh.. are you meaning that the top-level repository contains all the\ncommits in all the submodules?\n\n> We probably still want submodules to act like plain independant git\n> repositories, but one can still *fake* that this way: submodules have\n> only a .git/config file (also probably an index and a couple of things\n> like that, but that's almost a different issue for what I'm considering\n> now) that has the setting:\n>\n> [core]\n> submodule = true\n>\n> This could make all the builtins look for the real $GIT_DIR up, which in\n> turn gives the submodule \"name\". Then, for this submodule, every\n> reference, remote name, ... would be virtualized using the\n> \"remote/$submodule_name\" prefix. IOW, in a submodule \"some/sub/module\"\n> the branch \"origin/my/topic/branch\" is under:\n> refs/submodules/some/sub/module/origin/my/topic/branch\n> <-- submod. --><-- submod. --><-- --><-- branch -->\n> namespace path/name remote\n> Note that this doesn't mean that we must rip out .gitmodules, because\n> it's needed to help splitting the previous reference name properly, and\n> for bootstrapping purposes.\n>\n\nI was thinking a bit about submodules (because of the earlier\ndiscussions about submodule update only pulling from origin, and the\nassociated difficulties) and started wondering if the best place for\nthe git repository for (say) submoduleA was really\n<...>/submoduleA/.git/<> and not (say) something like\n.git/submodules/submoduleA/<>. This would be nicer for people trying\nto pull revisions from you because they could easily find submodule\nrepositories regardless or not of whether they currently exist in your\nWC.\n\nI got as far as looking at discussions around .gitlink but ran out of\navaiable time.\n\n>\n> Having that, one can probably extend most of the porcelains in _very_\n> straightforward ways. For example, a local topic branch `topic` would be\n> the union of the supermodule `topic` branch, and all the\n> `refs/submodules/$names/topic` ones.\n>\n> Most importantly, it would help implementing that tries to make your\n> submodules stay _on branch_. One irritating problem with submodules, is\n> that when someone else commited, and that you git submodule update,\n> you're on a detached head. Absolutely horrible. If you see your current\n> branch (assume it's master), then when you do that, you would update\n> your `refs/submodules/$name/master` references instead and keep the\n> submodule HEADs `on branch`. Of course we can _probably_ hack something\n> together along those lines with the current setup, but it would be _so_\n> much more convenient this way...\n>\n\nFor me, if I'm on heads/blah in the superproject, I probably want to\nbe on heads/blah in *all* submodules. But that's maybe just me.\n"},{"id":"85337","messageId":"20080728205545.GB10409@artemis.madism.org","threadId":"14722","inReplyTo":"320075ff0807281323l51bb6478j30e3e4c490974a70@mail.gmail.com","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-28T20:55:45Z","receivedAt":"2008-07-28T20:55:45Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Mon, Jul 28, 2008 at 08:23:39PM +0000, Nigel Magnay wrote:\n> >\n> > While trying to sum up some things I'd like submodules to do, and things\n> > like that, I came to ask myself why the heck we were doing things the\n> > way we currently do wrt submodules.\n> >\n> > This question is related to the `.git` directories of submodules. I\n> > wonder why we didn't chose to use a new reference namespace\n> > (refs/submodules/$path/$remote/$branch).\n> >\n> I'm maybe being a bit slow - what would be the contents of (say)\n> refs/submodules/moduleA/remotes/origin/master ? The ref\n> that's currently in moduleA/.git/refs/remotes/origin/master ?\n\n Yes.\n\n> > It also has the absolutely nice property to share objects, so that\n> > projects that replaced a subdirectory with a submodule don't see their\n> > checkouts grow too large.\n> >\n> \n> Ah.. are you meaning that the top-level repository contains all the\n> commits in all the submodules?\n\n Yes. My suggestion is to share all the references, prefixing the\nsubmodules ones with a distinct prefix (namely\nrefs/submodules/$name-of-the-submodule) to avoid any conflict, and share\nthe object store. You get coherent reflogs and stuff like that for free\non top.\n\n> I was thinking a bit about submodules (because of the earlier\n> discussions about submodule update only pulling from origin, and the\n> associated difficulties) and started wondering if the best place for\n> the git repository for (say) submoduleA was really\n> <...>/submoduleA/.git/<> and not (say) something like\n> ..git/submodules/submoduleA/<>. This would be nicer for people trying\n> to pull revisions from you because they could easily find submodule\n> repositories regardless or not of whether they currently exist in your\n> WC.\n\n That too indeed (the \"easier to clone\" bit). OTOH, I don't like the\n.git/submodules idea a lot, if you mean to put a usual $GIT_DIR layout\ninside of it. With what I propose, you find objects for all your\nsuper/sub-modules in the usual store, which eases many things.\nEspecially, I believe that when you replace a subdirectory of a project\nwith a submodule, git-blame could benefit quite a lot from this to be\nable to glue history back through the submodule limits, without having\nto refactor a _lot_ of code: it would merely have to dereference so\ncalled \"gitlinks\" to the commit then tree, hence twice, and just do its\nusual work, with your proposal, we still rely on having to recurse in\nsubdirectories which requires more boilerplate code.\n\n> I got as far as looking at discussions around .gitlink but ran out of\n> avaiable time.\n\n I shall say I never followed them, as I was uninterested with such\nsubjects before, (but now is as I use them at work). But I don't recall\nsuch an idea to have been discussed at all, so...\n\n> > Having that, one can probably extend most of the porcelains in _very_\n> > straightforward ways. For example, a local topic branch `topic` would be\n> > the union of the supermodule `topic` branch, and all the\n> > `refs/submodules/$names/topic` ones.\n> >\n> > Most importantly, it would help implementing that tries to make your\n> > submodules stay _on branch_. One irritating problem with submodules, is\n> > that when someone else commited, and that you git submodule update,\n> > you're on a detached head. Absolutely horrible. If you see your current\n> > branch (assume it's master), then when you do that, you would update\n> > your `refs/submodules/$name/master` references instead and keep the\n> > submodule HEADs `on branch`. Of course we can _probably_ hack something\n> > together along those lines with the current setup, but it would be _so_\n> > much more convenient this way...\n> >\n> \n> For me, if I'm on heads/blah in the superproject, I probably want to\n> be on heads/blah in *all* submodules. But that's maybe just me.\n\n Yes, that's what I tried to say, so if it wasn't clear, it's exactly\nwhat I would like to do/have.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85338","messageId":"20080728205923.GC10409@artemis.madism.org","threadId":"14722","inReplyTo":"20080728205545.GB10409@artemis.madism.org","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-28T20:59:23Z","receivedAt":"2008-07-28T20:59:23Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Mon, Jul 28, 2008 at 08:55:45PM +0000, Pierre Habouzit wrote:\n> On Mon, Jul 28, 2008 at 08:23:39PM +0000, Nigel Magnay wrote:\n> That too indeed (the \"easier to clone\" bit). OTOH, I don't like the\n> .git/submodules idea a lot, if you mean to put a usual $GIT_DIR layout\n> inside of it. With what I propose, you find objects for all your\n> super/sub-modules in the usual store, which eases many things.\n> Especially, I believe that when you replace a subdirectory of a project\n> with a submodule, git-blame could benefit quite a lot from this to be\n> able to glue history back through the submodule limits, without having\n> to refactor a _lot_ of code: it would merely have to dereference so\n> called \"gitlinks\" to the commit then tree, hence twice, and just do its\n> usual work, with your proposal, we still rely on having to recurse in\n> subdirectories which requires more boilerplate code.\n\n And of _course_ this is also true for git-log, which is like 10x as\nimportant for me (like I don't remember if I used git-blame this year,\nwhereas I used git-log in the last 10 minutes ;p)\n\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85343","messageId":"32541b130807281440v64f3cb9ci50cf6d16be4f2f82@mail.gmail.com","threadId":"14722","inReplyTo":"20080728205923.GC10409@artemis.madism.org","subject":"Re: git submodules","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-07-28T21:40:22Z","receivedAt":"2008-07-28T21:40:22Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 7/28/08, Pierre Habouzit wrote:\n> On Mon, Jul 28, 2008 at 08:55:45PM +0000, Pierre Habouzit wrote:\n> > That too indeed (the \"easier to clone\" bit). OTOH, I don't like the\n> > .git/submodules idea a lot, if you mean to put a usual $GIT_DIR layout\n> > inside of it. With what I propose, you find objects for all your\n> > super/sub-modules in the usual store, which eases many things.\n> > Especially, I believe that when you replace a subdirectory of a project\n> > with a submodule, git-blame could benefit quite a lot from this to be\n> > able to glue history back through the submodule limits, without having\n> > to refactor a _lot_ of code: it would merely have to dereference so\n> > called \"gitlinks\" to the commit then tree, hence twice, and just do its\n> > usual work, with your proposal, we still rely on having to recurse in\n> > subdirectories which requires more boilerplate code.\n>\n> And of _course_ this is also true for git-log, which is like 10x as\n> important for me (like I don't remember if I used git-blame this year,\n> whereas I used git-log in the last 10 minutes ;p)\n\nI don't think you're going to get away with *not* having a separate\n.git directory for each submodule. You'll just plain lose almost all\nthe features of submodules if you try to do that.\n\nMost importantly in my case, my submodules (libraries shared between\napps) have a very different branching structure than my supermodules.\nIt wouldn't be particularly meaningful to force them to use the same\nbranch names.\n\nFurther, if you don't have a separate .git directory for each\nsubmodule, you can't *switch* branches on the submodule independently\nof the supermodule in any obvious way. This is also useful; I might\nwant to test updating to the latest master of my submodule, see if it\nstill works with my supermodule, and if so, commit the new gitlink in\nthe supermodule. This is a very common workflow for me.\n\nOn the other hand, your thought about combining the \"git log\" messages\nis quite interesting. That *is* something I'd benefit from, along\nwith being able to git-bisect across submodules. If I'm in the\nsupermodule, I want to see *all* the commits that might have changed\nin my application, not just the ones in the supermodule itself. I\nsuspect this isn't simple at all to implement, however, as you'd have\nto look inside the file tree of a given commit in order to find\nwhether any submodule links have changed in that commit. It's\nunfortunate that submodules involve a commit->tree->commit link\nstructure.\n\n> One irritating problem with submodules, is\n> that when someone else commited, and that you git submodule update,\n> you're on a detached head. Absolutely horrible.\n\nI think that roughly everyone agrees with the above statement by now.\nIt would also be trivial to fix it, if only we knew what \"fix\" means.\nSo far, I haven't seen any good suggestions for what branch name to\nuse automatically in a submodule, and believe me, I've been looking\nfor one :)\n\nHave fun,\n\nAvery\n"},{"id":"85345","messageId":"20080728220308.GF10409@artemis.madism.org","threadId":"14722","inReplyTo":"32541b130807281440v64f3cb9ci50cf6d16be4f2f82@mail.gmail.com","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-28T22:03:08Z","receivedAt":"2008-07-28T22:03:08Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Mon, Jul 28, 2008 at 09:40:22PM +0000, Avery Pennarun wrote:\n> On 7/28/08, Pierre Habouzit wrote:\n> > On Mon, Jul 28, 2008 at 08:55:45PM +0000, Pierre Habouzit wrote:\n> > > That too indeed (the \"easier to clone\" bit). OTOH, I don't like the\n> > > .git/submodules idea a lot, if you mean to put a usual $GIT_DIR layout\n> > > inside of it. With what I propose, you find objects for all your\n> > > super/sub-modules in the usual store, which eases many things.\n> > > Especially, I believe that when you replace a subdirectory of a project\n> > > with a submodule, git-blame could benefit quite a lot from this to be\n> > > able to glue history back through the submodule limits, without having\n> > > to refactor a _lot_ of code: it would merely have to dereference so\n> > > called \"gitlinks\" to the commit then tree, hence twice, and just do its\n> > > usual work, with your proposal, we still rely on having to recurse in\n> > > subdirectories which requires more boilerplate code.\n> >\n> > And of _course_ this is also true for git-log, which is like 10x as\n> > important for me (like I don't remember if I used git-blame this year,\n> > whereas I used git-log in the last 10 minutes ;p)\n> \n> I don't think you're going to get away with *not* having a separate\n> ..git directory for each submodule. You'll just plain lose almost all\n> the features of submodules if you try to do that.\n> \n> Most importantly in my case, my submodules (libraries shared between\n> apps) have a very different branching structure than my supermodules.\n> It wouldn't be particularly meaningful to force them to use the same\n> branch names.\n\nWhy not ? We're talking local branches, that can track whatever you like\non the remote side. Of course, the globing refspec are probably going to\nbe too simple for you if your branching scheme is _that_ different, but\nif you can deal with that by hand _now_, I can't see why writing the\nadequate tracking maps by hand would be any harder.\n\n> Further, if you don't have a separate .git directory for each\n> submodule, you can't *switch* branches on the submodule independently\n> of the supermodule in any obvious way.\n\nYes you can, in what I propose you have a dummy .git in each submodule,\nwith probably an index, a HEAD and a config file (maybe some other\nthings along) to allow that especially.\n\n> This is also useful; I might want to test updating to the latest\n> master of my submodule, see if it still works with my supermodule, and\n> if so, commit the new gitlink in the supermodule. This is a very\n> common workflow for me.\n\nI agree.\n\n> It's unfortunate that submodules involve a commit->tree->commit link\n> structure.\n\nActually it's not a big problem, you just have to \"dereference\" twice\ninstead of one, and be prepared to the fact that the second dereference\nmay fail (because you miss some objects). I instead believe that\ngitlinks are a good idea.\n\n> > One irritating problem with submodules, is\n> > that when someone else commited, and that you git submodule update,\n> > you're on a detached head. Absolutely horrible.\n> \n> I think that roughly everyone agrees with the above statement by now.\n> It would also be trivial to fix it, if only we knew what \"fix\" means.\n> So far, I haven't seen any good suggestions for what branch name to\n> use automatically in a submodule, and believe me, I've been looking\n> for one :)\n\nWell, using the same as the supermodule is probably the less confusing\nway. Of course, not being in the \"same\" branch as the supermodule would\nclearly be a case of your tree being \"dirty\", and it would prevent a\n\"git checkout\" to work in the very same way that git checkout doesn't\nwork if you have locally modified files.\n\nIf your submodule branching layout uses the same names as the\nsupermodule branches then yes, it's going to hurt, but I believe it to\nbe unlikely (else you would become insane just trying to remember what\nyou are doing ;p). So even if say your work in `master` in your\nsupermodule, but for some reason use what is on the remote\n`release/10.2` branch for a given submodule, nothing prevents you to\nhave a local `master` branch in that submodule as well that tracks\n`release/10.2`. It's actually a quite sensible thing to do I believe.\n\nIt doesn't prevent you from switching your submodule to the `devel`\nbranch to perform some tests, or even make it the new state of the\nsubmodule inside your supermodule. This operation would just then move\nwhat is `master` in your submodule to track `devel` instead of\n`release/10.2`[0].\n\nI fail to see which current submodules features you would lose with this\nscheme. In fact, said differently, I more or less propose that when you\ncommit your supermodule state (providing some checks hold see [0]) it\nupdates the 'associated' branch in the submodule. The goal here, is that\nyou're always on a branch, set up to track a given remote branch, that\nwe can check against so that we can:\n\n (1) avoid the usual caveats of git-submodules (one can check when\n pushing the supermodule that all submodules are indeed pushed in\n the remote branch they track);\n\n (2) user can commit and don't bother resetting branches and doing\n tricks to be \"on branch again\".\n\nI don't think it prevents you from e.g. having topic branches in your\nsubmodules, provided that before commiting a new submodule change, you\nsomehow merge those in the \"matching\" branch that was set up for you.\n\n\n\n [0] of course we probably want to refuse such a thing if `devel` isn't\n a fast-forward from release/10.2. But that's not the point of the\n explanation so I skipped this bit for clarity of my point.\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85346","messageId":"m3r69dtzm9.fsf@localhost.localdomain","threadId":"14722","inReplyTo":"20080728220308.GF10409@artemis.madism.org","subject":"Re: git submodules","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-07-28T22:26:33Z","receivedAt":"2008-07-28T22:26:33Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Pierre Habouzit writes:\n> On Mon, Jul 28, 2008 at 09:40:22PM +0000, Avery Pennarun wrote:\n\n> > Further, if you don't have a separate .git directory for each\n> > submodule, you can't *switch* branches on the submodule independently\n> > of the supermodule in any obvious way.\n> \n> Yes you can, in what I propose you have a dummy .git in each submodule,\n> with probably an index, a HEAD and a config file (maybe some other\n> things along) to allow that especially.\n\nWhat you are (re)inventing here is something called gitlink (.git which\nis a file, or .gitlink file); not to be confused with 'sumbodule'/'commit'\nentry in a tree which is sometimes called gitlink. Alternate idea was\n'unionfs' like \"shadowing\" .git, with 'core.gitdir' in .git/config\n(which would contain .git/HEAD and .git/index, and all missing files\nand config would be taken from `core.gitdir').\n\nThere was even some preliminary implementation IIRC, but AFAIR it\nwas abandoned because of no \"real usage\".\n\nSee\n http://permalink.gmane.org/gmane.comp.version-control.msysgit/1868\n http://permalink.gmane.org/gmane.comp.version-control.git/72449\n http://permalink.gmane.org/gmane.comp.version-control.git/72457\n http://permalink.gmane.org/gmane.comp.version-control.git/72296\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"85348","messageId":"32541b130807281532v3eed94ebv8037247618e9bd55@mail.gmail.com","threadId":"14722","inReplyTo":"20080728220308.GF10409@artemis.madism.org","subject":"Re: git submodules","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-07-28T22:32:54Z","receivedAt":"2008-07-28T22:32:54Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 7/28/08, Pierre Habouzit wrote:\n> > It's unfortunate that submodules involve a commit->tree->commit link\n> > structure.\n>\n> Actually it's not a big problem, you just have to \"dereference\" twice\n> instead of one, and be prepared to the fact that the second dereference\n> may fail (because you miss some objects). I instead believe that\n> gitlinks are a good idea.\n\nIt's actually complicated to generate the log, however. To be 100%\naccurate in creating a combined log of the supermodule and submodule,\nyou'd have to check *for each supermodule commit* whether there were\nany changes in gitlinks. And gitlinks might move around between\nrevisions, so you can't just look up a particular path in each\nrevision; you have to traverse the entire tree. And you can't just\nlook at the start and end supermodule commits to see if the gitlinks\nchanged; they might have changed and then changed back, which is quite\nrelevant to log messages.\n\nProbably it's more useful to just commit the git-shortlog of the\nsubmodule whenever you update the gitlink. It won't work with bisect,\nexactly, but that's less important than generally having an idea of\nwhat happened by reading the log. ISTR somenoe submitted a\ngit-submodule patch for that already somewhere, but I've been known to\nimagine things.\n\n> Well, using the same [branch] as the supermodule is probably the less confusing\n> way. Of course, not being in the \"same\" branch as the supermodule would\n> clearly be a case of your tree being \"dirty\", and it would prevent a\n> \"git checkout\" to work in the very same way that git checkout doesn't\n> work if you have locally modified files.\n>\n> If your submodule branching layout uses the same names as the\n> supermodule branches then yes, it's going to hurt, but I believe it to\n> be unlikely (else you would become insane just trying to remember what\n> you are doing ;p).\n\nI think this is much more common than you think. An easy example is\nthat I'm developing a new version of my application in the\nsupermodule's \"master\", but it relies on a released version of my\nsubmodule, definitely not the experimental \"master\" version. Using\nyour logic, the local branch of the submodule would be called master,\nbut wouldn't correspond at all to the remote submodule's master.\n\nI believe such a situation would be even worse than no branch at all.\nIt could lead to people pushing/pulling all sorts of bad things from\nthe wrong places. At least right now, people become confused and ask\nfor help instead of becoming confused and making a mess.\n\nHave fun,\n\nAvery\n"},{"id":"85350","messageId":"7vfxptpr76.fsf@gitster.siamese.dyndns.org","threadId":"14722","inReplyTo":"m3r69dtzm9.fsf@localhost.localdomain","subject":"Re: git submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-28T22:41:17Z","receivedAt":"2008-07-28T22:41:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jakub Narebski writes:\n\n> Pierre Habouzit writes:\n>> On Mon, Jul 28, 2008 at 09:40:22PM +0000, Avery Pennarun wrote:\n>\n>> > Further, if you don't have a separate .git directory for each\n>> > submodule, you can't *switch* branches on the submodule independently\n>> > of the supermodule in any obvious way.\n>> \n>> Yes you can, in what I propose you have a dummy .git in each submodule,\n>> with probably an index, a HEAD and a config file (maybe some other\n>> things along) to allow that especially.\n>\n> What you are (re)inventing here is something called gitlink (.git which\n> is a file, or .gitlink file); not to be confused with 'sumbodule'/'commit'\n> entry in a tree which is sometimes called gitlink....\n> ...\n> There was even some preliminary implementation IIRC, but AFAIR it\n> was abandoned because of no \"real usage\".\n\nI am afraid you are confused. I think you are talking about \"gitfile\",\nnot \"gitlink\".\n\nIt is not abandoned; see e.g. read_gitfile_gently() in setup.c.\n\nI suspect the use of it may help the use case Pierre proposes, but its\nmain attractiveness as I understood it back when we discussed the facility\nwas that you could switch branches between 'maint' that did not have a\nsubmodule at \"path\" back then, and 'master' that does have one now,\nwithout losing the submodule repository. When checking out 'master' (and\nthat would probably mean you would update 'git-submodule init' and\n'git-submodule update' implementation), you would instanciate subdirectory\n\"path\", create \"path/.git\" that is such a regular file that that points at\nsomewhere inside the $GIT_DIR of superproject (say \".git/submodules/foo\").\nBy storing refs and object store are all safely away in the superproject\n$GIT_DIR, you can now safely switch back to 'maint', which would involve\nmaking sure there is no local change that will be lost and then removing\nthe \"path\" and everything underneath it.\n"},{"id":"85426","messageId":"20080728231220.GA22625@artemis.madism.org","threadId":"14722","inReplyTo":"32541b130807281532v3eed94ebv8037247618e9bd55@mail.gmail.com","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-28T23:12:20Z","receivedAt":"2008-07-28T23:12:20Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Mon, Jul 28, 2008 at 10:32:54PM +0000, Avery Pennarun wrote:\n> On 7/28/08, Pierre Habouzit wrote:\n> > > It's unfortunate that submodules involve a commit->tree->commit link\n> > > structure.\n> >\n> > Actually it's not a big problem, you just have to \"dereference\" twice\n> > instead of one, and be prepared to the fact that the second dereference\n> > may fail (because you miss some objects). I instead believe that\n> > gitlinks are a good idea.\n> \n> It's actually complicated to generate the log, however. To be 100%\n> accurate in creating a combined log of the supermodule and submodule,\n> you'd have to check *for each supermodule commit* whether there were\n> any changes in gitlinks. And gitlinks might move around between\n> revisions, so you can't just look up a particular path in each\n> revision; you have to traverse the entire tree. And you can't just\n> look at the start and end supermodule commits to see if the gitlinks\n> changed; they might have changed and then changed back, which is quite\n> relevant to log messages.\n\nI'm pretty clueless about how git-log works, but I fail to see how this\nis harder than following file moves e.g. Of course it's more expensive\nthan git log, but it shouldn't really be more expensive than\n`git log -M -C -C` already is.\n\n> > Well, using the same [branch] as the supermodule is probably the less confusing\n> > way. Of course, not being in the \"same\" branch as the supermodule would\n> > clearly be a case of your tree being \"dirty\", and it would prevent a\n> > \"git checkout\" to work in the very same way that git checkout doesn't\n> > work if you have locally modified files.\n> >\n> > If your submodule branching layout uses the same names as the\n> > supermodule branches then yes, it's going to hurt, but I believe it to\n> > be unlikely (else you would become insane just trying to remember what\n> > you are doing ;p).\n> \n> I think this is much more common than you think. An easy example is\n> that I'm developing a new version of my application in the\n> supermodule's \"master\", but it relies on a released version of my\n> submodule, definitely not the experimental \"master\" version. Using\n> your logic, the local branch of the submodule would be called master,\n> but wouldn't correspond at all to the remote submodule's master.\n\nProbably indeed, otoh the \"remote\" (assume it's origin) master state is\nstored in \"origin/master\", not \"master\".\n\n> I believe such a situation would be even worse than no branch at all.\n> It could lead to people pushing/pulling all sorts of bad things from\n> the wrong places. At least right now, people become confused and ask\n> for help instead of becoming confused and making a mess.\n\nIndeed. But that's only a name issue, I'm sure we can come up with\nsomething decent. What I (we ?) want is actually a way to make\ngit-checkout/git-reset work so that when you switch branches (reset\n--hard to a previous state) you remain on branch, because human brains\nusually don't remember those silly detached HEADs commits sha1 well ;)\nThe problem is, `the branch I'm on in my submodule when the supermodule\nis on $foo` is a quite local information. But that's really what we\nwould like to remember so that when you git checkout somewhere and git\ncheckout master back, submodules switches branch accordingly.\n\nSaying that, I realize that we probably _really_ want to name submodule\nbranches the same as the supermodule ones, but should manage to find UIs\nthat don't confuse users wrt the fact that it may be disconnected from\nthe remote branch nameing.\n\nI reckon that for my use, I would not have those problems, because we\nhave this kind of layout:\n\nlib-foo/ <-- submodule to share the foo library.\nlib-bar/ <-- submodule to share the bar library.\napp-frotz/ <-- the frotz product\n\nIn another repository we have app-quux and the same submodules, and so\non.\n\nWe always try that our `master`s (where devel happens unlike git.git ;p)\nuse the most recent `master` versions from the submodules. And the\nstable branches (IOW the software that we sold and released and for\nwhich we provide support), have branches named $product/$version. We\nhave $product/$version branches in those submodules as well. If we have\na bug that needs a patch in say, lib-foo, then we push the patch into a\ntopic branch, that we merge in all the $product/$version that need it,\nand into master as well. For such a setup that I believe to be sane,\nthen well, corresponding names fit the job perfectly.\n\nOf course if one of your submodule is git.git where the not too unstable\ncode lives in `next` and not `master` and another one is one of my\nproject at work, where not too unstable code lives in `master` then\nindeed you're somehow screwed because indeed the whole `master` concept\nwould be quite confusing. But honestly, I don't think it's less\nconfusing with the current way submodules work either.\n\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85416","messageId":"b3889dff0807282251t7096a8c9wf477cf4495749d34@mail.gmail.com","threadId":"14722","inReplyTo":"32541b130807281440v64f3cb9ci50cf6d16be4f2f82@mail.gmail.com","subject":"Re: git submodules","fromName":"Benjamin Collins","fromEmail":"aggieben@gmail.com","sentAt":"2008-07-29T05:51:31Z","receivedAt":"2008-07-29T05:51:31Z","isPatch":false,"sender":{"key":"aggieben@gmail.com","avatar":"https://gravatar.com/avatar/28fe19679c681bd9e42efa676b04199512aab92373ebc94e67537e14663b8d8a?d=mp&s=160"},"body":"On Mon, Jul 28, 2008 at 4:40 PM, Avery Pennarun wrote:\n> Most importantly in my case, my submodules (libraries shared between\n> apps) have a very different branching structure than my supermodules.\n> It wouldn't be particularly meaningful to force them to use the same\n> branch names.\n>\n> Further, if you don't have a separate .git directory for each\n> submodule, you can't *switch* branches on the submodule independently\n> of the supermodule in any obvious way. This is also useful; I might\n> want to test updating to the latest master of my submodule, see if it\n> still works with my supermodule, and if so, commit the new gitlink in\n> the supermodule. This is a very common workflow for me.\n\nI second this sentiment. I happen to very much *like* the fact that\nthe coupling between submodules and their super-projects is minimal.\nThe flexibility this allows is very useful. Of course, it brings to\nmind the comment Stroustrup once made about C++ blowing off your whole\nleg.\n\n> On the other hand, your thought about combining the \"git log\" messages\n> is quite interesting. That *is* something I'd benefit from, along\n> with being able to git-bisect across submodules. If I'm in the\n> supermodule, I want to see *all* the commits that might have changed\n> in my application, not just the ones in the supermodule itself. I\n> suspect this isn't simple at all to implement, however, as you'd have\n> to look inside the file tree of a given commit in order to find\n> whether any submodule links have changed in that commit. It's\n> unfortunate that submodules involve a commit->tree->commit link\n> structure.\n\nLet my contrariness begin...\nI can see how someone might find such a feature in \"git log\" useful,\nbut I don't think I would. I have 3 submodules in my project right\nnow, and I don't always want to see the changes. Most of the time, I\ndon't care, actually. When I do care, I can search the output of \"git\nlog\" for commits that touch the path where my submodule lives (through\nGitk, usually), and I can open another Gitk for details.\n\nAs for \"git bisect\": I haven't done this and I'm too busy to try to\ncontrive something for the purposes of this email, but wouldn't it\nbasically already do what you want? Seems that you'd just run \"git\nsubmodule update\" after each step of the bisect.\n\n>\n> > One irritating problem with submodules, is\n> > that when someone else commited, and that you git submodule update,\n> > you're on a detached head. Absolutely horrible.\n>\n> I think that roughly everyone agrees with the above statement by now.\n> It would also be trivial to fix it, if only we knew what \"fix\" means.\n> So far, I haven't seen any good suggestions for what branch name to\n> use automatically in a submodule, and believe me, I've been looking\n> for one :)\n\nI disagree with this completely. I think the detached head is\nactually fantastic because it tells you all the right things:\na) the branch your submodule is on is ultimately irrelevant\nb) it reminds you that this is not your project. It's part of your\nproject managed in a special way by Git, but your project is in ..\nc) if you want to do work in this part of your project that comes from\nsomewhere else, you need to be thoughtful about how you manage its\nbranches.\n\nI try to keep all my submodules on (no branch) as much as possible.\nIn a way, I feel like that kind of relieves me of the chore of keeping\nmapping superproject branches to submodule branches in my head.\n\nI pretty much support submodules as they are, with the exception of\nwanting \"git submodule update\" to be executed automatically at times.\n"},{"id":"85419","messageId":"20080729060449.GG11947@spearce.org","threadId":"14722","inReplyTo":"b3889dff0807282251t7096a8c9wf477cf4495749d34@mail.gmail.com","subject":"Re: git submodules","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-07-29T06:04:49Z","receivedAt":"2008-07-29T06:04:49Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Benjamin Collins wrote:\n> On Mon, Jul 28, 2008 at 4:40 PM, Avery Pennarun wrote:\n> >\n> > > One irritating problem with submodules, is\n> > > that when someone else commited, and that you git submodule update,\n> > > you're on a detached head. Absolutely horrible.\n> >\n> > I think that roughly everyone agrees with the above statement by now.\n> > It would also be trivial to fix it, if only we knew what \"fix\" means.\n> > So far, I haven't seen any good suggestions for what branch name to\n> > use automatically in a submodule, and believe me, I've been looking\n> > for one :)\n> \n> I disagree with this completely. I think the detached head is\n> actually fantastic [...]\n\nDitto with Benjamin. Detached head is a fantastic idea.\n\n> [...] because it tells you all the right things:\n> a) the branch your submodule is on is ultimately irrelevant\n> b) it reminds you that this is not your project. It's part of your\n> project managed in a special way by Git, but your project is in ..\n> c) if you want to do work in this part of your project that comes from\n> somewhere else, you need to be thoughtful about how you manage its\n> branches.\n> \n> I try to keep all my submodules on (no branch) as much as possible.\n> In a way, I feel like that kind of relieves me of the chore of keeping\n> mapping superproject branches to submodule branches in my head.\n\nAt my former day-job we wrote our own \"git submodule\" in our\nbuild system before gitlink was available in the core, let alone\ngit-submodule was a Porcelain command.\n\nMany developers who were new to Git found having a sea of 11 Git\nrepositories+working directories in a single build area difficult to\nmanage. They quickly found the detached HEAD feature in a submodule\nto be a really handy way to know if they made changes there or not.\n\nMost of our developers also modified __git_ps1() in their bash\ncompletion to use `git name-rev HEAD` to try and pick up a remote\nbranch name when on a detached HEAD. This slowed down their bash\nprompts a little bit, but they found that \"origin/foo\" hint very\nvaluable to let them know they should start a new branch before\nmaking changes.\n\nSo I'm just echoing what Benjamin said above, only we did it\nindependently, and came to the same conclusion.\n\n-- \nShawn.\n"},{"id":"85430","messageId":"320075ff0807290118o62a6fc1eq3e90e32ef7783a17@mail.gmail.com","threadId":"14722","inReplyTo":"20080729060449.GG11947@spearce.org","subject":"Re: git submodules","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-07-29T08:18:12Z","receivedAt":"2008-07-29T08:18:12Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":">> I try to keep all my submodules on (no branch) as much as possible.\n>> In a way, I feel like that kind of relieves me of the chore of keeping\n>> mapping superproject branches to submodule branches in my head.\n>\n> At my former day-job we wrote our own \"git submodule\" in our\n> build system before gitlink was available in the core, let alone\n> git-submodule was a Porcelain command.\n>\n> Many developers who were new to Git found having a sea of 11 Git\n> repositories+working directories in a single build area difficult to\n> manage. They quickly found the detached HEAD feature in a submodule\n> to be a really handy way to know if they made changes there or not.\n>\n> Most of our developers also modified __git_ps1() in their bash\n> completion to use `git name-rev HEAD` to try and pick up a remote\n> branch name when on a detached HEAD. This slowed down their bash\n> prompts a little bit, but they found that \"origin/foo\" hint very\n> valuable to let them know they should start a new branch before\n> making changes.\n>\n> So I'm just echoing what Benjamin said above, only we did it\n> independently, and came to the same conclusion.\n>\n\nHm.\nMy developers are (mostly) on windows, so \"altering PS1\" or even\nwriting \"shell scripts\" is way beyond them. They want it to \"just\nwork\" (where their previous experience is SVN superprojects with\nmultiple svn:externals). I have a hard time justifying the experience\nthat if we're all working on master, then as soon as Joe Q developer\ndoes 'submodule update' then poof - his heads are disconnected.\n\nThat said, I do also like the flexibility that having the superproject\non heads/foo and a submodule on heads/bar as it allows you to\nintegration test divergent submodule branches (indeed our CI system\nautomatically picks them up and tries all possible combinations).\n"},{"id":"85431","messageId":"20080729082135.GB32312@artemis.madism.org","threadId":"14722","inReplyTo":"b3889dff0807282251t7096a8c9wf477cf4495749d34@mail.gmail.com","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-29T08:21:35Z","receivedAt":"2008-07-29T08:21:35Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Tue, Jul 29, 2008 at 05:51:31AM +0000, Benjamin Collins wrote:\n> I try to keep all my submodules on (no branch) as much as possible.\n> In a way, I feel like that kind of relieves me of the chore of keeping\n> mapping superproject branches to submodule branches in my head.\n\n Why would _you_ map them to superproject branches ? I mean it's pretty\nmuch Git's matter. In fact, maybe calling them branches is not a\nbrilliant idea, what I would like would probably rather be some kind of\nreflog like thing, but with one reflog per submodule and supermodule\nbranch.\n\n I agree with you than when you don't have to do any change in the\nsubmodule, detached HEADs just work. But when you often have to push\nfixes in, it's a nightmare. Instead of just having to:\n\n $EDITOR mysubmodule/file.c\n git commit mysubmodule/file.c # that would ideally do a commit in the\n # submodule then update the submodule\n # state in the supermodule\n\n git submodule mysubmodule push origin HEAD # push the submodule mysubmodule\n # changes to the appropriate branch\n git push origin HEAD\n\n You have to:\n\n cd submodule\n git branch -D master\n git checkout -b master\n git commit file.c\n cd ..\n git commit submodule\n cd submodule\n git push origin HEAD:remote/branch/we/want/to/push/to\n cd ..\n git push origin HEAD\n\n *phew*\n\n I'm sorry but this is nowhere near a good UI. Of course the detached\nhead *currently* prevents you to shoot yourself in the foot, because\nsubmodules are _that_ dangerous. But those also are tedious to work\nwith, like a lot, which makes currently our answer to big projects \"do\nnot have GB-big repositories, split them in submodules\" a bad joke,\nbecause their ergonomy is nowhere near what you have with a monolithic\nrepository yet.\n\n\n I'm trying to see what to do better. I believe we _need_ those things:\n\n * a way to name the successive states of the submodule, a branch looks\n like a good idea, but maybe we can \"invent\" some different idea so\n that it looks and tastes like a branch, but is more automagic in the\n sense that it's just a prettier name than a sha1.\n\n This would allow to inspect the submodule history using \n `gitk $this_name`. The $PS1 thing is nice, but you have to cd into\n the submodule to see where it currently lives. So you rather need\n something else.\n\n\n * a way to remember where you want to push changes you do in the\n submodule to. That's a bit like branch tracking, but not quite. This\n is required so that we can (and I strongly believe we want that in\n the end) make many porcelain commands act on the full\n (super+sub)modules in a unified way, somehow hiding the submodules\n boundaries.\n\n For example, git commit file1 file2 file3 ... would do the\n submodules commits if any, and then the supermodule one. Alternatively, if\n you have e.g.:\n\n $ git add mysubmodule/file1.c\n $ git add superfile.c\n $ git add mysubmodule # tell the supermodule we want to commit what\n # is in the submodule index at the same time\n $ git commit\n\n Then if you run:\n\n $ git push # fails complaining that mysubmodule is\n # not pushed\n\n $ git submodule mysubmodule push\n $ git push # works\n\n\n * What you \"track\" must be a per supermodule branch thing, so that if\n you do things like that:\n\n # you are in master in the supermodule with non pushed commits in\n # the submodule\n\n <.. oh crap there is a bug in the supermodule that I need to fix in\n the production branch..>\n\n $ git checkout production # would checkout in the submodule what\n # matches\n $ $EDITOR mysubmodule/something\n $ git commit !$\n $ ..push everything..\n\n <.. okay let's now go back to master ..>\n\n $ git checkout master\n\n <... hack hack hack to finish the current WIP ...>\n <... okay we're ready to merge production in ...>\n\n $ git merge production # will DWYM with the submodules, IOW merge the\n # `production` state into the current `master` one.\n $ git sm mysubmodule push\n $ git push\n\n\n Try to write the same workflow with the current submodules, you'll\n end up with a script at least 3 times as long, because you would\n need to do everything by hand, including switching submodules,\n naming temporary branches in them so that you can work decently and\n perform the merges and so on.\n\n\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85435","messageId":"20080729083755.GC32312@artemis.madism.org","threadId":"14722","inReplyTo":"20080729082135.GB32312@artemis.madism.org","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-29T08:37:55Z","receivedAt":"2008-07-29T08:37:55Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On mar, jui 29, 2008 at 08:21:35 +0000, Pierre Habouzit wrote:\n> * a way to remember where you want to push changes you do in the\n> submodule to. That's a bit like branch tracking, but not quite. This\n> is required so that we can (and I strongly believe we want that in\n> the end) make many porcelain commands act on the full\n> (super+sub)modules in a unified way, somehow hiding the submodules\n> boundaries.\n\n> \n> \n> * What you \"track\" must be a per supermodule branch thing, so that if\n> you do things like that:\n\n\n In fact, nowhere I used the name of the current submodule branch in my\nexamples, so maybe we don't really need it. What we need though, is a\nway to tell where the submodules are pushed to, IO what they (try to)\ntrack remotely, IOW of which remote reference they should always be a\nparent.\n\n Such an information is probably to be put in .gitmodules, this way,\nyou have the per-supermodule-branch setting I would like to see. And\nthen one would not care about the submodules be in a detached HEAD\nbecause I believe those scenarii work well:\n\n * If you do no changes in the submodules, all just works like it does\n now.\n\n * If your only work in the submodule is to refresh its state to the\n tip of what it currently track, then well, we probably want a git\n submodule command for that, and no further ado is done.\n\n * If you just want a simple fix to go in the submodule, work from your\n supermodule, as if there was no submodule. git-commit your changes\n (which with a submodule aware git-commit would be transparent), then\n you can push your work. And in the worst case scenario where you\n cannot push because it's not a fast forward, you would fetch, merge\n and push again.\n\n You don't really need a name for the submodule, even if you want to\n reset to the state before the merge because you screwed it, as\n basically, git-reset would _also_ be submodule aware and DWYM\n without an explicit reference for the submodule.\n\n * If you have heavy works in the submodule, then you probably will\n setup many submodule topic branches, work inside of it like you\n already do, and the extra step of reattaching the HEAD somewhere is\n not as bad as it is with (3) as it's a tiny overhead compared to all\n you're going to do with your topic branches.\n\n\nSo okay, let's scratch this \"automatic reference\" thing, I see its\nlimits now, so what about having a .gitmodule entry look like:\n\n [submodule \"$path\"]\n\tpath = \"$path\"\n\turl = git://somewhere/\n\ttracks = master\n\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85437","messageId":"20080729084530.GD32312@artemis.madism.org","threadId":"14722","inReplyTo":"320075ff0807290118o62a6fc1eq3e90e32ef7783a17@mail.gmail.com","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-29T08:45:30Z","receivedAt":"2008-07-29T08:45:30Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Tue, Jul 29, 2008 at 08:18:12AM +0000, Nigel Magnay wrote:\n> >> I try to keep all my submodules on (no branch) as much as possible.\n> >> In a way, I feel like that kind of relieves me of the chore of keeping\n> >> mapping superproject branches to submodule branches in my head.\n> >\n> > At my former day-job we wrote our own \"git submodule\" in our\n> > build system before gitlink was available in the core, let alone\n> > git-submodule was a Porcelain command.\n> >\n> > Many developers who were new to Git found having a sea of 11 Git\n> > repositories+working directories in a single build area difficult to\n> > manage. They quickly found the detached HEAD feature in a submodule\n> > to be a really handy way to know if they made changes there or not.\n> >\n> > Most of our developers also modified __git_ps1() in their bash\n> > completion to use `git name-rev HEAD` to try and pick up a remote\n> > branch name when on a detached HEAD. This slowed down their bash\n> > prompts a little bit, but they found that \"origin/foo\" hint very\n> > valuable to let them know they should start a new branch before\n> > making changes.\n> >\n> > So I'm just echoing what Benjamin said above, only we did it\n> > independently, and came to the same conclusion.\n> >\n> \n> Hm.\n> My developers are (mostly) on windows, so \"altering PS1\" or even\n> writing \"shell scripts\" is way beyond them.\n\n More importantly, you don't have all your submodule states in your PS1\nso this argument is already moot for *nix users as well.\n\n> They want it to \"just work\" (where their previous experience is SVN\n> superprojects with multiple svn:externals). I have a hard time\n> justifying the experience that if we're all working on master, then as\n> soon as Joe Q developer does 'submodule update' then poof - his heads\n> are disconnected.\n\n Well, maybe it's not as hard, maybe what we lack are just submodule\naware porcelains (I mean we lack those for sure, but maybe it's also\nthe _only_ thing we miss to have a better user experience, and I begin\nto believe it).\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85438","messageId":"20080729085125.GJ32184@machine.or.cz","threadId":"14722","inReplyTo":"20080729083755.GC32312@artemis.madism.org","subject":"Re: git submodules","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-07-29T08:51:26Z","receivedAt":"2008-07-29T08:51:26Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Tue, Jul 29, 2008 at 10:37:55AM +0200, Pierre Habouzit wrote:\n> So okay, let's scratch this \"automatic reference\" thing, I see its\n> limits now, so what about having a .gitmodule entry look like:\n> \n> [submodule \"$path\"]\n\nThis is not a \"$path\" but arbitrary string. Please keep that in mind.\n\n> \tpath = \"$path\"\n> \turl = git://somewhere/\n> \ttracks = master\n\nI do like this (well, I'd just name it \"branch\" instead of \"tracks\").\nI use submodules very \"traditionally\" just to bind external projects of\ncertain version to my project, but I have been already thinking about\nimplementing this merely as a hint for others to know what branch should\nthe other developers follow when updating the submodule to a newer\nversion.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nAs in certain cults it is possible to kill a process if you know\nits true name. -- Ken Thompson and Dennis M. Ritchie\n"},{"id":"85456","messageId":"alpine.DEB.1.00.0807291413460.4631@eeepc-johanness","threadId":"14722","inReplyTo":"20080729085125.GJ32184@machine.or.cz","subject":"Re: git submodules","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-29T12:15:05Z","receivedAt":"2008-07-29T12:15:05Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 29 Jul 2008, Petr Baudis wrote:\n\n> On Tue, Jul 29, 2008 at 10:37:55AM +0200, Pierre Habouzit wrote:\n> \n> > \tpath = \"$path\"\n> > \turl = git://somewhere/\n> > \ttracks = master\n> \n> I do like this (well, I'd just name it \"branch\" instead of \"tracks\"). I \n> use submodules very \"traditionally\" just to bind external projects of \n> certain version to my project, but I have been already thinking about \n> implementing this merely as a hint for others to know what branch should \n> the other developers follow when updating the submodule to a newer \n> version.\n\nAs long as you only use it in \"submodule status\" to say what the relation \nof the current revision is with respect to the \"tracks\" branch...\n\nBut then, how does the relation to the currently _committed_ state get \ndisplayed?\n\nCiao,\nDscho\n"},{"id":"85466","messageId":"20080729130713.GF32312@artemis.madism.org","threadId":"14722","inReplyTo":"alpine.DEB.1.00.0807291413460.4631@eeepc-johanness","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-29T13:07:14Z","receivedAt":"2008-07-29T13:07:14Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Tue, Jul 29, 2008 at 12:15:05PM +0000, Johannes Schindelin wrote:\n> On Tue, 29 Jul 2008, Petr Baudis wrote:\n> > On Tue, Jul 29, 2008 at 10:37:55AM +0200, Pierre Habouzit wrote:\n> > \n> > > \tpath = \"$path\"\n> > > \turl = git://somewhere/\n> > > \ttracks = master\n[...]\n> But then, how does the relation to the currently _committed_ state get \n> displayed?\n\nHmm _that's_ why you need a name for it. Or you need the submodule to be\naware he's one, and then one would have some kind of \"magic\" word to\nname this sha1. And tools would find out in the supermodule what it\ntranslates into. I don't have any briliant idea for a proposal\n(COMMITED_HEAD is clearly too long ;p, BASE is not very explicit, ...)\nbut someone will have one.\n\nIdeally gitk would show some litle tag for this state too.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85467","messageId":"alpine.DEB.1.00.0807291511400.4631@eeepc-johanness","threadId":"14722","inReplyTo":"20080729130713.GF32312@artemis.madism.org","subject":"Re: git submodules","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-07-29T13:15:10Z","receivedAt":"2008-07-29T13:15:10Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 29 Jul 2008, Pierre Habouzit wrote:\n\n> On Tue, Jul 29, 2008 at 12:15:05PM +0000, Johannes Schindelin wrote:\n> > On Tue, 29 Jul 2008, Petr Baudis wrote:\n> > > On Tue, Jul 29, 2008 at 10:37:55AM +0200, Pierre Habouzit wrote:\n> > > \n> > > > \tpath = \"$path\"\n> > > > \turl = git://somewhere/\n> > > > \ttracks = master\n> [...]\n> > But then, how does the relation to the currently _committed_ state get \n> > displayed?\n> \n> Hmm _that's_ why you need a name for it.\n\nI do not understand. We are talking about three different things here:\n\n1) the committed state of the submodule\n2) the local state of the submodule\n3) the state of the \"tracks\" branch\n\nWe always have 1) and we have 2) _iff_ the submodule was checked out. We \nonly will have 3) if \"tracks\" is set in .git/config (for consistency's \nsake, we should not read that information directly from the .gitmodules \nfile, but let the user override it in .git/config after \"submodule init\".\n\n> Or you need the submodule to be aware he's one, and then one would have \n> some kind of \"magic\" word to name this sha1. And tools would find out in \n> the supermodule what it translates into.\n\nYou lost me there.\n\nCiao,\nDscho\n"},{"id":"85471","messageId":"20080729131908.GG32312@artemis.madism.org","threadId":"14722","inReplyTo":"alpine.DEB.1.00.0807291511400.4631@eeepc-johanness","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-29T13:19:08Z","receivedAt":"2008-07-29T13:19:08Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Tue, Jul 29, 2008 at 01:15:10PM +0000, Johannes Schindelin wrote:\n> Hi,\n> \n> On Tue, 29 Jul 2008, Pierre Habouzit wrote:\n> \n> > On Tue, Jul 29, 2008 at 12:15:05PM +0000, Johannes Schindelin wrote:\n> > > On Tue, 29 Jul 2008, Petr Baudis wrote:\n> > > > On Tue, Jul 29, 2008 at 10:37:55AM +0200, Pierre Habouzit wrote:\n> > > > \n> > > > > \tpath = \"$path\"\n> > > > > \turl = git://somewhere/\n> > > > > \ttracks = master\n> > [...]\n> > > But then, how does the relation to the currently _committed_ state get \n> > > displayed?\n\n> > Or you need the submodule to be aware he's one, and then one would have \n> > some kind of \"magic\" word to name this sha1. And tools would find out in \n> > the supermodule what it translates into.\n> \n> You lost me there.\n\n Then I didn't understand your question.\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"85472","messageId":"320075ff0807290631l1f9a1e70jcb73bde7e2c86000@mail.gmail.com","threadId":"14722","inReplyTo":"alpine.DEB.1.00.0807291511400.4631@eeepc-johanness","subject":"Re: git submodules","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-07-29T13:31:23Z","receivedAt":"2008-07-29T13:31:23Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":"> I do not understand. We are talking about three different things here:\n>\n> 1) the committed state of the submodule\n> 2) the local state of the submodule\n> 3) the state of the \"tracks\" branch\n>\n> We always have 1) and we have 2) _iff_ the submodule was checked out. We\n> only will have 3) if \"tracks\" is set in .git/config (for consistency's\n> sake, we should not read that information directly from the .gitmodules\n> file, but let the user override it in .git/config after \"submodule init\".\n>\n\nI think the implication is that .gitconfig states \"I'm expecting that\nsubmodule X will always be tracking branch name 'Y'\" and that you\nwouldn't ever override it in .git/config. If you then switched\nsubmodule X to branch Z, then committed the superproject, that commit\nwould contain a change to .gitconfig also (to say I'm expecting to\ntrack Z rather than X') ?\n\n>> Or you need the submodule to be aware he's one, and then one would have\n>> some kind of \"magic\" word to name this sha1. And tools would find out in\n>> the supermodule what it translates into.\n>\n> You lost me there.\n>\n\nThis sounds like it relates to the problem that what I call X and Z,\nyou might call Bibble and Bobble; you could use some kind of SHA1 in\nlieu of a textual name to make sure everyone was talking about the\nsame thing ?\n\n> Ciao,\n> Dscho\n>\n>\n"},{"id":"85475","messageId":"20080729144914.GI32312@artemis.madism.org","threadId":"14722","inReplyTo":"320075ff0807290631l1f9a1e70jcb73bde7e2c86000@mail.gmail.com","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-07-29T14:49:14Z","receivedAt":"2008-07-29T14:49:14Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Tue, Jul 29, 2008 at 01:31:23PM +0000, Nigel Magnay wrote:\n> > I do not understand. We are talking about three different things here:\n> >\n> > 1) the committed state of the submodule\n> > 2) the local state of the submodule\n> > 3) the state of the \"tracks\" branch\n> >\n> > We always have 1) and we have 2) _iff_ the submodule was checked out. We\n> > only will have 3) if \"tracks\" is set in .git/config (for consistency's\n> > sake, we should not read that information directly from the .gitmodules\n> > file, but let the user override it in .git/config after \"submodule init\".\n> >\n> \n> I think the implication is that .gitconfig states \"I'm expecting that\n> submodule X will always be tracking branch name 'Y'\" and that you\n> wouldn't ever override it in .git/config. If you then switched\n> submodule X to branch Z, then committed the superproject, that commit\n> would contain a change to .gitconfig also (to say I'm expecting to\n> track Z rather than X') ?\n\n Yes, tracks branch in .git/config doesn't fly. Or you need a\nbranch.$supermodule_branch.$submodule_name.tracks setting (oh god!)\n"},{"id":"85476","messageId":"7v7ib4ivxn.fsf@gitster.siamese.dyndns.org","threadId":"14722","inReplyTo":"320075ff0807290631l1f9a1e70jcb73bde7e2c86000@mail.gmail.com","subject":"Re: git submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-07-29T14:53:08Z","receivedAt":"2008-07-29T14:53:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Nigel Magnay\" writes:\n\n>> I do not understand. We are talking about three different things here:\n>>\n>> 1) the committed state of the submodule\n>> 2) the local state of the submodule\n>> 3) the state of the \"tracks\" branch\n>>\n>> We always have 1) and we have 2) _iff_ the submodule was checked out. We\n>> only will have 3) if \"tracks\" is set in .git/config (for consistency's\n>> sake, we should not read that information directly from the .gitmodules\n>> file, but let the user override it in .git/config after \"submodule init\".\n>\n> I think the implication is that .gitconfig states \"I'm expecting that\n> submodule X will always be tracking branch name 'Y'\" and that you\n> wouldn't ever override it in .git/config. If you then switched\n> submodule X to branch Z, then committed the superproject, that commit\n> would contain a change to .gitconfig also (to say I'm expecting to\n> track Z rather than X') ?\n\nYou are right. I think letting the user override with .git/config is a\ngood idea, but it shouldn't be \".gitmodules may say X or whatever, but I\nwant to use Y\".\n\nInstead, it should be more like \"On branches where .gitmodules says X, I\nwant to use Y.\"\n\nThis comment actually applies to the existing override of .gitmodules item\nwith .git/config (I think I've been saying it since the design phase).\n"},{"id":"87457","messageId":"20080817201336.GA17148@artemis","threadId":"14722","inReplyTo":"7vfxptpr76.fsf@gitster.siamese.dyndns.org","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-08-17T20:13:36Z","receivedAt":"2008-08-17T20:13:36Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Mon, Jul 28, 2008 at 10:41:17PM +0000, Junio C Hamano wrote:\n> I suspect the use of it may help the use case Pierre proposes, but its\n> main attractiveness as I understood it back when we discussed the facility\n> was that you could switch branches between 'maint' that did not have a\n> submodule at \"path\" back then, and 'master' that does have one now,\n> without losing the submodule repository. When checking out 'master' (and\n> that would probably mean you would update 'git-submodule init' and\n> 'git-submodule update' implementation), you would instanciate subdirectory\n> \"path\", create \"path/.git\" that is such a regular file that that points at\n> somewhere inside the $GIT_DIR of superproject (say \".git/submodules/foo\").\n> By storing refs and object store are all safely away in the superproject\n> $GIT_DIR, you can now safely switch back to 'maint', which would involve\n> making sure there is no local change that will be lost and then removing\n> the \"path\" and everything underneath it.\n\ngitfiles looks nifty for sure, though I've thought about it a bit, and\nI'm not sure if we don't want something a bit more powerful, though\nstill in the same vein.\n\nIf we look at submodules, I quite believe that we would benefit a lot\nfrom sharing the object directory accross the supermodule and all its\nsubmodules, because of the following reasons:\n\n * It could make things like git-blame better: at work, it's common for\n us to move files across submodules: we have a stable library shared\n accross projects, and move there C modules that have staged for\n quite some time in the applications and are stable enough, and it's\n pity to loose history then, whereas git could really guess about the\n move if it sees through GITLINKS in the same object repository.\n GITLINKS are not very different from trees actually if you can look\n through them, it's just a matter of dereferencing twice instead of\n once.\n\n * For people that have made a subdirectory become a submodule (and\n it's also something that can happen) it's likely that lots of blobs\n are shared. It would end up taking less disk space.\n\n * It helps people fixing situations where they pushed a supermodule\n with a substate that never existed without seeing it. Since the\n object store is shared, this commit that actually never existed will\n never ever be pruned, and at _least_ one person on earth will never\n lose it. With detached heads everywhere it's very easy to not name a\n detached head, and have it pruned at some point.\n\n * I _believe_ (just a hunch) that it helps knowing if it's possible to\n perform a \"recursive\" (wrt submodules) checkout/reset/$whatever,\n without having to spawn subcommands and quite unpleasant similar\n stuff.\n\n\nThough we would not like to have submodules suffer from reachability\nissues after a prune in the supermodule. That means that all references\nand reflogs of the submodules shall be accessible from the supermodule\nso that everything that could mess with the object store by removing\nobjects cannot remove interesting objects (that should limit the code\npaths to really seldom places actually).\n\n\nSo what I've thinked about was to extend gitfiles so that it can also\ndefine where to find not only the git_dir but also the object store.\nMoving the current \"faked symlink\" approach to a less terse file looking\nlike a standard git-config one. E.g.:\n\n [gitfile]\n\tgit_dir = some/path/.git/submodules/foo/\n objects = some/path/.git/objects\n # why not other settings in the future ?\n\nThis part is quite easy and straightforward (and it can be done while\nkeeping backward compatibility with the current way gitfiles work).\nWhat I can't decide is how we deal with the reflogs and references. I\nsee two choices. Assuming the submodules git_dir's are under the\nsupermodule $GIT_DIR/submodules/$name_of_the_super_module/:\n\n (1) we do nothing more.\n\n (2) we melt the submodules reflogs and references into the supermodule\n ones with appropriate namespacing. For example, for a submodule\n named \"foo/bar\" we would have its reflogs live in the supermodule\n .git/logs/submodules/foo/bar/logs/* and its references under\n .git/refs/submodules/foo/bar/refs/*. For that we add 'logs =' and\n 'refs =' to the gitfile.\n\nThe first approach need us to be able to somehow recurse under\n.git/submodules to understand what inside that looks like a git_dir, and\nteach reachability commands to look at the refs inside them. It can be\nquite a lot of work, especially since we can have submodules inside\nsubmodules at some point.\n\nThe second approach has the net benefit that no pruning command has to\nbe modified to work. Many commands that we want to act on the global\nrepository will just work. Though, we have to fix a couple of issues\ntoo:\n (1) be able to have a references directory that is not .git/refs. I\n looked at the source, I believe only 3 or 4 places in the C code\n have to be fixed for that to work, maybe a bit more in the shell\n commands, but that should be fairly easy.\n\n (2) it will break reference packing, because the submodules won't see\n the supermodule packed-refs file, and we will probably have to\n draft a new packed-refs thingy because of this issue. A simple\n possibility I see is to move packed-refs as refs/.packed-refs (as\n a starting dot cannot be a reference name). Then teach\n git-pack-refs to generate a .packed-refs each time it crosses a\n 'refs/' directory name, and finally learn how to load those (and\n no it won't require to recurse into the whole refs/, we can mark\n in the toplevel refs/.packed-refs that it has submodules and that\n there is a .packed-refs under\n refs/submodules/foo/bar/refs/.packed-refs and avoid the costly\n recursion).\n\n (3) we will have to teach for_each_ref to skip \"/submodules\",\n which is I believe fairly easy.\n\n\nI personnaly like the second approach better because it will scale\nbetter (I believe) when people will do submodules into submodules into\nsubmodules. But I'm unsure if it's too disruptive or not.\n\nSo .. comments thoughts remarks are welcomed :)\n\n\n\nNote: with enhanced gitfiles, and making workdirs use gitfiles, with any\n of those approaches, it's easy to make workdirs that won't have\n the \"if we repack we may lose things referenced from other\n workdir's reflogs\" problem anymore. Which is kind of a nifty side\n effect ;)\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"},{"id":"87500","messageId":"32541b130808171554o2b2f33d5q43d3bd517ed85e06@mail.gmail.com","threadId":"14722","inReplyTo":"20080817201336.GA17148@artemis","subject":"Re: git submodules","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-08-17T22:54:51Z","receivedAt":"2008-08-17T22:54:51Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Sun, Aug 17, 2008 at 4:13 PM, Pierre Habouzit wrote:\n> * It could make things like git-blame better: at work, it's common for\n> us to move files across submodules: we have a stable library shared\n> accross projects, and move there C modules that have staged for\n> quite some time in the applications and are stable enough, and it's\n> pity to loose history then, whereas git could really guess about the\n> move if it sees through GITLINKS in the same object repository.\n> GITLINKS are not very different from trees actually if you can look\n> through them, it's just a matter of dereferencing twice instead of\n> once.\n\nThat would be cool. I expect you could implement it independently of\neverything else by simply *trying* to dereference gitlinks in the\nlocal object repository if they exist, and not erroring out if they\ndon't.\n\nThe other reasons for combining the repos seem fine, but they mostly\nseem to come down to saving disk space. I like saving disk space, but\nit's not really that important to me.\n\nHave fun,\n\nAvery\n"},{"id":"87502","messageId":"7v1w0np7d4.fsf@gitster.siamese.dyndns.org","threadId":"14722","inReplyTo":"20080817201336.GA17148@artemis","subject":"Re: git submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-08-17T23:08:39Z","receivedAt":"2008-08-17T23:08:39Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Pierre Habouzit writes:\n\n> On Mon, Jul 28, 2008 at 10:41:17PM +0000, Junio C Hamano wrote:\n>\n>> I suspect the use of it may help the use case Pierre proposes, but its\n>> main attractiveness as I understood it back when we discussed the facility\n>> was that you could switch branches between 'maint' that did not have a\n>> submodule at \"path\" back then, and 'master' that does have one now,\n>> without losing the submodule repository. When checking out 'master' (and\n>> that would probably mean you would update 'git-submodule init' and\n>> 'git-submodule update' implementation), you would instanciate subdirectory\n>> \"path\", create \"path/.git\" that is such a regular file that that points at\n>> somewhere inside the $GIT_DIR of superproject (say \".git/submodules/foo\").\n>> By storing refs and object store are all safely away in the superproject\n>> $GIT_DIR, you can now safely switch back to 'maint', which would involve\n>> making sure there is no local change that will be lost and then removing\n>> the \"path\" and everything underneath it.\n>\n> gitfiles looks nifty for sure, though I've thought about it a bit, and\n> I'm not sure if we don't want something a bit more powerful, though\n> still in the same vein.\n>\n> If we look at submodules, I quite believe that we would benefit a lot\n> from sharing the object directory accross the supermodule and all its\n> submodules, because of the following reasons:\n\nI know there are cases where sharing object store is useful. Being able\nto share is one thing. Always having to share, without any other option,\nis another.\n\nUsing gitlink to keep the true repository data out of submodule checkout\narea so that branch switching can safely be done is orthogonal to the\nissue of how repositories of submodules and the superproject share their\nobject store. IOW, you would always use gitlink to solve the \"branch\nswitching may make the submodule checkout disappear\" issue, while you\ncould use alternates mechanism (or direct symlinking of $GIT_DIR/objects)\nacross these repositories *if* you want to share their object store.\n\n> Though we would not like to have submodules suffer from reachability\n> issues after a prune in the supermodule. That means that all references\n> and reflogs of the submodules shall be accessible from the supermodule\n> so that everything that could mess with the object store by removing\n> objects cannot remove interesting objects (that should limit the code\n> paths to really seldom places actually).\n\nI do not think this issue is limited to use of submodules. I'd imagine\nthat if you build this reachability protection into the alternates\nmechanism, you would automatically solve both \"multiple checkout of the\nsame project, via git-new-workdir\" issue as well as \"submodules that share\nits objects with the superproject\" issue.\n\nWhich leads me to conclude, at least for now, that it would not be a good\nidea to make this related to gitfile in any way. Object sharing between\nequal repositories (aka new-workdir) does not use gitfile, but it still\nneeds to have the same kind of reachability protection.\n"},{"id":"87508","messageId":"20080818004619.GD17148@artemis","threadId":"14722","inReplyTo":"7v1w0np7d4.fsf@gitster.siamese.dyndns.org","subject":"Re: git submodules","fromName":"Pierre Habouzit","fromEmail":"madcoder@debian.org","sentAt":"2008-08-18T00:46:19Z","receivedAt":"2008-08-18T00:46:19Z","isPatch":false,"sender":{"key":"madcoder@debian.org","avatar":"https://avatars.githubusercontent.com/u/44708?v=4"},"body":"On Sun, Aug 17, 2008 at 11:08:39PM +0000, Junio C Hamano wrote:\n> I know there are cases where sharing object store is useful. Being able\n> to share is one thing. Always having to share, without any other option,\n> is another.\n> \n> Using gitlink to keep the true repository data out of submodule checkout\n> area so that branch switching can safely be done is orthogonal to the\n> issue of how repositories of submodules and the superproject share their\n> object store. IOW, you would always use gitlink to solve the \"branch\n> switching may make the submodule checkout disappear\" issue, while you\n> could use alternates mechanism (or direct symlinking of $GIT_DIR/objects)\n> across these repositories *if* you want to share their object store.\n\n Fair enough. Though I'm not only interested into the branch switching\nissue. I'm seeing a bit farther, like in having many commands working as\nif there is no submodule involved. And having the same object store for\nall {sup,super}modules helps a lot. For example, there is probably quite\nsome plumbing to write if we have separate object stores if we expect\n(and frankly I do) to have git-commit work across submodule boundaries\n(doing what it should, IOW commit in the submodules, and then commit in\nthe supermodule).\n\n But maybe it's not as hard as it looks.\n\n> > Though we would not like to have submodules suffer from reachability\n> > issues after a prune in the supermodule. That means that all references\n> > and reflogs of the submodules shall be accessible from the supermodule\n> > so that everything that could mess with the object store by removing\n> > objects cannot remove interesting objects (that should limit the code\n> > paths to really seldom places actually).\n> \n> I do not think this issue is limited to use of submodules. I'd imagine\n> that if you build this reachability protection into the alternates\n> mechanism, you would automatically solve both \"multiple checkout of the\n> same project, via git-new-workdir\" issue as well as \"submodules that share\n> its objects with the superproject\" issue.\n> \n> Which leads me to conclude, at least for now, that it would not be a good\n> idea to make this related to gitfile in any way. Object sharing between\n> equal repositories (aka new-workdir) does not use gitfile, but it still\n> needs to have the same kind of reachability protection.\n\n Well somehow the repository that is the alternate (or the symlink, but\nthe latter isn't very windows friendly, not to mention vfat) has to know\nabout the other repository:\n * index ;\n * references ;\n * reflogs.\n\n Which means that alternates users have to register into the provider,\nwhich seems to be _usually_ brittle. I mean, for the current way of\nhow git-new-workdir works, if you register workdirs into the real\nrepository, if you just rename this workdir at some point, or move it to\nsome other place, you're screwed, silentely.\n\n If instead you force this workdir to use a gitfile, you _don't need_\nto register your workdir in the \"real\" repository, because all the data\nbelongs to the \"real\" repository. The workdir is just a \"detached\"\nworkdir, with only the checkout stuff, no index, no references no\nnothing. And if you move this workdir to a new place, it still works.\nOnly the central repository should not budge, which is already a\nlimitation of current workdirs and alternates anyways.\n\n Of course with submodules it's less of an issue since those arent as\nloosely coupled to the supermodule as workdir are to the main\nrepository, and it's unlikely that a submodule will move very often ;)\n\n\n-- \n·O· Pierre Habouzit\n··O madcoder@debian.org\nOOO http://www.madism.org\n"}]}