{"thread":{"id":"13323","subject":"Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","startedAt":"2008-04-30T04:08:00Z","lastAt":"2008-05-08T01:13:33Z","messageCount":22,"participants":["Tim Harper","Andreas Ericsson","Johannes Schindelin","Avery Pennarun","Ping Yin","Roman Shaposhnik","Finn Arne Gangstad","Steven Grimm"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"75627","messageId":"8B885217-8C18-417E-8F11-BB6661792CD3@gmail.com","threadId":"13323","inReplyTo":null,"subject":"Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Tim Harper","fromEmail":"timcharper@gmail.com","sentAt":"2008-04-30T04:08:00Z","receivedAt":"2008-04-30T04:08:00Z","isPatch":false,"sender":{"key":"timcharper@gmail.com","avatar":"https://gravatar.com/avatar/1a2e0c06c7862ff065ee6b1d53195333a5a0577c040ecb2856a150d8e0b00ecd?d=mp&s=160"},"body":"OVERVIEW:\nOn the Git TextMate Bundle, I've automated a lot of the submodule  \ncommands to make them not a terrible pain to work with. (don't get me  \nwrong - I really like the git submodule implementation, I just don't  \nlike how hard it is to work with).\n\n\"WARTS\" WITH EXISTING IMPLEMENTATION\n1) The submodule stays in the working copy when changing to a branch  \nthat does not have a submodule.  This can break a build and cause  \nproblems.  To work around, I have to delete the folder completely (git- \nclean).  Then, when I switch back to the branch again, I have to re- \ndownload the submodule.\n2) I have to type \"git checkout branch && git submodule init && git  \nsubmodule update\" to be sure that I really have the whole contents of  \nthe branch.  That's 3 commands, and a lot of typing.\n3) If I don't run \"git submodule update\", and carelessly run \"git  \ncommit -a\" or \"git add .\", I risk propagating a submodule version from  \nanother branch or undoing an important change.\n\nSUGGESTED ALGORITHM (AS HAS BEEN IMPLEMENTED IN THE GIT TEXTMATE BUNDLE)\nWhen pulling / merging / changing branches:\n1) cache all submodules to ~/.git/submodules_cache\n    a) move from the working directory to a folder that is a MD5 hex- \nhash of both the submodule path and the submodule url\n2) execute the pull / merge / branch change\n3) restore all defined submodules to ~/.git/submodules_cache (only the  \nsubmodules that are still defined after the merge / change / pull)\n4) execute git submodule init && git submodule update\n\n\nPITFALLS:\npitfall)\nIf you commit a change on a submodule that's not on a branch, auto- \nupdating submodules will make it difficult to revive that change.\n\nworkaround)\nDon't allow the user to commit unless they are on a branch.\n\n... couldn't think of anymore.  Anyone?\n\nCONCLUSION\nSo far, this algorithm holds up well in my use cases, and has made  \nsubmodule management seamless for me (I don't have to know that I'm  \nworking with submodules).  It's resolved every one of the above  \noutlined interface warts.\n\nWould it be a good idea to build this algorithm into git?  What would  \nbe the best approach?  Am I completely overlooking something by  \ndesigning the Git TextMate bundle to handle submodules this way?\n\nThanks,\nTim\n"},{"id":"75631","messageId":"EC4DE72E-665B-44C9-A85C-C12C26B00C3A@gmail.com","threadId":"13323","inReplyTo":"8B885217-8C18-417E-8F11-BB6661792CD3@gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Tim Harper","fromEmail":"timcharper@gmail.com","sentAt":"2008-04-30T04:47:25Z","receivedAt":"2008-04-30T04:47:25Z","isPatch":false,"sender":{"key":"timcharper@gmail.com","avatar":"https://gravatar.com/avatar/1a2e0c06c7862ff065ee6b1d53195333a5a0577c040ecb2856a150d8e0b00ecd?d=mp&s=160"},"body":"clarification:\n\nSUGGESTED ALGORITHM (AS HAS BEEN IMPLEMENTED IN THE GIT TEXTMATE BUNDLE)\nWhen pulling / merging / changing branches:\n1) cache all submodules to .git/submodules_cache\n-  a) move the submodule to .git/submodules_cache/`echo  \n\"$submodule_path\\n$submodule_url\" | md5`\n2) execute the pull / merge / branch change\n3) restore all defined submodules from .git/submodules_cache to their  \nrespective working directory locations (only the submodules that are  \nstill defined after the merge / change / pull)\n4) execute git \"submodule init && git submodule update\"\n\n\nOn Apr 29, 2008, at 10:08 PM, Tim Harper wrote:\n\n> OVERVIEW:\n> On the Git TextMate Bundle, I've automated a lot of the submodule  \n> commands to make them not a terrible pain to work with. (don't get  \n> me wrong - I really like the git submodule implementation, I just  \n> don't like how hard it is to work with).\n>\n> \"WARTS\" WITH EXISTING IMPLEMENTATION\n> 1) The submodule stays in the working copy when changing to a branch  \n> that does not have a submodule.  This can break a build and cause  \n> problems.  To work around, I have to delete the folder completely  \n> (git-clean).  Then, when I switch back to the branch again, I have  \n> to re-download the submodule.\n> 2) I have to type \"git checkout branch && git submodule init && git  \n> submodule update\" to be sure that I really have the whole contents  \n> of the branch.  That's 3 commands, and a lot of typing.\n> 3) If I don't run \"git submodule update\", and carelessly run \"git  \n> commit -a\" or \"git add .\", I risk propagating a submodule version  \n> from another branch or undoing an important change.\n>\n> SUGGESTED ALGORITHM (AS HAS BEEN IMPLEMENTED IN THE GIT TEXTMATE  \n> BUNDLE)\n> When pulling / merging / changing branches:\n> 1) cache all submodules to ~/.git/submodules_cache\n>   a) move from the working directory to a folder that is a MD5 hex- \n> hash of both the submodule path and the submodule url\n> 2) execute the pull / merge / branch change\n> 3) restore all defined submodules to ~/.git/submodules_cache (only  \n> the submodules that are still defined after the merge / change / pull)\n> 4) execute git submodule init && git submodule update\n>\n>\n> PITFALLS:\n> pitfall)\n> If you commit a change on a submodule that's not on a branch, auto- \n> updating submodules will make it difficult to revive that change.\n>\n> workaround)\n> Don't allow the user to commit unless they are on a branch.\n>\n> ... couldn't think of anymore.  Anyone?\n>\n> CONCLUSION\n> So far, this algorithm holds up well in my use cases, and has made  \n> submodule management seamless for me (I don't have to know that I'm  \n> working with submodules).  It's resolved every one of the above  \n> outlined interface warts.\n>\n> Would it be a good idea to build this algorithm into git?  What  \n> would be the best approach?  Am I completely overlooking something  \n> by designing the Git TextMate bundle to handle submodules this way?\n>\n> Thanks,\n> Tim\n"},{"id":"75641","messageId":"48180E2D.6020907@op5.se","threadId":"13323","inReplyTo":"8B885217-8C18-417E-8F11-BB6661792CD3@gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2008-04-30T06:14:05Z","receivedAt":"2008-04-30T06:14:05Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Tim Harper wrote:\n> OVERVIEW:\n> On the Git TextMate Bundle, I've automated a lot of the submodule \n> commands to make them not a terrible pain to work with. (don't get me \n> wrong - I really like the git submodule implementation, I just don't \n> like how hard it is to work with).\n> \n> \"WARTS\" WITH EXISTING IMPLEMENTATION\n> 1) The submodule stays in the working copy when changing to a branch \n> that does not have a submodule.  This can break a build and cause \n> problems.  To work around, I have to delete the folder completely \n> (git-clean).  Then, when I switch back to the branch again, I have to \n> re-download the submodule.\n> 2) I have to type \"git checkout branch && git submodule init && git \n> submodule update\" to be sure that I really have the whole contents of \n> the branch.  That's 3 commands, and a lot of typing.\n> 3) If I don't run \"git submodule update\", and carelessly run \"git commit \n> -a\" or \"git add .\", I risk propagating a submodule version from another \n> branch or undoing an important change.\n> \n> SUGGESTED ALGORITHM (AS HAS BEEN IMPLEMENTED IN THE GIT TEXTMATE BUNDLE)\n> When pulling / merging / changing branches:\n> 1) cache all submodules to ~/.git/submodules_cache\n>    a) move from the working directory to a folder that is a MD5 hex-hash \n> of both the submodule path and the submodule url\n> 2) execute the pull / merge / branch change\n> 3) restore all defined submodules to ~/.git/submodules_cache (only the \n> submodules that are still defined after the merge / change / pull)\n> 4) execute git submodule init && git submodule update\n> \n> \n> PITFALLS:\n> pitfall)\n> If you commit a change on a submodule that's not on a branch, \n> auto-updating submodules will make it difficult to revive that change.\n> \n> workaround)\n> Don't allow the user to commit unless they are on a branch.\n> \n> ... couldn't think of anymore.  Anyone?\n> \n> CONCLUSION\n> So far, this algorithm holds up well in my use cases, and has made \n> submodule management seamless for me (I don't have to know that I'm \n> working with submodules).  It's resolved every one of the above outlined \n> interface warts.\n> \n> Would it be a good idea to build this algorithm into git?  What would be \n> the best approach?  Am I completely overlooking something by designing \n> the Git TextMate bundle to handle submodules this way?\n> \n\nI don't use submodules at the moment, but I have several \"lib-ish\" pieces\nof code that would benefit greatly from becoming submodules. The not-exactly\nseamlessness of them has so far been a hindrance though, but it sounds as\nif your changes (assuming they don't affect anything else) should make lessen\nthe submodule headaches somewhat.\n\nSo, where be the patches?\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n"},{"id":"75667","messageId":"alpine.DEB.1.00.0804301121240.17469@eeepc-johanness","threadId":"13323","inReplyTo":"8B885217-8C18-417E-8F11-BB6661792CD3@gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-04-30T10:31:31Z","receivedAt":"2008-04-30T10:31:31Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 29 Apr 2008, Tim Harper wrote:\n\n> 1) The submodule stays in the working copy when changing to a branch \n>    that does not have a submodule.  This can break a build and cause \n>    problems.  To work around, I have to delete the folder completely \n>    (git-clean).  Then, when I switch back to the branch again, I have to \n>    re-download the submodule.\n\nThe problem, of course, is that you can easily have valuable, but \nnot-tracked, files in there.  Deleting the submodule is therefore no \noption.\n\n> 2) I have to type \"git checkout branch && git submodule init && git \n>    submodule update\" to be sure that I really have the whole contents of \n>    the branch.  That's 3 commands, and a lot of typing.\n\nThere is no way around \"checkout branch\", and I think that is a good \nthing.\n\nBut once you did \"submodule init\", you will never need to run it again, \nsince it edits your .git/config, which does not change when switching \nbranches.\n\nAnd as for \"submodule update\", I like the fact that the submodule is not \nupdated automatically.  For example, when I actively develop a submodule, \nbut have to rebase the superproject, I would _hate_ it if the submodule \nwass updated.\n\nThe whole idea about submodules is that they are repositories of their own \nright, and therefore the superproject should not mess with them, _unless_ \nexplicitely asked to, with \"submodule update\".\n\n> 3) If I don't run \"git submodule update\", and carelessly run \"git commit \n>    -a\"  or \"git add .\", I risk propagating a submodule version from \n>    another branch or undoing an important change.\n\ngit commit -a is something that might make sense for newbies, but you \nreally should learn to use git add -p and commit without -a.\n\n> SUGGESTED ALGORITHM (AS HAS BEEN IMPLEMENTED IN THE GIT TEXTMATE BUNDLE)\n> When pulling / merging / changing branches:\n> 1) cache all submodules to ~/.git/submodules_cache\n>   a) move from the working directory to a folder that is a MD5 hex-hash of\n> both the submodule path and the submodule url\n> 2) execute the pull / merge / branch change\n> 3) restore all defined submodules to ~/.git/submodules_cache (only the\n> submodules that are still defined after the merge / change / pull)\n> 4) execute git submodule init && git submodule update\n> \n> \n> PITFALLS:\n> pitfall)\n> If you commit a change on a submodule that's not on a branch, auto-updating\n> submodules will make it difficult to revive that change.\n> \n> workaround)\n> Don't allow the user to commit unless they are on a branch.\n> \n> ... couldn't think of anymore.  Anyone?\n\nI do not like that.  I think that the user should be responsible to take \ncare of the up-to-dateness of the submodule.  As far as the superproject \nis concerned, it just keeps track of the committed submodule state, but \ndoes not enforce it.\n\nFWIW I restarted on my --ignore-submodules patch, which can be seen here:\n\nhttp://repo.or.cz/w/git/dscho.git?a=shortlog;h=refs/heads/my-next\n\nIt lacks tests, therefore it has not been submitted for review yet.\n\nThe basic idea is that not only \"checkout\" ignores the _contents_ of the \nsubmodules, but optionally \"diff\", and using that option \"stash\" and \n\"rebase\".\n\nIOW you can \"stash apply\" even if the submodule is dirty.  Which makes \nsense, given that \"stash\" succeeds, too...\n\nCiao,\nDscho\n"},{"id":"75691","messageId":"32541b130804300947s6083156etc6514cc13c24af13@mail.gmail.com","threadId":"13323","inReplyTo":"alpine.DEB.1.00.0804301121240.17469@eeepc-johanness","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-30T16:47:47Z","receivedAt":"2008-04-30T16:47:47Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/30/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n>  On Tue, 29 Apr 2008, Tim Harper wrote:\n>\n>  > 1) The submodule stays in the working copy when changing to a branch\n>  >    that does not have a submodule.  This can break a build and cause\n>  >    problems.  To work around, I have to delete the folder completely\n>  >    (git-clean).  Then, when I switch back to the branch again, I have to\n>  >    re-download the submodule.\n>\n> The problem, of course, is that you can easily have valuable, but\n>  not-tracked, files in there.  Deleting the submodule is therefore no\n>  option.\n\nI agree that blindly deleting isn't a good choice.  For example, what\nif I've checked stuff into the submodule but never pushed it anywhere?\n Ugh.\n\nThat said, it makes submodule folders act completely inconsistently\nwith normal folders, which is highly undesirable.  The current\nbehaviour strongly encourages me to avoid submodules when I would\notherwise like to use them, just to keep the rest of my team members\n(who are not git experts) from going insane.\n\n>  But once you did \"submodule init\", you will never need to run it again,\n>  since it edits your .git/config, which does not change when switching\n>  branches.\n\nNot true.  If \".gitmodules\" is different between branches, then\n.git/config will have the wrong information.  I think this was the\nreason for the \"read .gitmodules directly and don't worry about\n.git/config\" discussion/patches earlier.\n\n>  And as for \"submodule update\", I like the fact that the submodule is not\n>  updated automatically.  For example, when I actively develop a submodule,\n>  but have to rebase the superproject, I would _hate_ it if the submodule\n>  wass updated.\n\nWhy?  Every other folder in your entire project gets updated when you\n\"git checkout\".  Why are submodules different?  I can personally vouch\nthat this is confusing for almost everyone I've seen who tried it.\n\n>  The whole idea about submodules is that they are repositories of their own\n>  right, and therefore the superproject should not mess with them, _unless_\n>  explicitely asked to, with \"submodule update\".\n\nI think perhaps this is the root of the problem; you're thinking of\nthe superproject as (A) \"a tool for helping me track multiple\nsubprojects automatically\", while I think most people (or at least I)\nthink of submodules as (B) \"folders that have their own series of\ncommits and thus can be shared between projects.\"\n\nIf you think of it as (A), it's inconvenient to have them shuffling\naround automatically.  But if you're in the (B) camp, having it not\nupdate automatically seems insane.\n\nI actually don't know any use cases for (A).  If my app depends on a\nlibrary in a submodule, for example, and I switch to a branch of my\napp that uses a different version of the submodule, why would I ever\n*not* want the submodule to switch too?  If it doesn't, it'll probably\nbreak the compile.\n\n>  > 3) If I don't run \"git submodule update\", and carelessly run \"git commit\n>  >    -a\"  or \"git add .\", I risk propagating a submodule version from\n>  >    another branch or undoing an important change.\n>\n> git commit -a is something that might make sense for newbies, but you\n>  really should learn to use git add -p and commit without -a.\n\nThis is a matter of opinion; pretty much every other major VCS does\nthe equivalent of \"git commit -a\" when you commit, and most people\ndon't seem to mind.  It is therefore possible that git is the weird\none, not every other VCS.\n\nIn particular, in this case what git is doing is, \"You checked out a\nnew branch, but I don't really trust you, so I actually made this\nmodification in your tree for you.  Now if you commit, you can check\nin that change that I think you meant to check in.\"  Wouldn't you\nexpect \"git checkout branch; git commit -a\" on a clean tree to never\ncommit anything?\n\n>  > PITFALLS:\n>  > pitfall)\n>  > If you commit a change on a submodule that's not on a branch, auto-updating\n>  > submodules will make it difficult to revive that change.\n>  >\n>  > workaround)\n>  > Don't allow the user to commit unless they are on a branch.\n>  >\n>  > ... couldn't think of anymore.  Anyone?\n\nWe had some discussion on the list earlier about having submodule\ncheckouts automatically acquire a branch name, so that commits don't\nget lost as easily.  I was going to think about this more and\neventually submit a patch, but I haven't gotten to it yet.  Anyway,\nthe idea is that you have a branch by default, so that you don't end\nup in the useless situation of not being on a branch, which encourages\nchecking in without being on a branch, in the first place.\n\nHave fun,\n\nAvery\n"},{"id":"75694","messageId":"46dff0320804301021t59740377vece412ea71d67cf4@mail.gmail.com","threadId":"13323","inReplyTo":"32541b130804300947s6083156etc6514cc13c24af13@mail.gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-30T17:21:16Z","receivedAt":"2008-04-30T17:21:16Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Thu, May 1, 2008 at 12:47 AM, Avery Pennarun <apenwarr@gmail.com> wrote:\n> On 4/30/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n>  >  On Tue, 29 Apr 2008, Tim Harper wrote:\n\n>\n>  Not true.  If \".gitmodules\" is different between branches, then\n>  .git/config will have the wrong information.  I think this was the\n>  reason for the \"read .gitmodules directly and don't worry about\n>  .git/config\" discussion/patches earlier.\n\nI tried hard to avoid 'git submodule init' in that series.\nUnfortuately, few people agreed with me.\n\n\n>  We had some discussion on the list earlier about having submodule\n>  checkouts automatically acquire a branch name, so that commits don't\n>  get lost as easily.  I was going to think about this more and\n>  eventually submit a patch, but I haven't gotten to it yet.  Anyway,\n>  the idea is that you have a branch by default, so that you don't end\n>  up in the useless situation of not being on a branch, which encourages\n>  checking in without being on a branch, in the first place.\n>\n\nI like this idea and it is very useful for me.\n\n\n-- \nPing Yin\n"},{"id":"75710","messageId":"1209585345.25663.797.camel@work.sfbay.sun.com","threadId":"13323","inReplyTo":"32541b130804300947s6083156etc6514cc13c24af13@mail.gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-30T19:55:45Z","receivedAt":"2008-04-30T19:55:45Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Wed, 2008-04-30 at 12:47 -0400, Avery Pennarun wrote:\n> On 4/30/08, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:\n> That said, it makes submodule folders act completely inconsistently\n> with normal folders, which is highly undesirable.  The current\n> behaviour strongly encourages me to avoid submodules when I would\n> otherwise like to use them, just to keep the rest of my team members\n> (who are not git experts) from going insane.\n\nWell said. I find myself in exactly the same situation wrt. submodules.\nTo give an analogy imagine that everytime you had a mount-point in\nyour path you'd have to do something special in order to cross it.\nIOW, open(\"/tmp/foo/bar/baz.c\") would just work, where\nopen(\"/mnt/point1/foo.c\") would require you to first cross /mnt/point1\nsomehow. Currently submodules feel very much like the second case\nto me. Now, I've shut up for the time being on that other thread,\nsince I'd like to come up with a clearly defined workflow first.\nBut for the record: the way submodules are implemented right now\nin a porcelain make them pretty much an impossible sell to those\nin my org who don't want to know much about SCMs.\n\n> >  But once you did \"submodule init\", you will never need to run it again,\n> >  since it edits your .git/config, which does not change when switching\n> >  branches.\n> \n> Not true.  If \".gitmodules\" is different between branches, then\n> .git/config will have the wrong information.  I think this was the\n> reason for the \"read .gitmodules directly and don't worry about\n> .git/config\" discussion/patches earlier.\n\nExactly!\n\n> >  And as for \"submodule update\", I like the fact that the submodule is not\n> >  updated automatically.  For example, when I actively develop a submodule,\n> >  but have to rebase the superproject, I would _hate_ it if the submodule\n> >  wass updated.\n> \n> Why?  Every other folder in your entire project gets updated when you\n> \"git checkout\".  Why are submodules different?  I can personally vouch\n> that this is confusing for almost everyone I've seen who tried it.\n\n+1 ;-)\n\n\n> >  > PITFALLS:\n> >  > pitfall)\n> >  > If you commit a change on a submodule that's not on a branch, auto-updating\n> >  > submodules will make it difficult to revive that change.\n> >  >\n> >  > workaround)\n> >  > Don't allow the user to commit unless they are on a branch.\n> >  >\n> >  > ... couldn't think of anymore.  Anyone?\n> \n> We had some discussion on the list earlier about having submodule\n> checkouts automatically acquire a branch name, so that commits don't\n> get lost as easily.  I was going to think about this more and\n> eventually submit a patch, but I haven't gotten to it yet.  Anyway,\n> the idea is that you have a branch by default, so that you don't end\n> up in the useless situation of not being on a branch, which encourages\n> checking in without being on a branch, in the first place.\n\nCan you give a gmane pointer to that thread?\n\nThanks,\nRoman.\n"},{"id":"75711","messageId":"BC221793-3FB5-4249-8E8D-819C1B413592@gmail.com","threadId":"13323","inReplyTo":"alpine.DEB.1.00.0804301121240.17469@eeepc-johanness","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Tim Harper","fromEmail":"timcharper@gmail.com","sentAt":"2008-04-30T20:19:38Z","receivedAt":"2008-04-30T20:19:38Z","isPatch":false,"sender":{"key":"timcharper@gmail.com","avatar":"https://gravatar.com/avatar/1a2e0c06c7862ff065ee6b1d53195333a5a0577c040ecb2856a150d8e0b00ecd?d=mp&s=160"},"body":"I had a feeling my proposal would be met with a mix of opposition and  \nacceptance, depending on which \"git camp\" you are currently in.\n\nOn Apr 30, 2008, at 4:31 AM, Johannes Schindelin wrote:\n> Hi,\n>\n> On Tue, 29 Apr 2008, Tim Harper wrote:\n>\n>> 1) The submodule stays in the working copy when changing to a branch\n>>   that does not have a submodule.  This can break a build and cause\n>>   problems.  To work around, I have to delete the folder completely\n>>   (git-clean).  Then, when I switch back to the branch again, I  \n>> have to\n>>   re-download the submodule.\n>\n> The problem, of course, is that you can easily have valuable, but\n> not-tracked, files in there.  Deleting the submodule is therefore no\n> option.\n>\n\nSubmodules are not deleted.  They are moved out of the working copy  \ninto a folder in .git.  Therefore, upon changing back to the branch  \nwith the submodule, they are restored, without nay a hair on their  \nhead lost.\n\n>> 2) I have to type \"git checkout branch && git submodule init && git\n>>   submodule update\" to be sure that I really have the whole  \n>> contents of\n>>   the branch.  That's 3 commands, and a lot of typing.\n>\n> There is no way around \"checkout branch\", and I think that is a good\n> thing.\n>\n\nNaturally\n\n\n> But once you did \"submodule init\", you will never need to run it  \n> again,\n> since it edits your .git/config, which does not change when switching\n> branches.\n>\n\nIf working in a collaborative environment, when someone adds a new  \nsubmodule, it's extra burden to have to know \"oh, there's a new  \nsubmodule now, so I have to run \"git submodule init\".  I find it's  \nmore than sufficient just to run it everytime, and costs me nothing.   \nWhy aren't \"gut submodule update\" and \"git submodule init\" the same  \ncommand?  Perhaps a compromise could be met with \"git submodule update  \n-i\".\n\n\n> And as for \"submodule update\", I like the fact that the submodule is  \n> not\n> updated automatically.  For example, when I actively develop a  \n> submodule,\n> but have to rebase the superproject, I would _hate_ it if the  \n> submodule\n> wass updated.\n>\n\nThen we could make my proposed idea a configuration variable :)  For  \nmy use case, I passionately dislike the fact that a submodule is not  \nupdated automatically.  There's never a time when I don't want to  \nupdate the submodule.  The submodule is a very important piece of our  \nproject and the super-project depends on it being at the right version.\n\nI suspect a large majority of git users who would use submodules, and  \na handful of those already using it, have a similar use case.  Though,  \nI have no evidence to support this, other than my own observations.\n\n> The whole idea about submodules is that they are repositories of  \n> their own\n> right, and therefore the superproject should not mess with them,  \n> _unless_\n> explicitely asked to, with \"submodule update\".\n>\n>> 3) If I don't run \"git submodule update\", and carelessly run \"git  \n>> commit\n>>   -a\"  or \"git add .\", I risk propagating a submodule version from\n>>   another branch or undoing an important change.\n>\n> git commit -a is something that might make sense for newbies, but you\n> really should learn to use git add -p and commit without -a.\n>\n\nOr... use my textmate bundle for git, which makes committing very  \neasy. :)  The newbies are pointed to use \"git add .\" - just try typing  \n\"git add\"\n\n>> SUGGESTED ALGORITHM (AS HAS BEEN IMPLEMENTED IN THE GIT TEXTMATE  \n>> BUNDLE)\n>> When pulling / merging / changing branches:\n>> 1) cache all submodules to ~/.git/submodules_cache\n>> a) move from the working directory to a folder that is a MD5 hex- \n>> hash of\n>> both the submodule path and the submodule url\n>> 2) execute the pull / merge / branch change\n>> 3) restore all defined submodules to ~/.git/submodules_cache (only  \n>> the\n>> submodules that are still defined after the merge / change / pull)\n>> 4) execute git submodule init && git submodule update\n>>\n>>\n>> PITFALLS:\n>> pitfall)\n>> If you commit a change on a submodule that's not on a branch, auto- \n>> updating\n>> submodules will make it difficult to revive that change.\n>>\n>> workaround)\n>> Don't allow the user to commit unless they are on a branch.\n>>\n>> ... couldn't think of anymore.  Anyone?\n>\n> I do not like that.  I think that the user should be responsible to  \n> take\n> care of the up-to-dateness of the submodule.  As far as the  \n> superproject\n> is concerned, it just keeps track of the committed submodule state,  \n> but\n> does not enforce it.\n>\n\nDoes git serve me or do I serve git?  The whole point of software is  \nto make our lives easier and more productive.  If submodules are apart  \nof our project, why would not want git to handle making sure your  \nsubmodules are at the right version?\n\nI could see an issue if you changed the submodule version, and forgot  \nto commit the version change in your super project, and then switched  \nbranches, and lost your version change.  Perhaps the correct behavior  \nin this case would be to only automatically update the submodule  \nversion if it didn't differ from the superproject version in the  \n\"from\" branch.\n\n> FWIW I restarted on my --ignore-submodules patch, which can be seen  \n> here:\n>\n> http://repo.or.cz/w/git/dscho.git?a=shortlog;h=refs/heads/my-next\n>\n> It lacks tests, therefore it has not been submitted for review yet.\n>\n> The basic idea is that not only \"checkout\" ignores the _contents_ of  \n> the\n> submodules, but optionally \"diff\", and using that option \"stash\" and\n> \"rebase\".\n>\n> IOW you can \"stash a\n\nLeaving me hanging?  lol.\n\nThanks,\nTim\n"},{"id":"75712","messageId":"32541b130804301326l6c740f79nac03f1045af4d6fc@mail.gmail.com","threadId":"13323","inReplyTo":"1209585345.25663.797.camel@work.sfbay.sun.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-30T20:26:43Z","receivedAt":"2008-04-30T20:26:43Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/30/08, Roman Shaposhnik <rvs@sun.com> wrote:\n> Well said.\n\nThanks.  I'm having a bad day so a little agreement helps a lot :)\n\n>  > We had some discussion on the list earlier about having submodule\n>  > checkouts automatically acquire a branch name, so that commits don't\n>  > get lost as easily.  I was going to think about this more and\n>  > eventually submit a patch, but I haven't gotten to it yet.  Anyway,\n>  > the idea is that you have a branch by default, so that you don't end\n>  > up in the useless situation of not being on a branch, which encourages\n>  > checking in without being on a branch, in the first place.\n>\n> Can you give a gmane pointer to that thread?\n\nTry this:\nhttp://article.gmane.org/gmane.comp.version-control.git/78675\n\nNote that this particular message is representative, but is not the\nbest of the solutions proposed during that thread.  Exactly which of\nthe solutions, if any, is the best is left as an exercise to the\nreader :)\n\nHave fun,\n\nAvery\n"},{"id":"75713","messageId":"32541b130804301331o70310831raf71db7cbb51d507@mail.gmail.com","threadId":"13323","inReplyTo":"BC221793-3FB5-4249-8E8D-819C1B413592@gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-30T20:31:43Z","receivedAt":"2008-04-30T20:31:43Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/30/08, Tim Harper <timcharper@gmail.com> wrote:\n> > The problem, of course, is that you can easily have valuable, but\n> > not-tracked, files in there.  Deleting the submodule is therefore no\n> > option.\n>\n>  Submodules are not deleted.  They are moved out of the working copy into a\n> folder in .git.  Therefore, upon changing back to the branch with the\n> submodule, they are restored, without nay a hair on their head lost.\n\n<drool>\n\nYes!  I'm a little sad that I didn't think of that, because it sounds\nlike *exactly* what I want.\n\nWhat about the following case:\n- submodule matches the super module checkin\n- make changes to submodule but to *not* commit them\n- switch supermodule branches, which should checkout a different submodule\n- submodule checkout causes a conflict with uncommitted files\n\nWhat will/should happen here?  It seems like either the supermodule's\nsubmodule pointer won't be set properly (ie. git-submodule-update will\nfail, but the supermodule won't be marked as conflicted, thus\ngit-commit in the supermodule will commit the wrong submodule\nrevision) or else submodule files might have to be lost or something.\n\nAlso, someone earlier asked for a link to your work.  I'd like to see\nit too, as I don't know what a \"textmate git bundle\" is.  I gather\ntextmate is a MacOS X program, but I don't know what that has to do\nwith git-submodule :)\n\nThanks,\n\nAvery\n"},{"id":"75717","messageId":"EFEF26F9-D5D6-4BAC-9A8F-6D96E45AFAF7@gmail.com","threadId":"13323","inReplyTo":"32541b130804301331o70310831raf71db7cbb51d507@mail.gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Tim Harper","fromEmail":"timcharper@gmail.com","sentAt":"2008-04-30T21:37:16Z","receivedAt":"2008-04-30T21:37:16Z","isPatch":false,"sender":{"key":"timcharper@gmail.com","avatar":"https://gravatar.com/avatar/1a2e0c06c7862ff065ee6b1d53195333a5a0577c040ecb2856a150d8e0b00ecd?d=mp&s=160"},"body":"\nOn Apr 30, 2008, at 2:31 PM, Avery Pennarun wrote:\n\n> On 4/30/08, Tim Harper <timcharper@gmail.com> wrote:\n>>> The problem, of course, is that you can easily have valuable, but\n>>> not-tracked, files in there.  Deleting the submodule is therefore no\n>>> option.\n>>\n>> Submodules are not deleted.  They are moved out of the working copy  \n>> into a\n>> folder in .git.  Therefore, upon changing back to the branch with the\n>> submodule, they are restored, without nay a hair on their head lost.\n>\n> <drool>\n>\n> Yes!  I'm a little sad that I didn't think of that, because it sounds\n> like *exactly* what I want.\n>\nGlad to hear I'm not the only one who feels that way :)\n\n\n> What about the following case:\n> - submodule matches the super module checkin\n\n?\n\n>\n> - make changes to submodule but to *not* commit them\n\nChanges will never be lost, because the submodule is either stashed  \naway, working copy and all, or a \"checkout\" command is issued to  \nchange the HEAD pointer.  The latter is like changing branches with  \nuncommitted code - the changes will either be carried, and possibly  \nconflict.\n>\n> - switch supermodule branches, which should checkout a different  \n> submodule\n\nSubmodule is automatically changed for you, so changing branches makes  \nsure  you always have the \"whole project\".  Recursion isn't handled at  \nthe time being, so it only works for 1 level deep.\n\n>\n> - submodule checkout causes a conflict with uncommitted files\n\nOooh - this could be a problem currently, especially if you're not on  \na branch.  If you have to resolve conflicts the way to resolve them is  \nto commit, and you can't switch to a branch to commit unless you  \nresolve your issues.  In which case, you'll probably want to resolve,  \ncommit, then create a new branch with your current HEAD, then merge it  \ninto a branch.  I'll visit that as issues arise.  This is bleeding  \nedge experimentation here :)\n>\n>\n> What will/should happen here?  It seems like either the supermodule's\n> submodule pointer won't be set properly (ie. git-submodule-update will\n> fail, but the supermodule won't be marked as conflicted, thus\n> git-commit in the supermodule will commit the wrong submodule\n> revision) or else submodule files might have to be lost or something.\n\nThis is a good point: I don't believe that submodules have anyway to  \nmark the submodule revision as \"conflicted\".  In order for this  \nconcept to be handled in its full glory, that will most certainly need  \nto be handled.\n\n>\n>\n> Also, someone earlier asked for a link to your work.  I'd like to see\n> it too, as I don't know what a \"textmate git bundle\" is.  I gather\n> textmate is a MacOS X program, but I don't know what that has to do\n> with git-submodule :)\n>\n\nTextMate is an editor for MacOSX.\n\nHere's a link to the bundle.  It's written in ruby.:\nhttp://github.com/timcharper/git-tmbundle/\n\nWhat does it have to do with submodules?  It's essentially a \"GUI\" for  \ngit.  It provides automation for a lot of common tasks also.  My team  \nhas a need for submodules, but unfortunately in their current state I  \ncouldn't recommend it to them, so I \"smoothed over\" the rough edges by  \nautomating a lot of the awkwardness.  So far, it's been working well  \nfor us.\n\n> Thanks,\n>\n> Avery\n\nThank you!\n\nTim\n"},{"id":"75718","messageId":"32541b130804301448i537a0b98ta01cecc472e20aec@mail.gmail.com","threadId":"13323","inReplyTo":"EFEF26F9-D5D6-4BAC-9A8F-6D96E45AFAF7@gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-30T21:48:54Z","receivedAt":"2008-04-30T21:48:54Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/30/08, Tim Harper <timcharper@gmail.com> wrote:\n> > What about the following case:\n> > - submodule matches the super module checkin\n> > - make changes to submodule but to *not* commit them\n> > - switch supermodule branches, which should checkout a different submodule\n> > - submodule checkout causes a conflict with uncommitted files\n> > [all of the above were meant to be steps in the same test case]\n>\n> Oooh - this could be a problem currently, especially if you're not on a\n> branch.  If you have to resolve conflicts the way to resolve them is to\n> commit, and you can't switch to a branch to commit unless you resolve your\n> issues.  In which case, you'll probably want to resolve, commit, then create\n> a new branch with your current HEAD, then merge it into a branch.  I'll\n> visit that as issues arise.  This is bleeding edge experimentation here :)\n\nI know how to deal with it manually, but I'm more concerned about what\nwould happen in the automatic case, that is, the submodule is dirty\nand the supermodule tries to switch branches.\n\nI guess the standard thing to do would be to just have git-checkout in\nthe supermodule check all the submodules first, and if any of them are\ndirty, refuse to do anything at all.\n\n>  Here's a link to the bundle.  It's written in ruby.:\n>  http://github.com/timcharper/git-tmbundle/\n>\n> What does it have to do with submodules?  It's essentially a \"GUI\" for git.\n> It provides automation for a lot of common tasks also.  My team has a need\n> for submodules, but unfortunately in their current state I couldn't\n> recommend it to them, so I \"smoothed over\" the rough edges by automating a\n> lot of the awkwardness.  So far, it's been working well for us.\n\nHmm, so the bad news is that it doesn't really help me then :(\n\nIt would be awesome if you could turn the fancy behaviour of this\nbundle into patches to git-submodule, for example, and then have your\ntextmate macros call the modified git-submodule.  It might be a bit of\nan uphill battle to get the patches accepted into the release, but I\nthink it's worth the effort, as git-submodule in its current state is\njust a non-starter for my group at least.\n\nHave fun,\n\nAvery\n"},{"id":"75719","messageId":"1209594215.25663.864.camel@work.sfbay.sun.com","threadId":"13323","inReplyTo":"32541b130804301448i537a0b98ta01cecc472e20aec@mail.gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-04-30T22:23:35Z","receivedAt":"2008-04-30T22:23:35Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Wed, 2008-04-30 at 17:48 -0400, Avery Pennarun wrote:\n> It would be awesome if you could turn the fancy behaviour of this\n> bundle into patches to git-submodule, for example, and then have your\n> textmate macros call the modified git-submodule.  It might be a bit of\n> an uphill battle to get the patches accepted into the release, but I\n> think it's worth the effort, as git-submodule in its current state is\n> just a non-starter for my group at least.\n\nDoesn't the fact that the workflows around submodules tend to differ so\nmuch call for different incarnations of git-submodule? IOW, wouldn't\nit be ok to have an alternative to git-submodule somewhere in contrib?\n\nThanks,\nRoman.\n"},{"id":"75720","messageId":"32541b130804301528k70ae2f7eq5229c0b4bb1d3788@mail.gmail.com","threadId":"13323","inReplyTo":"1209594215.25663.864.camel@work.sfbay.sun.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-30T22:28:00Z","receivedAt":"2008-04-30T22:28:00Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/30/08, Roman Shaposhnik <rvs@sun.com> wrote:\n> On Wed, 2008-04-30 at 17:48 -0400, Avery Pennarun wrote:\n>  > It would be awesome if you could turn the fancy behaviour of this\n>  > bundle into patches to git-submodule, for example, and then have your\n>  > textmate macros call the modified git-submodule.  It might be a bit of\n>  > an uphill battle to get the patches accepted into the release, but I\n>  > think it's worth the effort, as git-submodule in its current state is\n>  > just a non-starter for my group at least.\n>\n> Doesn't the fact that the workflows around submodules tend to differ so\n>  much call for different incarnations of git-submodule? IOW, wouldn't\n>  it be ok to have an alternative to git-submodule somewhere in contrib?\n\nThis would be okay with me.  git-submodule itself doesn't seem to do\nanything very complex.  I would be happy to help contribute to such a\nthing, as my group needs it rather desperately.\n\nAvery\n"},{"id":"75731","messageId":"46dff0320804302156w7ed0e139pae171fa51a9e8cb8@mail.gmail.com","threadId":"13323","inReplyTo":"BC221793-3FB5-4249-8E8D-819C1B413592@gmail.com","subject":"Re: Making submodules easier to work with (auto-update on checkout or merge, stash & restore submodules)","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-05-01T04:56:31Z","receivedAt":"2008-05-01T04:56:31Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Thu, May 1, 2008 at 4:19 AM, Tim Harper <timcharper@gmail.com> wrote:\n\n>\n>  I could see an issue if you changed the submodule version, and forgot to\n> commit the version change in your super project, and then switched branches,\n> and lost your version change.  Perhaps the correct behavior in this case\n> would be to only automatically update the submodule version if it didn't\n> differ from the superproject version in the \"from\" branch.\n>\n\nAgree. Doing this will make submodule be more consistent with the ordinary file.\nFurther, when switching to another branch with diffrent recorded\ncommit about the submodule and the submodule in the from branch has\nbeen modified, just refuse to switch because this can be considered a\nconflict and there is no easy way to merge.\n\n\n-- \nPing Yin\n"},{"id":"75778","messageId":"20080501183837.GA4772@pvv.org","threadId":"13323","inReplyTo":"32541b130804301528k70ae2f7eq5229c0b4bb1d3788@mail.gmail.com","subject":"Re: Making submodules easier to work with","fromName":"Finn Arne Gangstad","fromEmail":"finnag@pvv.org","sentAt":"2008-05-01T18:38:37Z","receivedAt":"2008-05-01T18:38:37Z","isPatch":false,"sender":{"key":"finnag@pvv.org","avatar":"https://gravatar.com/avatar/b421ddd58c3f0f93aa473e17b98bb8d53c221fef741746bc8cb59fae4ec6d95e?d=mp&s=160"},"body":"On Wed, Apr 30, 2008 at 06:28:00PM -0400, Avery Pennarun wrote:\n> On 4/30/08, Roman Shaposhnik <rvs@sun.com> wrote:\n> > On Wed, 2008-04-30 at 17:48 -0400, Avery Pennarun wrote:\n> >  > It would be awesome if you could turn the fancy behaviour of this\n> >  > bundle into patches to git-submodule, for example, and then have your\n> >  > textmate macros call the modified git-submodule.  It might be a bit of\n> >  > an uphill battle to get the patches accepted into the release, but I\n> >  > think it's worth the effort, as git-submodule in its current state is\n> >  > just a non-starter for my group at least.\n> >\n> > Doesn't the fact that the workflows around submodules tend to differ so\n> >  much call for different incarnations of git-submodule? IOW, wouldn't\n> >  it be ok to have an alternative to git-submodule somewhere in contrib?\n> \n> This would be okay with me.  git-submodule itself doesn't seem to do\n> anything very complex.  I would be happy to help contribute to such a\n> thing, as my group needs it rather desperately.\n\nWhere do we want to go with submodules?\n\nToday, submodules seem to be a \"read-only\" implementation of the\nsupermodule. By that I mean that it is (only?) suited for creating a\nsupermodule that consists of independently released submodules, where\nall development happens in the submodules, and you sometimes update\nthe supermodule to refer to a new version of a submodule.\n\nWhat I've tried to achieve with submodules is a bit different: I want\nmost development to happen in the supermodule _as if_ the submodules\nwere part of the supermodule. There are two reasons for not doing it\nwith one big module: Total size can be a bit too big, but most\nimportantly, some submodules are shared between different super\nmodules and there is a certain level of synchronizing. Does this match\nyour scenarios in any way?\n\nSimplified but realistic scenario [*]:\n\nSupermodule \"crawler\" has some native code, and uses submodule \"os-lib\".\nSupermodule \"indexer\" has some native code, and uses submodule \"os-lib\".\n\nOne group works on \"crawler\" and releases a new crawler every 3 months.\nAnother group works on \"indexer\" and releases a new indexer every 2 weeks.\n\nThe guy who wrote \"os-lib\" quit 2 years ago, and no one really\nmaintains it. Once a year or so someone might try to unify the\n\"os-lib\" releases (so it should be POSSIBLE, just not the norm).\n\n\nWhat I would like: People should be able to work on \"crawler\" as if it\nwas a single git repository, in particular this means:\n\no Branching \"crawler\" means branching \"os-lib\"\no You can send a patch that contains changes both to \"crawler\" and \"os-lib\"\n  and get it applied in a resonable way as ONE modification (and git-am\n  would do the right thing)\no Merging branch a and branch b in \"crawler\" also merges the matching\n  branches a and b in \"os-lib\".\no Pushing the supermodule also pushes the submodules\n\nThe branch names in \"os-lib\" can (and probably should) be mangled, so\nthey are a combination of the supermodule and the branch\nname. E.g. something like crawler::branch, crawler/branch, or even\n{crawler}.branch. (Is there a character that is currently not allowed in\nbranch names that can be used as an escape?)\n\nSome lose implementation ideas: (code would be better, I know):\n\n- Enable new behaviour with \"git subdirectory\" instead of \"git submodule\",\n  and let \"git submodule\" keep the old behaviour.\n- subdirectories get the following magic additions:\n  - branching in parent creates {parent}.<branch> in subdirectories\n  - merging branches in parent merges the corresponding {parent}.branches\n    in subdirectories\n  - pushing a parent also pushes the subdirectories\n  - changing the parent to something that does not contain the subdirectory\n    is only allowed if the subdirectory is \"clean\" (no uncommitted changes).\n\n\n[*] In reality: there are 300+ submodules, in all states from\n\"actively maintained\" to \"last person who understood the code left 10\nyears ago\". There are maybe 5 or 10 supermodules.\n\n- Finn Arne\n"},{"id":"75781","messageId":"32541b130805011255t4b37a73cx9d670b9250e787c6@mail.gmail.com","threadId":"13323","inReplyTo":"20080501183837.GA4772@pvv.org","subject":"Re: Making submodules easier to work with","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-05-01T19:55:22Z","receivedAt":"2008-05-01T19:55:22Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Thu, May 1, 2008 at 2:38 PM, Finn Arne Gangstad <finnag@pvv.org> wrote:\n>  Today, submodules seem to be a \"read-only\" implementation of the\n>  supermodule. By that I mean that it is (only?) suited for creating a\n>  supermodule that consists of independently released submodules, where\n>  all development happens in the submodules, and you sometimes update\n>  the supermodule to refer to a new version of a submodule.\n>\n>  What I've tried to achieve with submodules is a bit different: I want\n>  most development to happen in the supermodule _as if_ the submodules\n>  were part of the supermodule. There are two reasons for not doing it\n>  with one big module: Total size can be a bit too big, but most\n>  importantly, some submodules are shared between different super\n>  modules and there is a certain level of synchronizing. Does this match\n>  your scenarios in any way?\n\nYour version (the second paragraph) matches my usage exactly.  The\nfirst paragraph does not, but I gather from some discussion on this\nlist that it does match some people's use cases, so I guess it should\ncontinue to be available.\n\n>  o Branching \"crawler\" means branching \"os-lib\"\n>  o You can send a patch that contains changes both to \"crawler\" and \"os-lib\"\n>   and get it applied in a resonable way as ONE modification (and git-am\n>   would do the right thing)\n>  o Merging branch a and branch b in \"crawler\" also merges the matching\n>   branches a and b in \"os-lib\".\n>  o Pushing the supermodule also pushes the submodules\n\nThe above would fit fine into my workflow, although it might be more\nfancy than I really need.  Personally, I don't mind thinking of my\nsubmodules as separate projects (ie. I should expect to commit,\nbranch, merge, and push separately).  But if the above features\nexisted I would adjust my working style to use them, just for the\nadded day-to-day convenience factor.\n\nDoing things like a single patch against one repo is a bit messy,\nbecause (presumably) you'd have the same commit message in both repos,\nwhich wouldn't really make sense.\n\n>  - Enable new behaviour with \"git subdirectory\" instead of \"git submodule\",\n>   and let \"git submodule\" keep the old behaviour.\n\nIf we get to the point where patchsets are gettingd sent around to\nplay with this, is it better to modify git-submodule or to create an\nentirely new file?  I don't know the preferred way of doing this.\n\nHave fun,\n\nAvery\n"},{"id":"75790","messageId":"3B9C16C0-0D0C-42D3-8315-72C8D8C88452@midwinter.com","threadId":"13323","inReplyTo":"20080501183837.GA4772@pvv.org","subject":"Re: Making submodules easier to work with","fromName":"Steven Grimm","fromEmail":"koreth@midwinter.com","sentAt":"2008-05-01T23:29:17Z","receivedAt":"2008-05-01T23:29:17Z","isPatch":false,"sender":{"key":"koreth@midwinter.com","avatar":"https://gravatar.com/avatar/71b4d2e8b62f168bdc9e9205341159e3567003b4f9e2127c617c5fa0a1f5bad2?d=mp&s=160"},"body":"On May 1, 2008, at 11:38 AM, Finn Arne Gangstad wrote:\n> Where do we want to go with submodules?\n\nI have two uses for submodules.\n\nThe first one is... well, not really submodules at all. I really want  \nsparse checkout so I can avoid bothering with parts of the larger  \nproject tree I don't care about or need. But right now submodules are  \nthe only way to approximate sparse checkout, so until we have true  \nsparse checkout support, I want to be able to get semantics that act  \nlike the submodules are just subdirectories in the supermodule with no  \nspecial care required. For this use case, the \"automatically update  \nall my submodules and push all of them when I push the supermodule\"  \nbehavior being discussed here, along with synchronizing local branches  \nbetween the super and submodules, would be a huge usability improvement.\n\nThe second is closer to the current design's intended use case: a sort  \nof version-controlled symbolic link. This is for libraries and other  \nexternal dependencies. The current model is not too bad for this use  \ncase, though ideally I'd like to see it get as low-touch as, say, svn  \nexternals, where only one person on a development team needs to worry  \nabout setting up and updating submodules and they cause the right  \nfiles to show up automatically for everyone else.\n\n-Steve\n"},{"id":"76229","messageId":"1210115872.25663.1097.camel@work.sfbay.sun.com","threadId":"13323","inReplyTo":"3B9C16C0-0D0C-42D3-8315-72C8D8C88452@midwinter.com","subject":"Re: Making submodules easier to work with","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-05-06T23:17:52Z","receivedAt":"2008-05-06T23:17:52Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Thu, 2008-05-01 at 16:29 -0700, Steven Grimm wrote:\n> On May 1, 2008, at 11:38 AM, Finn Arne Gangstad wrote:\n> > Where do we want to go with submodules?\n> \n> I have two uses for submodules.\n> \n> The first one is... well, not really submodules at all. I really want  \n> sparse checkout so I can avoid bothering with parts of the larger  \n> project tree I don't care about or need.\n\nWhen you say \"checkout\" do you really mean \"git checkout\" (IOW, you\nstill have the history for *everything* in .git but you just\ndon't want your working tree to be polluted by the stuff you don't\nneed at the moment) or are you talking about \"partial cloning\"?\nThe reason I'm asking is because a need for \"partial cloning\"\nis very real around where I work. Which is easy to understand\nbecause TeamWare (a precursor to BitKeeper) supports such a\nconcept. Now, before I get beaten to death: I DO understand why \n\"partial cloning\" and changeset-oriented SCMs don't go together\nwell. Yet, it seems to me that a sort of \"transitive closure\"\nof the changesets for the files I'm actually interested in might\nbe a natural way of having such a feature.\n\n>  But right now submodules are  \n> the only way to approximate sparse checkout, \n\nFor us submodules seem to be  the only way to approximate \"sparse\ncloning\" and that's why the usage patterns I worry about don't\nnecessarily match the ones that drive \"git submodule\"\n\nThanks,\nRoman.\n"},{"id":"76231","messageId":"1210117622.25663.1110.camel@work.sfbay.sun.com","threadId":"13323","inReplyTo":"32541b130805011255t4b37a73cx9d670b9250e787c6@mail.gmail.com","subject":"Re: Making submodules easier to work with","fromName":"Roman Shaposhnik","fromEmail":"rvs@sun.com","sentAt":"2008-05-06T23:47:02Z","receivedAt":"2008-05-06T23:47:02Z","isPatch":false,"sender":{"key":"rvs@sun.com","avatar":null},"body":"On Thu, 2008-05-01 at 15:55 -0400, Avery Pennarun wrote:\n> >  o Branching \"crawler\" means branching \"os-lib\"\n> >  o You can send a patch that contains changes both to \"crawler\" and \"os-lib\"\n> >   and get it applied in a resonable way as ONE modification (and git-am\n> >   would do the right thing)\n> >  o Merging branch a and branch b in \"crawler\" also merges the matching\n> >   branches a and b in \"os-lib\".\n> >  o Pushing the supermodule also pushes the submodules\n> \n> The above would fit fine into my workflow, although it might be more\n> fancy than I really need.  Personally, I don't mind thinking of my\n> submodules as separate projects (ie. I should expect to commit,\n> branch, merge, and push separately).  But if the above features\n> existed I would adjust my working style to use them, just for the\n> added day-to-day convenience factor.\n> \n> Doing things like a single patch against one repo is a bit messy,\n> because (presumably) you'd have the same commit message in both repos,\n> which wouldn't really make sense.\n\nMay be my brain is saturated with \"partial cloning\" but somehow the \nfollowing looks like an interesting twist on a decentralized SCM:\nimagine that the picture given by Finn Arne Gangstad weren't \nstatic. IOW, os-lib wasn't really a separate component to begin\nwith but was first developed as part of a \"crawler\" and only\nwhen the other team started to implement \"indexer\" there was\na need for os-lib to be shared between two independent projects.\nIs there any nice way to express such a dynamic history sharing,\nshort of truly refactoring os-lib into a separate Git repository\nand treating it either as a submodule or a subtree-merge?\n\nThanks,\nRoman.\n"},{"id":"76301","messageId":"32541b130805070914n2d090971rf367c838dcfd9557@mail.gmail.com","threadId":"13323","inReplyTo":"1210117622.25663.1110.camel@work.sfbay.sun.com","subject":"Re: Making submodules easier to work with","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-05-07T16:14:43Z","receivedAt":"2008-05-07T16:14:43Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 5/6/08, Roman Shaposhnik <rvs@sun.com> wrote:\n> May be my brain is saturated with \"partial cloning\" but somehow the\n>  following looks like an interesting twist on a decentralized SCM:\n>  imagine that the picture given by Finn Arne Gangstad weren't\n>  static. IOW, os-lib wasn't really a separate component to begin\n>  with but was first developed as part of a \"crawler\" and only\n>  when the other team started to implement \"indexer\" there was\n>  a need for os-lib to be shared between two independent projects.\n>  Is there any nice way to express such a dynamic history sharing,\n>  short of truly refactoring os-lib into a separate Git repository\n>  and treating it either as a submodule or a subtree-merge?\n\nPersonally, I think it would be good enough to split out the os-lib\ninto its own repo using \"git-filter-branch --subdirectory-filter\", and\nthen link to it in newer versions of your crawler and indexer projects\nusing git-submodule.\n\nThere would be a bit of wastage here, since crawler still contains the\nhistory of os-lib, which is the same (even the same tree and file\nobjects!) as the ones in os-lib.\n\nI can think of two responses to that:\n\n1) It's not very important, disk space is cheap and git repositories\nare small, and it's much better to have an accurate history of the\ncrawler project than to overoptimize for space.\n\n2) If the supermodule and submodule shared the same object repository\n(eg. the submodule was checked out with\n--alternate=<supermodule-gitdir>), there would be no need to waste\nstorage space.  If/when I get my act in gear and start submitting\ngit-submodule patches, supporting this behaviour is the direction I'll\nprobably start in.\n(http://article.gmane.org/gmane.comp.version-control.git/78675)\n\nHave fun,\n\nAvery\n"},{"id":"76338","messageId":"46dff0320805071813p34abde3aye7f954708e0bc6a7@mail.gmail.com","threadId":"13323","inReplyTo":"32541b130805070914n2d090971rf367c838dcfd9557@mail.gmail.com","subject":"Re: Making submodules easier to work with","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-05-08T01:13:33Z","receivedAt":"2008-05-08T01:13:33Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Thu, May 8, 2008 at 12:14 AM, Avery Pennarun <apenwarr@gmail.com> wrote:\n> On 5/6/08, Roman Shaposhnik <rvs@sun.com> wrote:\n>  > May be my brain is saturated with \"partial cloning\" but somehow the\n>  >  following looks like an interesting twist on a decentralized SCM:\n>  >  imagine that the picture given by Finn Arne Gangstad weren't\n>  >  static. IOW, os-lib wasn't really a separate component to begin\n>  >  with but was first developed as part of a \"crawler\" and only\n>  >  when the other team started to implement \"indexer\" there was\n>  >  a need for os-lib to be shared between two independent projects.\n>  >  Is there any nice way to express such a dynamic history sharing,\n>  >  short of truly refactoring os-lib into a separate Git repository\n>  >  and treating it either as a submodule or a subtree-merge?\n>\n>  Personally, I think it would be good enough to split out the os-lib\n>  into its own repo using \"git-filter-branch --subdirectory-filter\", and\n>  then link to it in newer versions of your crawler and indexer projects\n>  using git-submodule.\n>\n>  There would be a bit of wastage here, since crawler still contains the\n>  history of os-lib, which is the same (even the same tree and file\n>  objects!) as the ones in os-lib.\n>\n>  I can think of two responses to that:\n>\n>  1) It's not very important, disk space is cheap and git repositories\n>  are small, and it's much better to have an accurate history of the\n>  crawler project than to overoptimize for space.\n>\n>  2) If the supermodule and submodule shared the same object repository\n>  (eg. the submodule was checked out with\n>  --alternate=<supermodule-gitdir>), there would be no need to waste\n>  storage space.  If/when I get my act in gear and start submitting\n>  git-submodule patches, supporting this behaviour is the direction I'll\n>  probably start in.\n>  (http://article.gmane.org/gmane.comp.version-control.git/78675)\n>\n\nIt doesn't change the workflow whether using --alternative (although i\nthink it is a good idea).\n\nThe most interesting idea in that thread is the \"When checking out a\nsubmodule, give the submodule's current commit a useful branch name\".\nAs a heavy submodule user, this will make my life much easier.\n\n\n-- \nPing Yin\n"}]}