{"thread":{"id":"26800","subject":"dependable submodules","startedAt":"2011-03-20T00:53:26Z","lastAt":"2011-03-22T18:56:31Z","messageCount":11,"participants":["Daniel","Junio C Hamano","Jens Lehmann","Robert Pollak","Avery Pennarun","in-gitvger@baka.org"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"163797","messageId":"AANLkTi=JVO+KhnLKR-PvNQQFaZLhUmiVbKFxuytYEpmc@mail.gmail.com","threadId":"26800","inReplyTo":null,"subject":"dependable submodules","fromName":"Daniel","fromEmail":"daniel@netwalk.org","sentAt":"2011-03-20T00:53:26Z","receivedAt":"2011-03-20T00:53:26Z","isPatch":false,"sender":{"key":"daniel@netwalk.org","avatar":"https://gravatar.com/avatar/e956ff5c25bba17003ba59ad40911443001963c7a5f6f4b0102b21167d349021?d=mp&s=160"},"body":"I have a submodule from, example, github. I do not however want to be\nreliant years from now, that this remote repository on github is in\nexistant. Yet, I want to be able to easily do a git pull in the\nsubmodule as long as it's around.\n\nIs there a way to achieve this?\n\nIOW, I may have\n\n myrepo/somefile.txt\n myrepo/another_project/.git/...\n myrepo/another_project/another_project_file.txt\n\nand I'd like to, when I clone repo, to be able to get all files from\nrepo independently of the remote repo's existence.\n\nCurrently, if I do `git clone --recursive myrepo new_repo` I can get\nalmost the behavior I want with the exception of the other project\nbeing pulled remotely, which means I would *have* to have a network\nconnection to do so, but as mentioned, I don't want that dependency.\n\n-d\n"},{"id":"163800","messageId":"7vd3lmv2k1.fsf@alter.siamese.dyndns.org","threadId":"26800","inReplyTo":"AANLkTi=JVO+KhnLKR-PvNQQFaZLhUmiVbKFxuytYEpmc@mail.gmail.com","subject":"Re: dependable submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-03-20T04:27:58Z","receivedAt":"2011-03-20T04:27:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Daniel writes:\n\n> IOW, I may have\n>\n> myrepo/somefile.txt\n> myrepo/another_project/.git/...\n> myrepo/another_project/another_project_file.txt\n>\n> and I'd like to, when I clone repo, to be able to get all files from\n> repo independently of the remote repo's existence.\n\nCloned repositories from your repository can say where they get from\n\"another_project\" embedded in it in .git/config of the clone of the\nsuperproject. The supermodule may have .gitmodules that points at github\nor whereever your original upstream repository resided, but that is\nsupposed to be used to only give you the default.\n\nWhen \"another_project\" you borrowed really dies and becomes useless for\nanybody, you may update its entry in .gitmodules of the superproject to\npoint at the public repository that inherited that submodule project.\nThe project might be simply abandoned, in which case you end up being\nthat inheritor.\n"},{"id":"163801","messageId":"AANLkTin8Mr5xLtLqHSVuEOzzfmqnR2LU5vDdVfPprNXn@mail.gmail.com","threadId":"26800","inReplyTo":"7vd3lmv2k1.fsf@alter.siamese.dyndns.org","subject":"Re: dependable submodules","fromName":"Daniel","fromEmail":"daniel@netwalk.org","sentAt":"2011-03-20T04:32:45Z","receivedAt":"2011-03-20T04:32:45Z","isPatch":false,"sender":{"key":"daniel@netwalk.org","avatar":"https://gravatar.com/avatar/e956ff5c25bba17003ba59ad40911443001963c7a5f6f4b0102b21167d349021?d=mp&s=160"},"body":"On Sat, Mar 19, 2011 at 9:27 PM, Junio C Hamano wrote:\n> Daniel writes:\n>\n>> IOW, I may have\n>>\n>>     myrepo/somefile.txt\n>>     myrepo/another_project/.git/...\n>>     myrepo/another_project/another_project_file.txt\n>>\n>> and I'd like to, when I clone repo, to be able to get all files from\n>> repo independently of the remote repo's existence.\n>\n> Cloned repositories from your repository can say where they get from\n> \"another_project\" embedded in it in .git/config of the clone of the\n> superproject. The supermodule may have .gitmodules that points at github\n> or whereever your original upstream repository resided, but that is\n> supposed to be used to only give you the default.\n>\n> When \"another_project\" you borrowed really dies and becomes useless for\n> anybody, you may update its entry in .gitmodules of the superproject to\n> point at the public repository that inherited that submodule project.\n> The project might be simply abandoned, in which case you end up being\n> that inheritor.\n>\n\nSo would it be correct to say that even though those files are\nremotely fetched on a clone of the repository, they are actually\nrevisioned and stored in the super project?\n\nThis would have to be true for me to be able to still be able to\naccess those files in which case the project was abandoned (or if I\nlacked internet access).\n\n-d\n\n\n-- \n\"America was founded by men who understood that the threat of domestic\ntyranny is as great as any threat from abroad. If we want to be worthy\nof their legacy, we must resist the rush toward ever-increasing state\ncontrol of our society. Otherwise, our own government will become a\ngreater threat to our freedoms than any foreign terrorist.\"\n - Ron Paul, Texas Straight Talk, May 31, 2004\n"},{"id":"163816","messageId":"4D85F4E1.1020607@web.de","threadId":"26800","inReplyTo":"AANLkTin8Mr5xLtLqHSVuEOzzfmqnR2LU5vDdVfPprNXn@mail.gmail.com","subject":"Re: dependable submodules","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-20T12:36:49Z","receivedAt":"2011-03-20T12:36:49Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 20.03.2011 05:32, schrieb Daniel:\n> So would it be correct to say that even though those files are\n> remotely fetched on a clone of the repository, they are actually\n> revisioned and stored in the super project?\n\nNot in the superproject, but you'll have your copy of the history\nof the submodule in its own .git directory on your harddrive.\n\n> This would have to be true for me to be able to still be able to\n> access those files in which case the project was abandoned (or if I\n> lacked internet access).\n\nIf the remote side vanishes you still have your local copy. And if\nyou need to share that with others you can push the submodule\nsomewhere else and adjust the .gitmodules entry accordingly.\n"},{"id":"163829","messageId":"AANLkTimA-MKXrYfwE+WDOaPd7H1P6vvtcHTC6ODO5kAd@mail.gmail.com","threadId":"26800","inReplyTo":"4D85F4E1.1020607@web.de","subject":"Re: dependable submodules","fromName":"Daniel","fromEmail":"daniel@netwalk.org","sentAt":"2011-03-20T16:06:44Z","receivedAt":"2011-03-20T16:06:44Z","isPatch":false,"sender":{"key":"daniel@netwalk.org","avatar":"https://gravatar.com/avatar/e956ff5c25bba17003ba59ad40911443001963c7a5f6f4b0102b21167d349021?d=mp&s=160"},"body":"On Sun, Mar 20, 2011 at 5:36 AM, Jens Lehmann wrote:\n> Am 20.03.2011 05:32, schrieb Daniel:\n>> So would it be correct to say that even though those files are\n>> remotely fetched on a clone of the repository, they are actually\n>> revisioned and stored in the super project?\n>\n> Not in the superproject, but you'll have your copy of the history\n> of the submodule in its own .git directory on your harddrive.\n>\n>> This would have to be true for me to be able to still be able to\n>> access those files in which case the project was abandoned (or if I\n>> lacked internet access).\n>\n> If the remote side vanishes you still have your local copy. And if\n> you need to share that with others you can push the submodule\n> somewhere else and adjust the .gitmodules entry accordingly.\n>\n\n\nBecause I'm using it to version control my own home directory, and not\nall files are checked in to to git, this would post a problem for\nbackup. I only back up the .git folder itself since it's all I need\n(running git checkout . to restore all the files) and individually\nselecting each file to restore is a lot of work.\n\nPerhaps I just have an odd use-case.\n"},{"id":"163922","messageId":"4D877A70.8070907@jku.at","threadId":"26800","inReplyTo":"AANLkTin8Mr5xLtLqHSVuEOzzfmqnR2LU5vDdVfPprNXn@mail.gmail.com","subject":"Re: dependable submodules","fromName":"Robert Pollak","fromEmail":"robert.pollak@jku.at","sentAt":"2011-03-21T16:18:56Z","receivedAt":"2011-03-21T16:18:56Z","isPatch":false,"sender":{"key":"robert.pollak@jku.at","avatar":null},"body":" On 2011-03-20 Daniel wrote:\n[...]\n> So would it be correct to say that even though those files are\n> remotely fetched on a clone of the repository, they are actually\n> revisioned and stored in the super project?\n\nYou might want to try Avery Pennarun's git-subtree, which seems to\nprovide this functionality.\nIt's at https://github.com/apenwarr/git-subtree.\n"},{"id":"164011","messageId":"AANLkTikv+Wf_nSt0GZj0WgPjpbk6Kr_WG-ueO6US9bUM@mail.gmail.com","threadId":"26800","inReplyTo":"4D877A70.8070907@jku.at","subject":"Re: dependable submodules","fromName":"Daniel","fromEmail":"daniel@netwalk.org","sentAt":"2011-03-22T03:56:52Z","receivedAt":"2011-03-22T03:56:52Z","isPatch":false,"sender":{"key":"daniel@netwalk.org","avatar":"https://gravatar.com/avatar/e956ff5c25bba17003ba59ad40911443001963c7a5f6f4b0102b21167d349021?d=mp&s=160"},"body":"On Mon, Mar 21, 2011 at 9:18 AM, Robert Pollak wrote:\n>  On 2011-03-20 Daniel wrote:\n> [...]\n>> So would it be correct to say that even though those files are\n>> remotely fetched on a clone of the repository, they are actually\n>> revisioned and stored in the super project?\n>\n> You might want to try Avery Pennarun's git-subtree, which seems to\n> provide this functionality.\n> It's at https://github.com/apenwarr/git-subtree.\n\nI tried git-subtree. thanks but this is not what I wanted. This\nremoved the .git dir for the subtree and hence updating the subtree is\nnot easy.\n\nI want the functionality of git submodule except that I don't want the\nversion checked in as part of the superproject to have to be fetched\nremotely.\n\n-d\n\n\n\n\n-- \n\"America was founded by men who understood that the threat of domestic\ntyranny is as great as any threat from abroad. If we want to be worthy\nof their legacy, we must resist the rush toward ever-increasing state\ncontrol of our society. Otherwise, our own government will become a\ngreater threat to our freedoms than any foreign terrorist.\"\n - Ron Paul, Texas Straight Talk, May 31, 2004\n"},{"id":"164012","messageId":"AANLkTi=yb24MDN8JzBa0WFaZ+g4iF5a=OQNw0yOM0k73@mail.gmail.com","threadId":"26800","inReplyTo":"AANLkTikv+Wf_nSt0GZj0WgPjpbk6Kr_WG-ueO6US9bUM@mail.gmail.com","subject":"Re: dependable submodules","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2011-03-22T05:13:20Z","receivedAt":"2011-03-22T05:13:20Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Mon, Mar 21, 2011 at 8:56 PM, Daniel wrote:\n> I tried git-subtree. thanks but this is not what I wanted. This\n> removed the .git dir for the subtree and hence updating the subtree is\n> not easy.\n>\n> I want the functionality of git submodule except that I don't want the\n> version checked in as part of the superproject to have to be fetched\n> remotely.\n\nWell, as the logic goes, you commit to the imported project a lot more\noften than you submit upstream, so git-subtree makes the common part\nmuch easier and the less-common part slightly harder, which is a\nbetter tradeoff than git submodules. But it does seem to be a matter\nof preference.\n\nIf you just want the submodule's commit to be auto-fetched from the\nsame repo as the supermodule, I've been experimenting with a tool I've\ncreated called 'git tie' that makes this possible. It's not yet ready\nfor prime time, but you can use the same trick yourself if you're in a\nhurry. The idea is simple enough: just maintain a branch named\nrefs/heads/tie that is a continually updated \"merge -s ours\" of all\nyour submodule commits, all jammed together in a single branch that\nnobody will ever look at.\n\nThen when you 'git fetch origin', it'll fetch both the 'tie' branch\nand all your other branches, which have references to the commits in\nthe 'tie' branch. The 'tie' branch is what makes it so they'll get\nauto-downloaded. (Don't forget to push it, too, when you push\nmaster.)\n\nThe tool I'm working on is basically just one that will auto-maintain\nthe 'tie' branch for you.\n\nOh, and the second half of this is to just use '.' in your .gitmodules\nas the path to your submodule repo. It would be nice if 'git\nsubmodule update' would use the alternates mechanism (git clone\n--reference) to get all objects from your subproject's local repo,\nsince we know they're there anyway. I was going to make 'git tie' do\nthis part right too, so you wouldn't even need a .gitmodules file.\nBut either way, you'll survive, it might just be slower than optimal\nif it downloads everything twice.\n\nAnd there you go... I've now told you my secrets and regaled you with\nvapourware. Hope that helps more than it hurts :)\n\nHave fun,\n\nAvery\n"},{"id":"164053","messageId":"201103221428.p2MESAkG018553@no.baka.org","threadId":"26800","inReplyTo":"AANLkTikv+Wf_nSt0GZj0WgPjpbk6Kr_WG-ueO6US9bUM@mail.gmail.com","subject":"Re: dependable submodules","fromName":"","fromEmail":"in-gitvger@baka.org","sentAt":"2011-03-22T14:28:10Z","receivedAt":"2011-03-22T14:28:10Z","isPatch":false,"sender":{"key":"in-gitvger@baka.org","avatar":null},"body":"\nIn message , Dani\nel writes:\n\n I tried git-subtree. thanks but this is not what I wanted. This\n removed the .git dir for the subtree and hence updating the subtree is\n not easy.\n\n I want the functionality of git submodule except that I don't want the\n version checked in as part of the superproject to have to be fetched\n remotely.\n\nCould \"remotely\" be a local shadow master? gitslave is another option\nto git-submodules and git subtree merge for provide a superproject\nwith attached slaves, but like git submodules it does require fetching\nfrom \"upstream\". However, there is nothing requiring that \"upstream\"\nbe the true Internet upstream. It could be a local shadow copy of the\nmaster. I describe this scenario in more detail on the gitslave home\npage starting with the third paragraph under \"Gitslave is not\nperfect\". I wrote this with gitslave in mind, but there is nothing\nstopping you from using the concept with git-submodules (aside from\nthe pain of the submodule update pain added to the shadow repository\nupdate pain breeding).\n\nHowever, if you really really have a good reason to want *everything*\nin one repo, perhaps you could play games with branches and subtree\nmerge. For each slave repository, create a new branch with the\ncontent you want (or do a merge from the slave onto the new branch to\nget the history or whatever). Then attach the local repository to\nitself using git-submodules (bonus points if you can use a\nrepository-relative URL), set the correct SHA corresponding to the\nbranch of the \"slave\", and then (if it works, and I don't see why it\nwould not) you are all done. I would hate to have to use and update\nsuch a repository, though.\n\n\t\t\t\t\t-Seth Robertson\n"},{"id":"164055","messageId":"AANLkTinrvg=UkZJWCQt3hZXpAAgW=manhGPvk9s9KsCs@mail.gmail.com","threadId":"26800","inReplyTo":"201103221428.p2MESAkG018553@no.baka.org","subject":"Re: dependable submodules","fromName":"Daniel","fromEmail":"daniel@netwalk.org","sentAt":"2011-03-22T14:55:26Z","receivedAt":"2011-03-22T14:55:26Z","isPatch":false,"sender":{"key":"daniel@netwalk.org","avatar":"https://gravatar.com/avatar/e956ff5c25bba17003ba59ad40911443001963c7a5f6f4b0102b21167d349021?d=mp&s=160"},"body":"On Tue, Mar 22, 2011 at 7:28 AM, wrote:\n>\n> In message , Dani\n> el writes:\n>\n>    I tried git-subtree. thanks but this is not what I wanted. This\n>    removed the .git dir for the subtree and hence updating the subtree is\n>    not easy.\n>\n>    I want the functionality of git submodule except that I don't want the\n>    version checked in as part of the superproject to have to be fetched\n>    remotely.\n>\n> Could \"remotely\" be a local shadow master?  gitslave is another option\n> to git-submodules and git subtree merge for provide a superproject\n> with attached slaves, but like git submodules it does require fetching\n> from \"upstream\".  However, there is nothing requiring that \"upstream\"\n> be the true Internet upstream.  It could be a local shadow copy of the\n> master.  I describe this scenario in more detail on the gitslave home\n> page starting with the third paragraph under \"Gitslave is not\n> perfect\".  I wrote this with gitslave in mind, but there is nothing\n> stopping you from using the concept with git-submodules (aside from\n> the pain of the submodule update pain added to the shadow repository\n> update pain breeding).\n>\n> However, if you really really have a good reason to want *everything*\n> in one repo, perhaps you could play games with branches and subtree\n> merge.  For each slave repository, create a new branch with the\n> content you want (or do a merge from the slave onto the new branch to\n> get the history or whatever).  Then attach the local repository to\n> itself using git-submodules (bonus points if you can use a\n> repository-relative URL), set the correct SHA corresponding to the\n> branch of the \"slave\", and then (if it works, and I don't see why it\n> would not) you are all done.  I would hate to have to use and update\n> such a repository, though.\n>\n>                                        -Seth Robertson\n>\n\nMy current setup is that I'm simply versioning my home directory with\nmercurial. Some vim plugins are under git and since it's a different\nversion control system I can check in the plugins easily. When I need\nto update those plugins, I just run git update, and check them into\nmercurial again. No problem, very easy. I'm looking into what it\nwould take to convert my repo to be completely git, and so far I have\nno exact way of dealing with those vim plugins since via submodules I\nhave to rely on the external repos being present when I clone my\nhomedir repo. Granted, if the repos have gone away a decade from now\nand my vim plugins are not present, it won't be the end of the world,\nbut I want to see how close I can get to my current functionality when\nusing git instead of mercurial.\n"},{"id":"164076","messageId":"4D88F0DF.8080808@web.de","threadId":"26800","inReplyTo":"AANLkTinrvg=UkZJWCQt3hZXpAAgW=manhGPvk9s9KsCs@mail.gmail.com","subject":"Re: dependable submodules","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-22T18:56:31Z","receivedAt":"2011-03-22T18:56:31Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 22.03.2011 15:55, schrieb Daniel:\n> My current setup is that I'm simply versioning my home directory with\n> mercurial. Some vim plugins are under git and since it's a different\n> version control system I can check in the plugins easily. When I need\n> to update those plugins, I just run git update, and check them into\n> mercurial again. No problem, very easy. I'm looking into what it\n> would take to convert my repo to be completely git, and so far I have\n> no exact way of dealing with those vim plugins since via submodules I\n> have to rely on the external repos being present when I clone my\n> homedir repo. Granted, if the repos have gone away a decade from now\n> and my vim plugins are not present, it won't be the end of the world,\n> but I want to see how close I can get to my current functionality when\n> using git instead of mercurial.\n\nYou could add the .git directories of the vim plugin direcories that\nare under git to the .gitignore in your home directory. Then you can\njust continue to do what you are used to: Go into the vim plugin dir,\nrun git update there, go back into your home directory and commit the\nnew files and push everything to your remote. You'll lose the pointers\nfrom the vim plugin directories to their origins that way, but that\ndoesn't seem to be a problem for you.\n"}]}