{"thread":{"id":"15520","subject":"externals program, way to do svn:externals-like subproject management without git-submodule","startedAt":"2008-09-13T21:17:18Z","lastAt":"2008-10-18T06:48:00Z","messageCount":6,"participants":["Miles Georgi","Avery Pennarun"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"90647","messageId":"853238710809131417v3818955sed4c0d3dd411a540@mail.gmail.com","threadId":"15520","inReplyTo":null,"subject":"externals program, way to do svn:externals-like subproject management without git-submodule","fromName":"Miles Georgi","fromEmail":"azimux@gmail.com","sentAt":"2008-09-13T21:17:18Z","receivedAt":"2008-09-13T21:17:18Z","isPatch":false,"sender":{"key":"azimux@gmail.com","avatar":"https://gravatar.com/avatar/aaa2d33b2ca63db5dd1309c1cfcf2257a01f39b100274a4ccd5a860906bb6c07?d=mp&s=160"},"body":"I recently wrote a program called externals to make it more simple for\nme to manage/deploy subprojects in my ruby on rails apps.  The problem\nwas that my projects had a bunch of subprojects, some of which were in\nsvn some in git, and when it came to deploying/managing these\nsubprojects it was kind of a pain in the neck.  Also, I wasn't fully\nsatisfied with git-submodule.  I'm assuming that several people on\nthis list prefer the git-submodule workflow over svn:externals, but\nfor me, I find myself performing a lot of extra steps that were\nunnecessary when working with svn:externals.  I've found the code I\nwrote to be really useful so I cleaned it up a bit and released it to\nthe public as the externals project.  It provides an SCM agnostic way\nof managing subprojects with an svn:externals-like workflow.\n\nI posted this on the rails list a week ago (though it can certainly be\nused in non-rails applications (it does require ruby to be installed,\nand is best installed via rubygems))   I didn't get any feedback at\nall which was kind of dissapointing.  I decided to mention it here to\nsee if I can get any feedback.\n\nI have a tutorial demonstrating how to use ext here:\nhttp://nopugs.com/ext-tutorial\n\nAny feedback is greatly appreciated\n\nMiles\n"},{"id":"90649","messageId":"32541b130809131517k6d1e5e4dsc5f72d54c7e71e55@mail.gmail.com","threadId":"15520","inReplyTo":"853238710809131417v3818955sed4c0d3dd411a540@mail.gmail.com","subject":"Re: externals program, way to do svn:externals-like subproject management without git-submodule","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-09-13T22:17:05Z","receivedAt":"2008-09-13T22:17:05Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Sat, Sep 13, 2008 at 5:17 PM, Miles Georgi <azimux@gmail.com> wrote:\n> I posted this on the rails list a week ago (though it can certainly be\n> used in non-rails applications (it does require ruby to be installed,\n> and is best installed via rubygems))   I didn't get any feedback at\n> all which was kind of dissapointing.  I decided to mention it here to\n> see if I can get any feedback.\n>\n> I have a tutorial demonstrating how to use ext here:\n> http://nopugs.com/ext-tutorial\n\nHi Miles,\n\nWell, you asked for feedback :)\n\nI myself have found that git-submodule doesn't really do it for me\neither.  Someday, like you, I was planning to break down and write a\nreplacement that works more like how I want.  I haven't done that, and\nnow here you are, with a replacement that's *more* like what I want,\nbut not quite.  With that in mind, here is an assortment of\nsuggestions that I myself have been too lazy to implement.\n\n(The suggestions are based on the tutorial you posted.  I haven't\ntried ext itself yet.)\n\n1. 'ext export' is not a good name for what it does.  It doesn't\nactually export anything.  It's putting files *into* your working set!\n Worse, 'cvs export' for example does something totally different.\n'ext qcheckout' (for \"quick\") or 'ext shallow' (because it makes\nshallow clones) might be better names.\n\n2. Your ini file is not quite in ini format.  To feel \"normal\", all\nlines should be either [section] lines or key=value lines.  It's worth\nhaving your ini-looking files really be ini-like partly because there\nare existing tools for parsing/creating those files in any language,\nand partly because humans get confused if they don't see what they\nexpect.  Section lines should also have a preceding blank line rather\nthan one following, though maybe that's just a side effect of your\nblog software.\n\n3. The ini file breaks things into sections based on scm, rather than\nbased on folder.  This is restrictive in multiple ways: you can't\ndefine *options* per-scm because the per-scm section is already used\nfor lists of folders.  And you can't define options per-folder because\nthere is no section to put them into.  I'd recommend an ini structure\nmore like this:\n\n      $ cat .externals\n\n      [main]\n      scm = git\n      layout = rails\n\n      [vendor/rails]\n      url = git://github.com/rails/rails.git\n      scm = git\n\n      [vendor/plugins/acts_as_list]\n      url = git://github.com/rails/acts_as_list.git\n      scm = git\n\n      [vendor/plugins/foreign_key_migrations]\n      url = svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/foreign_key_migrations\n      scm = svn\n\n      [vendor/plugins/redhillonrails_core]\n      url = svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/redhillonrails_core\n      scm = svn\n\nNote that it's longer this way, but much more extensible and it's\npretty obvious what it means just by looking at it.  Also, most of the\n\"scm=\" lines are actually redundant: externals should be able to\nauto-detect them anyhow.  In the [main] section, the scm in question\nis obvious, because if you can read the .externals file, you know\nwhich scm it is.  In the other sections, the URL method: string will\n(most of the time) reveal the scm type.  For ones where it doesn't,\nlike http:, then you could supply scm= or default to the same scm as\n[main].\n\n4. The built-in rails directory layout support makes me nervous and is\ndecidedly non-obvious.  As a Rails developer, presumably I know that\nwhen I check out (magic url string) that it should to into (magic path\nstring), and I can supply both.  I'd rather not supply just one and\n*hope* that the ext program knows wtf I want.  In my own case, I'm not\neven a Rails developer, so I'm even more nervous.\n\n5. There is no reason to use the ext command to create new subtrees\nanyway.  Why not let me check out the subtrees for the first time\nhowever I might normally do it (git clone, svn co, etc) and *then*\ntell ext to \"take a look and write down what I just did\"? It saves you\nteaching people new command line syntax.  Plus I can do a trial run\nwithout touching the ext command, and if it works, I can just say\n\"great, save these settings\" by typing something like \"ext save\".\n\n6. What about .externals files inside nested projects?  I (really do)\nhave a project that uses a git-submodule called myproj, and *that*\nproject contains a directory called mylib (ie. myproj/mylib).  I would\nexpect to have a main .externals file and a myproj/.externals file.\nIf I call ext from the toplevel, I'd expect it to be smart enough to\nsee both files and do something sane.\n\n7. I really appreciate git-submodule's ability to lock to a specific\ncommit, although apparently you don't like this.  The reason it's\nvaluable is for consistency: whenever I check out my project, the unit\ntests should *always* pass.  But if I automatically get the very\nlatest version of some subproject, they might break my tests, which is\nhighly annoying.  Sometimes we want to get the latest version of\neverything (which svn:externals and current ext makes easy and\ngit-submodule makes hard).  But other times, we just want to get a\nprecisely accurate historical version, which git-submodule makes easy\nbut ext makes hard.  This would actually be pretty easy for you to\nimplement if we use the new ini file layout above: just add an\n*optional* entry in each section called \"revision =\".  In svn, it's a\nrevision number; in git, it's a commit id.  Also, as you correctly\nobserved, git-submodule's tendency to check things out on a\ndisconnected HEAD is horrible and error prone.  A \"branch =\" ini\nvariable would solve that right away (defaulting to 'master' if not\nprovided), and also let you tell 'git clone' which branch to checkout\nin the first place, which is not actually obvious.  You could\nauto-record these variables as part of \"ext save\", and auto-grab the\nlatest in a new \"ext latest\" command.  (\"ext update\" is not a good\nname, as updating does different things on different scms, while\n\"latest\" is unambiguous).\n\n8. Branch switching.  If I switch my main project from one branch to\nanother using git (and yes, I do this very frequently!) I need my\nsubmodules to keep working properly.  git-submodule fails\nspectacularly at this: 'git submodule init' writes stuff into my\n.git/config file, which isn't per-branch, even though .gitmodules is,\nbut .gitmodules is ignored (except for *new* submodules) after the\nfirst init.  It also never auto-unregisters submodules when they\ndisappear from .gitmodules, and totally can't handle a submodule that\nmoves from one place to another in the local repo.  svn:externals is\ndifferently, but about equally, awful; it goes on wild deletion sprees\nwhen it's in a bad mood.  Pretty much anything remotely sane you do\nhere will beat any existing implementation I know of.  Key feature:\ndon't *delete* submodule dirs when I switch to a branch that doesn't\nneed them, but stash them away somewhere and bring them back when I\nswitch back - remembering that when you go to bring them back, we may\nwant a different revision, branch, and directory name, but it still\nmakes sense to start with the version we had before.\n\n9. Ordering of 'git push' operations.  Currently with git-submodule\nit's easy to check something into a subproject, commit the new gitlink\ninto the parent project, and then push the parent project, while\nforgetting to push the subproject back where it came from.  If ext\ncould enforce this somehow (eg. have a 'recursive push' sort of\noperation that does innermost first) it would be awesome.\n\n10. Self hosting.  Because ext isn't included with any scm, if I use\nit in my project I have an extra dependency that people need in order\nto build it, and not everybody has 'gem'.  It would be great if ext\ncould drop itself into a subdir of my project and manage updating\nitself to the latest version automatically when I ask.  Sadly, I guess\nit can't itself be a submodule because of the chicken-and-egg problem\n:)\n\nI look forward to hearing from you tomorrow when all my suggestions\nare implemented :)\n\nBut seriously, I don't mind helping out with a few of these if you\nlike the sound of these ideas.  I don't have a tonne of time, but the\ncorrect working of submodules is very important to me, and I'm\nsuffering a lot at work from the lack of it, so alleviating suffering\n== worth my time.\n\nHave fun,\n\nAvery\n"},{"id":"90650","messageId":"853238710809132057o22520f92qd1970847ae6a25f8@mail.gmail.com","threadId":"15520","inReplyTo":"32541b130809131517k6d1e5e4dsc5f72d54c7e71e55@mail.gmail.com","subject":"Re: externals program, way to do svn:externals-like subproject management without git-submodule","fromName":"Miles Georgi","fromEmail":"azimux@gmail.com","sentAt":"2008-09-14T03:57:47Z","receivedAt":"2008-09-14T03:57:47Z","isPatch":false,"sender":{"key":"azimux@gmail.com","avatar":"https://gravatar.com/avatar/aaa2d33b2ca63db5dd1309c1cfcf2257a01f39b100274a4ccd5a860906bb6c07?d=mp&s=160"},"body":"Wow!  Thanks for the extremely thorough reply.\n\nI'll just answer by numbers, I hope it's not too inconvenient to have\nto scroll between emails to see which point I'm talking about :/\n\n\n1.  Oh, I'm not really aware of what \"cvs export\" does.  I assumed it\nmeant something very similar to \"svn export\"  Running \"ext export URL\"\non a project manage by subversion with all the subprojects managed by\nsubversion simply runs \"svn export URL\" followed by an \"svn export\"\nfor each subproject, hence the name.  Calling it \"shallow\" makes sense\nfor git since I'm using --depth 1, but since \"svn export\" doesn't\ncreate the .svn directories necessary to make it a working directory,\nI don't really know if I would call that shallow.  For git's\nimplementation of export, I was thinking of just doing a clone and\nthen deleting .git/ which also wouldn't be shallow.\n\n2.  Agreed.  I wasn't really sure what proper ini format is.  I mostly\njust superficially mimicked the .git/config format.  I had considered\nXML but thought it was way too verbose, and considered YAML but it\nseemed awkward.  I would definitely be happy to change the format to\ncomply with whatever defines a valid ini file.\n\n3.  I am also bothered by the inconsistency of the meaning of [main]\nand [git/svn] along with everything you pointed out.\n\nQuestion?  Is it okay to have \"/\" characters in the ini section name?\nI'm assuming it is otherwise you wouldn't have recommended the format\nthat you did. I wonder if it would be okay to just put the repository\nas the section name.  That way scm and path could be excluded for\nsituations where the defaults are sufficient.  Although I guess this\nwould have the implication of changing a projects repository\n(something that happens way more frequently than changing the path\nit's installed to) some what messy since it's being identified by its\nrepository.\n\nThat's an interesting idea having the subprojects default to the same\nSCM as the main project.\n\n4.  Using the default paths is optional.  In rails applications, 99%\nof the time if a subproject is called rails, it is intended by the\ndeveloper to belong in vendor/rails, otherwise, 99% of the time it's a\nplugin.  Rails developers actually got kind of used to using\nsvn:externals to install plugins because the plugin install script has\na -x option that uses svn:externals.  But let's say you have something\nyou want in the lib directory, you could do:\n\next install some://repository/urlsomelib lib/somelib\n\nand it will not put it in vendor/plugins.  Rails developers tend to\nautomatically re-factor code that they intend to use in another\nproject into a plugin.  This is always what I do since plugins are as\neasy to make as putting a subproject in lib/ (or elsewhere.)\n\nBut yeah, if this feature makes somebody nervous, they can just always\nexplicitly give the path (which they'll have to do if it's not a rails\napp.)  It also detects that it's a rails app so it will complain if\nyou leave the path off in a non-rails app.  It won't try to put\nsomething in vendor/plugins of a non-rails app just because a path is\nmissing.\n\n5.  Yeah that's a good idea.  I could create a command that adds an\nexisting subproject to the .externals and updates the ignore properly.\n To get this effect at the moment, one would manually add the\nsubproject to .externals and run \"ext update_ignore\"  I personally\ndon't usually find myself doing this though.  I almost always want the\nsubproject to be added to the project.  I need to make an \"ext\nuninstall\" for backing out if you don't like it.  I personally would\nrather manually remove the item from .externals than have to issue two\ncommands to install it.  But your idea of a save command is very\npowerful, especially with the branch switching ideas you have in #8\nbelow.\n\n6.  Hey that's a pretty interesting idea!  I hadn't even thought about\nthat as I have not yet personally encountered that use case.  I've\nbeen hoping that rails plugins could have their own vendor/plugins\ndirectory because if they did I would also use something like that all\nthe time.  This is definitely something that should be implemented at\nsome point.\n\n7.  Agreed (for the most part.)  I almost never want to do this for my\nown projects as I almost always want the branch tip no matter what.\nI'd rather run tests and discover that something has broke before\ndeploying than have to go to all my projects and advance the commit\nmanually.\n\nHowever, for large projects that are not written by me, this would be\nuseful.  For example, rails and engines are 2 projects that frequently\nbreak my code.  If I blindly deploy an application with the rails\nmaster branch as a subproject after about a month, there's probably a\n15% chance it won't function properly (or at all.)  the way I\ncurrently have been handling this is by having my own rails repository\nat git://github.com/azimux/rails.git that I occaisionally will merge\nrails/rails.git with and run tests for my larger applications.  If I\nlike how it behaves I push the merged tip back out to azimux/rails.git\nwhich is what all my projects reference in their .externals file.\n\nI like your idea of a revision= and branch= syntax for the externals\nfile instead of the syntax I've been using.  I also like your idea of\nusing unambiguous comands like \"latest\"  And I would definitely not\nhave it disconnected from a branch if it can be avoided.  I guess it\nwould checkout a specific commit but be on a branch, and then if the\ndeveloper needed to edit the subproject, they would realize it was\nnon-fastforward when they pushed and then do a rebase against the\ncurrent tip?  I'm not 100% sure how this workflow would normally occur\nas I've not yet had to rebase anything, and when I was using\ngit-submodule, I would always checkout a branch tip if I had to edit\nthe subproject and then would do a git add submodule/path to point at\nthe new commit when I was done.\n\n8.  Oh yes, I feel your pain on this one.  I guess there would need to\nbe a combination of something like \"ext switch\" if you really want it\nswitched at the main project level for everybody, or maybe have \"ext\nsave\" be smart enough to detect a branch has been switched for\nsituations where the developer is trying to switch it for good from\nthe main project level.  The implementation of this probably requires\nsome more thinking but it's definitely very useful if implemented.\n\n9.  I wasn't planning on having any sort of commit/push-type features\nsince the workflows of the SCMs are different when it comes to\ncommiting/pushing.  For example, if it's a subversion project it needs\nto do an \"svn commit\" requiring a log message.  And sometimes I really\nonly want to push a subset of the subprojects that I've modified.\n\nMaybe what I could do is have \"ext status\" also see if a subproject\ncould be pushed and issue a warning like \"<subproject> has commits\nthat are missing from <remote name>.  Did you forget to push?\"  This\nwould be unnecessary for subversion managed subprojects as if it's up\nto date, then it's pushed and vice versa, but for git it would be very\nhelpful for when the status is empty due to commits being performed,\nbut is not yet pushed out to it's origin.  I don't know if I would\nlike it automatically doing the pushing though, but it could be\ndoable, maybe with a command with the name \"git\" in it to specify it's\nscm specific, like \"ext gitpush\" or I could just have an \"ext push\"\nthat does nothing for svn/cvs projects.\n\n10.  That's an interesting idea but I'm not sure it would be worth the\neffort to implement.  I would still need to require ruby to be\ninstalled (unless I rewrote it), and then the user would need to\nmanage the local ext for each project.  The chicken/egg thing actually\nkinda works out though.  You could do \"git clone blah/blah.git\"\nfollowed by \"cd blah; ./ext co\"  When I deploy I do something similar\nwith an after_update hook since the deployment software I'm using\ndoesn't know anything about ext... yet :)\n\nI truely appreciate all the ideas and criticisms.  I definitely intend\nto implement some your ideas, probably starting with restructuring the\nini file.  I should probably start by using a 3rd party ini file\nparser instead of the parser I wrote.  That would get me on track to\nchanging the format and changing the format is something that should\nprobably be done first before people start using it and expecting\nbackwards compatibility.\n\nThanks again Avery\n\nMiles\n\nOn Sat, Sep 13, 2008 at 3:17 PM, Avery Pennarun <apenwarr@gmail.com> wrote:\n> On Sat, Sep 13, 2008 at 5:17 PM, Miles Georgi <azimux@gmail.com> wrote:\n>> I posted this on the rails list a week ago (though it can certainly be\n>> used in non-rails applications (it does require ruby to be installed,\n>> and is best installed via rubygems))   I didn't get any feedback at\n>> all which was kind of dissapointing.  I decided to mention it here to\n>> see if I can get any feedback.\n>>\n>> I have a tutorial demonstrating how to use ext here:\n>> http://nopugs.com/ext-tutorial\n>\n> Hi Miles,\n>\n> Well, you asked for feedback :)\n>\n> I myself have found that git-submodule doesn't really do it for me\n> either.  Someday, like you, I was planning to break down and write a\n> replacement that works more like how I want.  I haven't done that, and\n> now here you are, with a replacement that's *more* like what I want,\n> but not quite.  With that in mind, here is an assortment of\n> suggestions that I myself have been too lazy to implement.\n>\n> (The suggestions are based on the tutorial you posted.  I haven't\n> tried ext itself yet.)\n>\n> 1. 'ext export' is not a good name for what it does.  It doesn't\n> actually export anything.  It's putting files *into* your working set!\n>  Worse, 'cvs export' for example does something totally different.\n> 'ext qcheckout' (for \"quick\") or 'ext shallow' (because it makes\n> shallow clones) might be better names.\n>\n> 2. Your ini file is not quite in ini format.  To feel \"normal\", all\n> lines should be either [section] lines or key=value lines.  It's worth\n> having your ini-looking files really be ini-like partly because there\n> are existing tools for parsing/creating those files in any language,\n> and partly because humans get confused if they don't see what they\n> expect.  Section lines should also have a preceding blank line rather\n> than one following, though maybe that's just a side effect of your\n> blog software.\n>\n> 3. The ini file breaks things into sections based on scm, rather than\n> based on folder.  This is restrictive in multiple ways: you can't\n> define *options* per-scm because the per-scm section is already used\n> for lists of folders.  And you can't define options per-folder because\n> there is no section to put them into.  I'd recommend an ini structure\n> more like this:\n>\n>      $ cat .externals\n>\n>      [main]\n>      scm = git\n>      layout = rails\n>\n>      [vendor/rails]\n>      url = git://github.com/rails/rails.git\n>      scm = git\n>\n>      [vendor/plugins/acts_as_list]\n>      url = git://github.com/rails/acts_as_list.git\n>      scm = git\n>\n>      [vendor/plugins/foreign_key_migrations]\n>      url = svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/foreign_key_migrations\n>      scm = svn\n>\n>      [vendor/plugins/redhillonrails_core]\n>      url = svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/redhillonrails_core\n>      scm = svn\n>\n> Note that it's longer this way, but much more extensible and it's\n> pretty obvious what it means just by looking at it.  Also, most of the\n> \"scm=\" lines are actually redundant: externals should be able to\n> auto-detect them anyhow.  In the [main] section, the scm in question\n> is obvious, because if you can read the .externals file, you know\n> which scm it is.  In the other sections, the URL method: string will\n> (most of the time) reveal the scm type.  For ones where it doesn't,\n> like http:, then you could supply scm= or default to the same scm as\n> [main].\n>\n> 4. The built-in rails directory layout support makes me nervous and is\n> decidedly non-obvious.  As a Rails developer, presumably I know that\n> when I check out (magic url string) that it should to into (magic path\n> string), and I can supply both.  I'd rather not supply just one and\n> *hope* that the ext program knows wtf I want.  In my own case, I'm not\n> even a Rails developer, so I'm even more nervous.\n>\n> 5. There is no reason to use the ext command to create new subtrees\n> anyway.  Why not let me check out the subtrees for the first time\n> however I might normally do it (git clone, svn co, etc) and *then*\n> tell ext to \"take a look and write down what I just did\"? It saves you\n> teaching people new command line syntax.  Plus I can do a trial run\n> without touching the ext command, and if it works, I can just say\n> \"great, save these settings\" by typing something like \"ext save\".\n>\n> 6. What about .externals files inside nested projects?  I (really do)\n> have a project that uses a git-submodule called myproj, and *that*\n> project contains a directory called mylib (ie. myproj/mylib).  I would\n> expect to have a main .externals file and a myproj/.externals file.\n> If I call ext from the toplevel, I'd expect it to be smart enough to\n> see both files and do something sane.\n>\n> 7. I really appreciate git-submodule's ability to lock to a specific\n> commit, although apparently you don't like this.  The reason it's\n> valuable is for consistency: whenever I check out my project, the unit\n> tests should *always* pass.  But if I automatically get the very\n> latest version of some subproject, they might break my tests, which is\n> highly annoying.  Sometimes we want to get the latest version of\n> everything (which svn:externals and current ext makes easy and\n> git-submodule makes hard).  But other times, we just want to get a\n> precisely accurate historical version, which git-submodule makes easy\n> but ext makes hard.  This would actually be pretty easy for you to\n> implement if we use the new ini file layout above: just add an\n> *optional* entry in each section called \"revision =\".  In svn, it's a\n> revision number; in git, it's a commit id.  Also, as you correctly\n> observed, git-submodule's tendency to check things out on a\n> disconnected HEAD is horrible and error prone.  A \"branch =\" ini\n> variable would solve that right away (defaulting to 'master' if not\n> provided), and also let you tell 'git clone' which branch to checkout\n> in the first place, which is not actually obvious.  You could\n> auto-record these variables as part of \"ext save\", and auto-grab the\n> latest in a new \"ext latest\" command.  (\"ext update\" is not a good\n> name, as updating does different things on different scms, while\n> \"latest\" is unambiguous).\n>\n> 8. Branch switching.  If I switch my main project from one branch to\n> another using git (and yes, I do this very frequently!) I need my\n> submodules to keep working properly.  git-submodule fails\n> spectacularly at this: 'git submodule init' writes stuff into my\n> .git/config file, which isn't per-branch, even though .gitmodules is,\n> but .gitmodules is ignored (except for *new* submodules) after the\n> first init.  It also never auto-unregisters submodules when they\n> disappear from .gitmodules, and totally can't handle a submodule that\n> moves from one place to another in the local repo.  svn:externals is\n> differently, but about equally, awful; it goes on wild deletion sprees\n> when it's in a bad mood.  Pretty much anything remotely sane you do\n> here will beat any existing implementation I know of.  Key feature:\n> don't *delete* submodule dirs when I switch to a branch that doesn't\n> need them, but stash them away somewhere and bring them back when I\n> switch back - remembering that when you go to bring them back, we may\n> want a different revision, branch, and directory name, but it still\n> makes sense to start with the version we had before.\n>\n> 9. Ordering of 'git push' operations.  Currently with git-submodule\n> it's easy to check something into a subproject, commit the new gitlink\n> into the parent project, and then push the parent project, while\n> forgetting to push the subproject back where it came from.  If ext\n> could enforce this somehow (eg. have a 'recursive push' sort of\n> operation that does innermost first) it would be awesome.\n>\n> 10. Self hosting.  Because ext isn't included with any scm, if I use\n> it in my project I have an extra dependency that people need in order\n> to build it, and not everybody has 'gem'.  It would be great if ext\n> could drop itself into a subdir of my project and manage updating\n> itself to the latest version automatically when I ask.  Sadly, I guess\n> it can't itself be a submodule because of the chicken-and-egg problem\n> :)\n>\n> I look forward to hearing from you tomorrow when all my suggestions\n> are implemented :)\n>\n> But seriously, I don't mind helping out with a few of these if you\n> like the sound of these ideas.  I don't have a tonne of time, but the\n> correct working of submodules is very important to me, and I'm\n> suffering a lot at work from the lack of it, so alleviating suffering\n> == worth my time.\n>\n> Have fun,\n>\n> Avery\n>\n"},{"id":"90651","messageId":"32541b130809132121g53edc7b2k6b6b87c401bb704f@mail.gmail.com","threadId":"15520","inReplyTo":"32541b130809132001s5eb79752y9d3a716490ae6c91@mail.gmail.com","subject":"Re: externals program, way to do svn:externals-like subproject management without git-submodule","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-09-14T04:21:49Z","receivedAt":"2008-09-14T04:21:49Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Sat, Sep 13, 2008 at 9:06 PM, Miles Georgi <azimux@gmail.com> wrote:\n> I'll just answer by numbers, I hope it's not too inconvenient to have\n> to scroll between emails to see which point I'm talking about :/\n\nIn general, it's better to use inline quoting.  It's okay if the\nquotes get long, because long messages are manageable, but messages\nthat require you to look at *other* messages at the same time are just\na PITA.  Luckily, however, I at least remember what we're talking\nabout :)\n\n> 1.  Oh, I'm not really aware of what \"cvs export\" does.  I assumed it\n> meant something very similar to \"svn export\"  Running \"ext export URL\"\n> on a project manage by subversion with all the subprojects managed by\n> subversion simply runs \"svn export URL\" followed by an \"svn export\"\n> for each subproject, hence the name.  Calling it \"shallow\" makes sense\n> for git since I'm using --depth 1, but since \"svn export\" doesn't\n> create the .svn directories necessary to make it a working directory,\n> I don't really know if I would call that shallow.  For git's\n> implementation of export, I was thinking of just doing a clone and\n> then deleting .git/ which also wouldn't be shallow.\n\nAha, sorry.  In that case, it does do exactly what cvs export and svn\nexport do, so I take it back.  I would never use it (because I always\ndeploy my .git directories along with the app itself... free\nbackups!).  But the name is correct.\n\n> 3.  I am also bothered by the inconsistency of the meaning of [main]\n> and [git/svn] along with everything you pointed out.\n>\n> Question?  Is it okay to have \"/\" characters in the ini section name?\n\nYes.  In general, a section name can have anything except a ] in it,\nand a key can have anything but an = in it.  (Well, obviously we\nshould expect that there are no CR, LF, or NUL characters either.)\n\nIf you want to do it exactly like git, you would name your sections like\n\n   [ext \"libs/my/plugin\"]\n\ninstead of my earlier suggestion of\n\n  [libs/my/plugin]\n\nThis would be compatible with git-config.  Also, this method means\nthat the [main] section could never be confused with a particular ext\nsection.\n\n> I wonder if it would be okay to just put the repository\n> as the section name.  That way scm and path could be excluded for\n> situations where the defaults are sufficient.  Although I guess this\n> would have the implication of changing a projects repository\n> (something that happens way more frequently than changing the path\n> it's installed to) some what messy since it's being identified by its\n> repository.\n\nI think you answered your own question there :)  Personally, it feels\nnatural to me to index a submodule by its local path, and have the URL\nbe an attribute of that submodule, rather than the other way around.\nThis makes sense since every local path can only have one submodule on\nit, while in insane situations, you can imagine having exactly the\nsame repo (perhaps different revisions or branches?) cloned into two\ndifferent locations.\n\n> But yeah, if this feature makes somebody nervous, they can just always\n> explicitly give the path (which they'll have to do if it's not a rails\n> app.)  It also detects that it's a rails app so it will complain if\n> you leave the path off in a non-rails app.  It won't try to put\n> something in vendor/plugins of a non-rails app just because a path is\n> missing.\n\nOkay, if it's not in rails mode, I guess it won't cause problems... as\nlong as the rails auto-detection is extremely reliable and can be\nturned off somehow.\n\n> 5.  Yeah that's a good idea.  I could create a command that adds an\n> existing subproject to the .externals and updates the ignore properly.\n>  To get this effect at the moment, one would manually add the\n> subproject to .externals and run \"ext update_ignore\"  I personally\n> don't usually find myself doing this though.  I almost always want the\n> subproject to be added to the project.  I need to make an \"ext\n> uninstall\" for backing out if you don't like it.  I personally would\n> rather manually remove the item from .externals than have to issue two\n> commands to install it.  But your idea of a save command is very\n> powerful, especially with the branch switching ideas you have in #8\n> below.\n\nRight, it wouldn't matter which way you did it, but the other options\nbelow fit really nicely into a 'git save' operation.\n\nAlso, I personally think it's more elegant to not have to do \"undo\"\noperations as a normal part of your workflow.  Checkout work work work\nsave seems nicer to me than Checkout uninstall work work work.\n\n[nested submodules]\n> 6.  Hey that's a pretty interesting idea!  I hadn't even thought about\n> that as I have not yet personally encountered that use case.  I've\n> been hoping that rails plugins could have their own vendor/plugins\n> directory because if they did I would also use something like that all\n> the time.  This is definitely something that should be implemented at\n> some point.\n\nWell, like I said, I'm doing it right now in one of my projects :)\nBut it doesn't have to be a high priority.\n\n> 7.  Agreed (for the most part.)  I almost never want to do this for my\n> own projects as I almost always want the branch tip no matter what.\n> I'd rather run tests and discover that something has broke before\n> deploying than have to go to all my projects and advance the commit\n> manually.\n\nOuch, I would never want to do it that way!  If my app has been\ndeployed for three months and someone reports a minor bug, I just want\nto fix that one bug and redeploy safely and immediately. I certainly\ndon't want to risk breaking everything by upgrading a bunch of\nunrelated modules.\n\n> the way I\n> currently have been handling this is by having my own rails repository\n> at git://github.com/azimux/rails.git that I occaisionally will merge\n> rails/rails.git with and run tests for my larger applications.  If I\n> like how it behaves I push the merged tip back out to azimux/rails.git\n> which is what all my projects reference in their .externals file.\n\nThis sounds like a convoluted way of just doing what I suggested. :)\n\nOh, that reminds me of another feature that would be awesome:\n*alternate* repository URLs.  In your example above, let's say you've\nmade some rails customizations, anyone cloning your project will have\nto get your version.  But one day, the rails developers merge in your\nchanges, so now their primary repo (probably with more bandwidth,\nuptime, etc) will work again.  If the ext lists *all* the repositories\nyou might need, ext can just git fetch from all of them, then checkout\nthe commit in question, and it will always work.\n\n> And I would definitely not\n> have it disconnected from a branch if it can be avoided.  I guess it\n> would checkout a specific commit but be on a branch, and then if the\n> developer needed to edit the subproject, they would realize it was\n> non-fastforward when they pushed and then do a rebase against the\n> current tip?  I'm not 100% sure how this workflow would normally occur\n> as I've not yet had to rebase anything, and when I was using\n> git-submodule, I would always checkout a branch tip if I had to edit\n> the subproject and then would do a git add submodule/path to point at\n> the new commit when I was done.\n\nIt can be simpler than that.  Essentially, what you do is:\n\n   git fetch origin\n   git checkout COMMITID\n   git branch -D BRANCHNAME\n   git checkout -b BRANCHNAME\n\nThe first checkout puts you on a disconnected HEAD temporarily, just\nin case you were previously on a branch called BRANCHNAME, which would\nhave prevented the \"delete branch\" operation from working.  Then we\ndelete the branch, and recreate it at the new location (COMMITID).\n\nWith that in place, git will know how to do everything else\nautomatically, but it's up to the user.  You can pull/merge the latest\nchanges from another branch, or rebase, etc.\n\nNote that as a safety measure, you probably want to refuse to actually\ndo the above if either of:\na) The git filesystem or index shows uncommitted files\nb) The current git commit != the commit ext thinks is checked out right now.\n\n> 9.  I wasn't planning on having any sort of commit/push-type features\n> since the workflows of the SCMs are different when it comes to\n> commiting/pushing.  For example, if it's a subversion project it needs\n> to do an \"svn commit\" requiring a log message.  And sometimes I really\n> only want to push a subset of the subprojects that I've modified.\n\nI think it would be really cool if ext could just figure this out.\nFor example, it's easy to tell that in svn, your changes have to be\ncommitted (ie. no dirty tree) and in git, the upstream repo needs to\ncontain your commitid (it's been pushed).\n\nIf you only want to push a subset of projects, that's fine, but you\ncan do it in the individual subdirs by hand anyhow.  On the other\nhand, I'd say you *don't* want to ever push the parent repo without\nfirst pushing all your changes, particularly if you have the commitid\nencoded in the .externals file, or you'll produce an un-checkout-able\nrepository.\n\n> Maybe what I could do is have \"ext status\" also see if a subproject\n> could be pushed and issue a warning like \"<subproject> has commits\n> that are missing from <remote name>.  Did you forget to push?\"  This\n> would be unnecessary for subversion managed subprojects as if it's up\n> to date, then it's pushed and vice versa, but for git it would be very\n> helpful for when the status is empty due to commits being performed,\n> but is not yet pushed out to it's origin.\n\nYes, that would be almost as good.\n\n>  I don't know if I would\n> like it automatically doing the pushing though, but it could be\n> doable, maybe with a command with the name \"git\" in it to specify it's\n> scm specific, like \"ext gitpush\" or I could just have an \"ext push\"\n> that does nothing for svn/cvs projects.\n\nI don't think it should do nothing for svn/cvs projects: it should\neither help you commit, or abort with a helpful message if your repo\nis dirty, or something like that.\n\nHope this helps.\n\nHave fun,\n\nAvery\n"},{"id":"91561","messageId":"853238710809250037q448046cekc901e5f48acd14a3@mail.gmail.com","threadId":"15520","inReplyTo":"32541b130809132121g53edc7b2k6b6b87c401bb704f@mail.gmail.com","subject":"Re: externals program, way to do svn:externals-like subproject management without git-submodule","fromName":"Miles Georgi","fromEmail":"azimux@gmail.com","sentAt":"2008-09-25T07:37:46Z","receivedAt":"2008-09-25T07:37:46Z","isPatch":false,"sender":{"key":"azimux@gmail.com","avatar":"https://gravatar.com/avatar/aaa2d33b2ca63db5dd1309c1cfcf2257a01f39b100274a4ccd5a860906bb6c07?d=mp&s=160"},"body":"Avery, and anybody else who might be interested:\n\nI have released a new version of externals which uses the format\nrecommended by Avery.\n\nThat should make it easier to add new features, such as the ones\nmentioned in Avery's email.  I've switched some of my projects to it\nand it seems to be working fine.\nI've bumped the version to 0.1.0 to signify a more betaish stage :)\n\nMiles\n\nOn Sat, Sep 13, 2008 at 9:21 PM, Avery Pennarun <apenwarr@gmail.com> wrote:\n> On Sat, Sep 13, 2008 at 9:06 PM, Miles Georgi <azimux@gmail.com> wrote:\n>> I'll just answer by numbers, I hope it's not too inconvenient to have\n>> to scroll between emails to see which point I'm talking about :/\n>\n> In general, it's better to use inline quoting.  It's okay if the\n> quotes get long, because long messages are manageable, but messages\n> that require you to look at *other* messages at the same time are just\n> a PITA.  Luckily, however, I at least remember what we're talking\n> about :)\n>\n>> 1.  Oh, I'm not really aware of what \"cvs export\" does.  I assumed it\n>> meant something very similar to \"svn export\"  Running \"ext export URL\"\n>> on a project manage by subversion with all the subprojects managed by\n>> subversion simply runs \"svn export URL\" followed by an \"svn export\"\n>> for each subproject, hence the name.  Calling it \"shallow\" makes sense\n>> for git since I'm using --depth 1, but since \"svn export\" doesn't\n>> create the .svn directories necessary to make it a working directory,\n>> I don't really know if I would call that shallow.  For git's\n>> implementation of export, I was thinking of just doing a clone and\n>> then deleting .git/ which also wouldn't be shallow.\n>\n> Aha, sorry.  In that case, it does do exactly what cvs export and svn\n> export do, so I take it back.  I would never use it (because I always\n> deploy my .git directories along with the app itself... free\n> backups!).  But the name is correct.\n>\n>> 3.  I am also bothered by the inconsistency of the meaning of [main]\n>> and [git/svn] along with everything you pointed out.\n>>\n>> Question?  Is it okay to have \"/\" characters in the ini section name?\n>\n> Yes.  In general, a section name can have anything except a ] in it,\n> and a key can have anything but an = in it.  (Well, obviously we\n> should expect that there are no CR, LF, or NUL characters either.)\n>\n> If you want to do it exactly like git, you would name your sections like\n>\n>   [ext \"libs/my/plugin\"]\n>\n> instead of my earlier suggestion of\n>\n>  [libs/my/plugin]\n>\n> This would be compatible with git-config.  Also, this method means\n> that the [main] section could never be confused with a particular ext\n> section.\n>\n>> I wonder if it would be okay to just put the repository\n>> as the section name.  That way scm and path could be excluded for\n>> situations where the defaults are sufficient.  Although I guess this\n>> would have the implication of changing a projects repository\n>> (something that happens way more frequently than changing the path\n>> it's installed to) some what messy since it's being identified by its\n>> repository.\n>\n> I think you answered your own question there :)  Personally, it feels\n> natural to me to index a submodule by its local path, and have the URL\n> be an attribute of that submodule, rather than the other way around.\n> This makes sense since every local path can only have one submodule on\n> it, while in insane situations, you can imagine having exactly the\n> same repo (perhaps different revisions or branches?) cloned into two\n> different locations.\n>\n>> But yeah, if this feature makes somebody nervous, they can just always\n>> explicitly give the path (which they'll have to do if it's not a rails\n>> app.)  It also detects that it's a rails app so it will complain if\n>> you leave the path off in a non-rails app.  It won't try to put\n>> something in vendor/plugins of a non-rails app just because a path is\n>> missing.\n>\n> Okay, if it's not in rails mode, I guess it won't cause problems... as\n> long as the rails auto-detection is extremely reliable and can be\n> turned off somehow.\n>\n>> 5.  Yeah that's a good idea.  I could create a command that adds an\n>> existing subproject to the .externals and updates the ignore properly.\n>>  To get this effect at the moment, one would manually add the\n>> subproject to .externals and run \"ext update_ignore\"  I personally\n>> don't usually find myself doing this though.  I almost always want the\n>> subproject to be added to the project.  I need to make an \"ext\n>> uninstall\" for backing out if you don't like it.  I personally would\n>> rather manually remove the item from .externals than have to issue two\n>> commands to install it.  But your idea of a save command is very\n>> powerful, especially with the branch switching ideas you have in #8\n>> below.\n>\n> Right, it wouldn't matter which way you did it, but the other options\n> below fit really nicely into a 'git save' operation.\n>\n> Also, I personally think it's more elegant to not have to do \"undo\"\n> operations as a normal part of your workflow.  Checkout work work work\n> save seems nicer to me than Checkout uninstall work work work.\n>\n> [nested submodules]\n>> 6.  Hey that's a pretty interesting idea!  I hadn't even thought about\n>> that as I have not yet personally encountered that use case.  I've\n>> been hoping that rails plugins could have their own vendor/plugins\n>> directory because if they did I would also use something like that all\n>> the time.  This is definitely something that should be implemented at\n>> some point.\n>\n> Well, like I said, I'm doing it right now in one of my projects :)\n> But it doesn't have to be a high priority.\n>\n>> 7.  Agreed (for the most part.)  I almost never want to do this for my\n>> own projects as I almost always want the branch tip no matter what.\n>> I'd rather run tests and discover that something has broke before\n>> deploying than have to go to all my projects and advance the commit\n>> manually.\n>\n> Ouch, I would never want to do it that way!  If my app has been\n> deployed for three months and someone reports a minor bug, I just want\n> to fix that one bug and redeploy safely and immediately. I certainly\n> don't want to risk breaking everything by upgrading a bunch of\n> unrelated modules.\n>\n>> the way I\n>> currently have been handling this is by having my own rails repository\n>> at git://github.com/azimux/rails.git that I occaisionally will merge\n>> rails/rails.git with and run tests for my larger applications.  If I\n>> like how it behaves I push the merged tip back out to azimux/rails.git\n>> which is what all my projects reference in their .externals file.\n>\n> This sounds like a convoluted way of just doing what I suggested. :)\n>\n> Oh, that reminds me of another feature that would be awesome:\n> *alternate* repository URLs.  In your example above, let's say you've\n> made some rails customizations, anyone cloning your project will have\n> to get your version.  But one day, the rails developers merge in your\n> changes, so now their primary repo (probably with more bandwidth,\n> uptime, etc) will work again.  If the ext lists *all* the repositories\n> you might need, ext can just git fetch from all of them, then checkout\n> the commit in question, and it will always work.\n>\n>> And I would definitely not\n>> have it disconnected from a branch if it can be avoided.  I guess it\n>> would checkout a specific commit but be on a branch, and then if the\n>> developer needed to edit the subproject, they would realize it was\n>> non-fastforward when they pushed and then do a rebase against the\n>> current tip?  I'm not 100% sure how this workflow would normally occur\n>> as I've not yet had to rebase anything, and when I was using\n>> git-submodule, I would always checkout a branch tip if I had to edit\n>> the subproject and then would do a git add submodule/path to point at\n>> the new commit when I was done.\n>\n> It can be simpler than that.  Essentially, what you do is:\n>\n>   git fetch origin\n>   git checkout COMMITID\n>   git branch -D BRANCHNAME\n>   git checkout -b BRANCHNAME\n>\n> The first checkout puts you on a disconnected HEAD temporarily, just\n> in case you were previously on a branch called BRANCHNAME, which would\n> have prevented the \"delete branch\" operation from working.  Then we\n> delete the branch, and recreate it at the new location (COMMITID).\n>\n> With that in place, git will know how to do everything else\n> automatically, but it's up to the user.  You can pull/merge the latest\n> changes from another branch, or rebase, etc.\n>\n> Note that as a safety measure, you probably want to refuse to actually\n> do the above if either of:\n> a) The git filesystem or index shows uncommitted files\n> b) The current git commit != the commit ext thinks is checked out right now.\n>\n>> 9.  I wasn't planning on having any sort of commit/push-type features\n>> since the workflows of the SCMs are different when it comes to\n>> commiting/pushing.  For example, if it's a subversion project it needs\n>> to do an \"svn commit\" requiring a log message.  And sometimes I really\n>> only want to push a subset of the subprojects that I've modified.\n>\n> I think it would be really cool if ext could just figure this out.\n> For example, it's easy to tell that in svn, your changes have to be\n> committed (ie. no dirty tree) and in git, the upstream repo needs to\n> contain your commitid (it's been pushed).\n>\n> If you only want to push a subset of projects, that's fine, but you\n> can do it in the individual subdirs by hand anyhow.  On the other\n> hand, I'd say you *don't* want to ever push the parent repo without\n> first pushing all your changes, particularly if you have the commitid\n> encoded in the .externals file, or you'll produce an un-checkout-able\n> repository.\n>\n>> Maybe what I could do is have \"ext status\" also see if a subproject\n>> could be pushed and issue a warning like \"<subproject> has commits\n>> that are missing from <remote name>.  Did you forget to push?\"  This\n>> would be unnecessary for subversion managed subprojects as if it's up\n>> to date, then it's pushed and vice versa, but for git it would be very\n>> helpful for when the status is empty due to commits being performed,\n>> but is not yet pushed out to it's origin.\n>\n> Yes, that would be almost as good.\n>\n>>  I don't know if I would\n>> like it automatically doing the pushing though, but it could be\n>> doable, maybe with a command with the name \"git\" in it to specify it's\n>> scm specific, like \"ext gitpush\" or I could just have an \"ext push\"\n>> that does nothing for svn/cvs projects.\n>\n> I don't think it should do nothing for svn/cvs projects: it should\n> either help you commit, or abort with a helpful message if your repo\n> is dirty, or something like that.\n>\n> Hope this helps.\n>\n> Have fun,\n>\n> Avery\n>\n"},{"id":"93343","messageId":"853238710810172348g44919ab1hd34af0b4bd585a14@mail.gmail.com","threadId":"15520","inReplyTo":"853238710809250037q448046cekc901e5f48acd14a3@mail.gmail.com","subject":"Re: externals program, way to do svn:externals-like subproject management without git-submodule","fromName":"Miles Georgi","fromEmail":"azimux@gmail.com","sentAt":"2008-10-18T06:48:00Z","receivedAt":"2008-10-18T06:48:00Z","isPatch":false,"sender":{"key":"azimux@gmail.com","avatar":"https://gravatar.com/avatar/aaa2d33b2ca63db5dd1309c1cfcf2257a01f39b100274a4ccd5a860906bb6c07?d=mp&s=160"},"body":"Avery, and to whoever else might be interested in this app:\n\nI have added a freeze feature to ext\n\nIt works like this:\n\next freeze project_name [revision]\n\nThis records the revision in the .externals file under that\nsubproject's entry.  When it is checked out/exported it fetches that\nrevision.\n\nIf you leave the revision off of the command, it uses the revision the\nproject is currently checked out at.\n\nAlso, I got a request to add a cross repository tagging feature, so\nthat you can create a tag at the top level that remembers\nthe revisions of all the subprojects at that time.  I thought this was\npretty interesting, and could just implement it as a hash\nof project to revision numbers/sha1s in a seperate file, perhaps\ncalled .exttags  Doesn't seem like it would be hard to implement\nnow that subprojects can be checked out at specific revisions.\n\nMiles\n\nOn Thu, Sep 25, 2008 at 12:37 AM, Miles Georgi <azimux@gmail.com> wrote:\n> Avery, and anybody else who might be interested:\n>\n> I have released a new version of externals which uses the format\n> recommended by Avery.\n>\n> That should make it easier to add new features, such as the ones\n> mentioned in Avery's email.  I've switched some of my projects to it\n> and it seems to be working fine.\n> I've bumped the version to 0.1.0 to signify a more betaish stage :)\n>\n> Miles\n>\n> On Sat, Sep 13, 2008 at 9:21 PM, Avery Pennarun <apenwarr@gmail.com> wrote:\n>> On Sat, Sep 13, 2008 at 9:06 PM, Miles Georgi <azimux@gmail.com> wrote:\n>>> I'll just answer by numbers, I hope it's not too inconvenient to have\n>>> to scroll between emails to see which point I'm talking about :/\n>>\n>> In general, it's better to use inline quoting.  It's okay if the\n>> quotes get long, because long messages are manageable, but messages\n>> that require you to look at *other* messages at the same time are just\n>> a PITA.  Luckily, however, I at least remember what we're talking\n>> about :)\n>>\n>>> 1.  Oh, I'm not really aware of what \"cvs export\" does.  I assumed it\n>>> meant something very similar to \"svn export\"  Running \"ext export URL\"\n>>> on a project manage by subversion with all the subprojects managed by\n>>> subversion simply runs \"svn export URL\" followed by an \"svn export\"\n>>> for each subproject, hence the name.  Calling it \"shallow\" makes sense\n>>> for git since I'm using --depth 1, but since \"svn export\" doesn't\n>>> create the .svn directories necessary to make it a working directory,\n>>> I don't really know if I would call that shallow.  For git's\n>>> implementation of export, I was thinking of just doing a clone and\n>>> then deleting .git/ which also wouldn't be shallow.\n>>\n>> Aha, sorry.  In that case, it does do exactly what cvs export and svn\n>> export do, so I take it back.  I would never use it (because I always\n>> deploy my .git directories along with the app itself... free\n>> backups!).  But the name is correct.\n>>\n>>> 3.  I am also bothered by the inconsistency of the meaning of [main]\n>>> and [git/svn] along with everything you pointed out.\n>>>\n>>> Question?  Is it okay to have \"/\" characters in the ini section name?\n>>\n>> Yes.  In general, a section name can have anything except a ] in it,\n>> and a key can have anything but an = in it.  (Well, obviously we\n>> should expect that there are no CR, LF, or NUL characters either.)\n>>\n>> If you want to do it exactly like git, you would name your sections like\n>>\n>>   [ext \"libs/my/plugin\"]\n>>\n>> instead of my earlier suggestion of\n>>\n>>  [libs/my/plugin]\n>>\n>> This would be compatible with git-config.  Also, this method means\n>> that the [main] section could never be confused with a particular ext\n>> section.\n>>\n>>> I wonder if it would be okay to just put the repository\n>>> as the section name.  That way scm and path could be excluded for\n>>> situations where the defaults are sufficient.  Although I guess this\n>>> would have the implication of changing a projects repository\n>>> (something that happens way more frequently than changing the path\n>>> it's installed to) some what messy since it's being identified by its\n>>> repository.\n>>\n>> I think you answered your own question there :)  Personally, it feels\n>> natural to me to index a submodule by its local path, and have the URL\n>> be an attribute of that submodule, rather than the other way around.\n>> This makes sense since every local path can only have one submodule on\n>> it, while in insane situations, you can imagine having exactly the\n>> same repo (perhaps different revisions or branches?) cloned into two\n>> different locations.\n>>\n>>> But yeah, if this feature makes somebody nervous, they can just always\n>>> explicitly give the path (which they'll have to do if it's not a rails\n>>> app.)  It also detects that it's a rails app so it will complain if\n>>> you leave the path off in a non-rails app.  It won't try to put\n>>> something in vendor/plugins of a non-rails app just because a path is\n>>> missing.\n>>\n>> Okay, if it's not in rails mode, I guess it won't cause problems... as\n>> long as the rails auto-detection is extremely reliable and can be\n>> turned off somehow.\n>>\n>>> 5.  Yeah that's a good idea.  I could create a command that adds an\n>>> existing subproject to the .externals and updates the ignore properly.\n>>>  To get this effect at the moment, one would manually add the\n>>> subproject to .externals and run \"ext update_ignore\"  I personally\n>>> don't usually find myself doing this though.  I almost always want the\n>>> subproject to be added to the project.  I need to make an \"ext\n>>> uninstall\" for backing out if you don't like it.  I personally would\n>>> rather manually remove the item from .externals than have to issue two\n>>> commands to install it.  But your idea of a save command is very\n>>> powerful, especially with the branch switching ideas you have in #8\n>>> below.\n>>\n>> Right, it wouldn't matter which way you did it, but the other options\n>> below fit really nicely into a 'git save' operation.\n>>\n>> Also, I personally think it's more elegant to not have to do \"undo\"\n>> operations as a normal part of your workflow.  Checkout work work work\n>> save seems nicer to me than Checkout uninstall work work work.\n>>\n>> [nested submodules]\n>>> 6.  Hey that's a pretty interesting idea!  I hadn't even thought about\n>>> that as I have not yet personally encountered that use case.  I've\n>>> been hoping that rails plugins could have their own vendor/plugins\n>>> directory because if they did I would also use something like that all\n>>> the time.  This is definitely something that should be implemented at\n>>> some point.\n>>\n>> Well, like I said, I'm doing it right now in one of my projects :)\n>> But it doesn't have to be a high priority.\n>>\n>>> 7.  Agreed (for the most part.)  I almost never want to do this for my\n>>> own projects as I almost always want the branch tip no matter what.\n>>> I'd rather run tests and discover that something has broke before\n>>> deploying than have to go to all my projects and advance the commit\n>>> manually.\n>>\n>> Ouch, I would never want to do it that way!  If my app has been\n>> deployed for three months and someone reports a minor bug, I just want\n>> to fix that one bug and redeploy safely and immediately. I certainly\n>> don't want to risk breaking everything by upgrading a bunch of\n>> unrelated modules.\n>>\n>>> the way I\n>>> currently have been handling this is by having my own rails repository\n>>> at git://github.com/azimux/rails.git that I occaisionally will merge\n>>> rails/rails.git with and run tests for my larger applications.  If I\n>>> like how it behaves I push the merged tip back out to azimux/rails.git\n>>> which is what all my projects reference in their .externals file.\n>>\n>> This sounds like a convoluted way of just doing what I suggested. :)\n>>\n>> Oh, that reminds me of another feature that would be awesome:\n>> *alternate* repository URLs.  In your example above, let's say you've\n>> made some rails customizations, anyone cloning your project will have\n>> to get your version.  But one day, the rails developers merge in your\n>> changes, so now their primary repo (probably with more bandwidth,\n>> uptime, etc) will work again.  If the ext lists *all* the repositories\n>> you might need, ext can just git fetch from all of them, then checkout\n>> the commit in question, and it will always work.\n>>\n>>> And I would definitely not\n>>> have it disconnected from a branch if it can be avoided.  I guess it\n>>> would checkout a specific commit but be on a branch, and then if the\n>>> developer needed to edit the subproject, they would realize it was\n>>> non-fastforward when they pushed and then do a rebase against the\n>>> current tip?  I'm not 100% sure how this workflow would normally occur\n>>> as I've not yet had to rebase anything, and when I was using\n>>> git-submodule, I would always checkout a branch tip if I had to edit\n>>> the subproject and then would do a git add submodule/path to point at\n>>> the new commit when I was done.\n>>\n>> It can be simpler than that.  Essentially, what you do is:\n>>\n>>   git fetch origin\n>>   git checkout COMMITID\n>>   git branch -D BRANCHNAME\n>>   git checkout -b BRANCHNAME\n>>\n>> The first checkout puts you on a disconnected HEAD temporarily, just\n>> in case you were previously on a branch called BRANCHNAME, which would\n>> have prevented the \"delete branch\" operation from working.  Then we\n>> delete the branch, and recreate it at the new location (COMMITID).\n>>\n>> With that in place, git will know how to do everything else\n>> automatically, but it's up to the user.  You can pull/merge the latest\n>> changes from another branch, or rebase, etc.\n>>\n>> Note that as a safety measure, you probably want to refuse to actually\n>> do the above if either of:\n>> a) The git filesystem or index shows uncommitted files\n>> b) The current git commit != the commit ext thinks is checked out right now.\n>>\n>>> 9.  I wasn't planning on having any sort of commit/push-type features\n>>> since the workflows of the SCMs are different when it comes to\n>>> commiting/pushing.  For example, if it's a subversion project it needs\n>>> to do an \"svn commit\" requiring a log message.  And sometimes I really\n>>> only want to push a subset of the subprojects that I've modified.\n>>\n>> I think it would be really cool if ext could just figure this out.\n>> For example, it's easy to tell that in svn, your changes have to be\n>> committed (ie. no dirty tree) and in git, the upstream repo needs to\n>> contain your commitid (it's been pushed).\n>>\n>> If you only want to push a subset of projects, that's fine, but you\n>> can do it in the individual subdirs by hand anyhow.  On the other\n>> hand, I'd say you *don't* want to ever push the parent repo without\n>> first pushing all your changes, particularly if you have the commitid\n>> encoded in the .externals file, or you'll produce an un-checkout-able\n>> repository.\n>>\n>>> Maybe what I could do is have \"ext status\" also see if a subproject\n>>> could be pushed and issue a warning like \"<subproject> has commits\n>>> that are missing from <remote name>.  Did you forget to push?\"  This\n>>> would be unnecessary for subversion managed subprojects as if it's up\n>>> to date, then it's pushed and vice versa, but for git it would be very\n>>> helpful for when the status is empty due to commits being performed,\n>>> but is not yet pushed out to it's origin.\n>>\n>> Yes, that would be almost as good.\n>>\n>>>  I don't know if I would\n>>> like it automatically doing the pushing though, but it could be\n>>> doable, maybe with a command with the name \"git\" in it to specify it's\n>>> scm specific, like \"ext gitpush\" or I could just have an \"ext push\"\n>>> that does nothing for svn/cvs projects.\n>>\n>> I don't think it should do nothing for svn/cvs projects: it should\n>> either help you commit, or abort with a helpful message if your repo\n>> is dirty, or something like that.\n>>\n>> Hope this helps.\n>>\n>> Have fun,\n>>\n>> Avery\n>>\n>\n"}]}