{"thread":{"id":"11754","subject":"Submodules use case: web development based on modular CMS","startedAt":"2008-01-27T00:39:56Z","lastAt":"2008-01-28T10:44:48Z","messageCount":4,"participants":["Jakub Narebski","Martin Langhoff"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"66653","messageId":"200801270139.57830.jnareb@gmail.com","threadId":"11754","inReplyTo":null,"subject":"Submodules use case: web development based on modular CMS","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-01-27T00:39:56Z","receivedAt":"2008-01-27T00:39:56Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"A bit of time ago I have stumbled upon the following blog entry \n(question): \"Agaric wants version control that lets Drupal core and \ncontrib replace entire directories within our checkouts\"[1]\n  http://tinyurl.com/yv3jp4\n\nHere the author want to use version control system for customized \nversion of CMS system, namely:\n * Drupal core\n * selected Drupal contributed modules (\"inside\" Drupal core)\n * own work (theme, custom modules) (also \"inside\" Drupal core)\nAnd of course he wants to have development branches: dev, test and live, \nand want to always checkout correct versions of Drupal, Drupal modules \nand own work.\n\nSomebody (in comments) proposed using Subversion externals; I have \nproposed using Git submodules (subproject) support.  Nevertheless while \nI have at one time manage PHPNuke portal using CVS and vendor branches, \nand even had half-written automatic fast checkout/fast export script, \nI haven't worked with submodules in Git, nor with using Git to manage \nweb site.\n\nComments (here, and as comments to mentioned blog article) about how \nsuch thing should be done with Git very appreciated. (And for example \nhow Git submodules differ from svn:externals and why git way is \nbetter ;-)).\n\n\nP.S. BTW. Drupal is considering moving from CVS to some other SCM, see\nhttp://groups.drupal.org/revision-control-systems .  Perhaps Git would \nbe the choice?  Is anybody here active in Drupal development?  Is for \nexample their Version Control API too CVS-centric (single file history \ncentric)?\n\n[1]http://agaricdesign.com/note/agaric-wants-version-control-lets-drupal-core-and-contrib-replace-entire-directories-within-our-checkouts\n-- \nJakub Narebski\nPoland\n"},{"id":"66658","messageId":"46a038f90801261900s2c4fa348wd5e0aea4ed9f1c12@mail.gmail.com","threadId":"11754","inReplyTo":"200801270139.57830.jnareb@gmail.com","subject":"Re: Submodules use case: web development based on modular CMS","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2008-01-27T03:00:52Z","receivedAt":"2008-01-27T03:00:52Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Jan 27, 2008 1:39 PM, Jakub Narebski <jnareb@gmail.com> wrote:\n> A bit of time ago I have stumbled upon the following blog entry\n> (question): \"Agaric wants version control that lets Drupal core and\n> contrib replace entire directories within our checkouts\"[1]\n>   http://tinyurl.com/yv3jp4\n\nWhile it is possible to use submodules, I tend to think that you can\njust use your own repo / branch \"off\" the main project, just like\nunofficial kernel modules do.\n\nIt does have a number of advantages -- including being explicit wrt to\nwhat version of the core project the module is developed against.\nDrupal and similar projects (like Moodle) do have a module API, but\nit's not *that* stable -- in spite of the best intentions, those APIs\nget subtle changes all the time. You should have your \"release\" script\npackage only the module subdirectories -- and perhaps any delta in the\ncore (of Drupal) that needs to be applied. In many \"contrib\" projects\nI've seen module files and little patch files that you are supposed to\napply - and maintenance of those is a pain. It makes more sense to use\ngit over the whole tree.\n\nSo it's a tradeoff -- IMHO, at first blush it looks more \"natural\" to\nuse submodules, but as things progress over time I think it forces a\nlot of additional and ill-fitted work like maintaining patches, or\nadding tags that indicate \"developed against Drupal v1.2.3\". And the\nday the contrib module becomes official, \"merging it in\" is a bit of a\nmess if you want to preserve history. So in the long haul, it makes a\nlot more sense to use the \"branch off the full project\" approach that\nhas served the kernel modules so well.\n\nIn that sense, I think that gitk's tree getting merged in a\nsubdirectory is good as an example of what git can do, but a bit\npointless as gitk depend quite tightly on the behaviour of git\ncommands. So distributing gitk separately would be a big pain as each\ngitk version is usable against a narrow set of git revisions. Luckily,\nit gets merged into git and that's how it gets distributed.\n\nMy .2c anyway ;-)\n\n\nm\n"},{"id":"66683","messageId":"200801271923.43212.jnareb@gmail.com","threadId":"11754","inReplyTo":"46a038f90801261900s2c4fa348wd5e0aea4ed9f1c12@mail.gmail.com","subject":"Re: Submodules use case: web development based on modular CMS","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-01-27T18:23:42Z","receivedAt":"2008-01-27T18:23:42Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"I CC'ed drupal mainling list...\n\nOn Sun, 27 Jan 2008, Martin Langhoff wrote:\n> On Jan 27, 2008 1:39 PM, Jakub Narebski <jnareb@gmail.com> wrote:\n>>\n>> A bit of time ago I have stumbled upon the following blog entry\n>> (question): \"Agaric wants version control that lets Drupal core and\n>> contrib replace entire directories within our checkouts\"[1]\n>>   http://tinyurl.com/yv3jp4\n> \n> While it is possible to use submodules, I tend to think that you can\n> just use your own repo / branch \"off\" the main project, just like\n> unofficial kernel modules do.\n\nI think it is why they currently do, and they want to change it...\n\n<blockquote href=\"http://tinyurl.com/yv3jp4\">\n  Right now we have each of these three areas checked out independently,\n  and while they do happily reside inside one another, there's some\n  inconvenience and risk of not keeping everything in synch.\n\n  The solution we're looking at now is shell scripts but if there's\n  a way for a version control system to just understand this need and\n  accept it, that would be best.\n</quote>\n\n...but I guess they use CVS just like Drupal does, which might make it\npainful.\n\n> It does have a number of advantages -- including being explicit wrt to\n> what version of the core project the module is developed against.\n> Drupal and similar projects (like Moodle) do have a module API, but\n> it's not *that* stable -- in spite of the best intentions, those APIs\n> get subtle changes all the time. You should have your \"release\" script\n> package only the module subdirectories -- and perhaps any delta in the\n> core (of Drupal) that needs to be applied. In many \"contrib\" projects\n> I've seen module files and little patch files that you are supposed to\n> apply - and maintenance of those is a pain. It makes more sense to use\n> git over the whole tree.\n> \n> So it's a tradeoff -- IMHO, at first blush it looks more \"natural\" to\n> use submodules, but as things progress over time I think it forces a\n> lot of additional and ill-fitted work like maintaining patches, or\n> adding tags that indicate \"developed against Drupal v1.2.3\". And the\n> day the contrib module becomes official, \"merging it in\" is a bit of a\n> mess if you want to preserve history. So in the long haul, it makes a\n> lot more sense to use the \"branch off the full project\" approach that\n> has served the kernel modules so well.\n\nI think that choice between developing \"in core\" / on branch like Linux \nkernel modules and subsystems vs using submodules support like msysGit, \ncgit and (perhaps) in the future git-gui and gitk, depends on a few \nthings:\n 1. How tightly coupled are modules with core / how independent\n    development of modules is\n 2. How tightly coupled / how independent modules are from each other,\n    both from the development point of view, but also from testing point\n    of view\n 3. How much work has to be done to add / enable module\n\nHow coupled modules are with the core depends among others on the \nstability of modular API.  The dependency of modules on core \ndevelopment might be measured by [relative] number of commits which \nchange core and module, or which change module responding to change\nin core (bugfixes and workaround for core bugs excluded).  And of course \nif module development _is_ done independently.\n\nIf modules are not developed in isolation (e.g. code movement), or \ncannot be tested in isolation (unit tests for modules alone, whole \ntests for core + single module) then of course it doesn't make much \nsense to use submodules support.  (For example it would be hard to test \nfor example networking module without filesystem module, for Linux \nkernel modules.)\n\nIf you have to run some install script to install / start using modules \n(for example phpBB modules are custom files + specialized patch-like \nfile, which allows modifying not only files but also databases), then \nthat's of no avail to use submodules because you have to use [custom] \nscripts to update nevertheless...\n\n> In that sense, I think that gitk's tree getting merged in a\n> subdirectory is good as an example of what git can do, but a bit\n> pointless as gitk depend quite tightly on the behaviour of git\n> commands. So distributing gitk separately would be a big pain as each\n> gitk version is usable against a narrow set of git revisions. Luckily,\n> it gets merged into git and that's how it gets distributed.\n\nErrr... AFAICT it is not true. It is true that gitk and git-gui require \nsufficiently new git version to work, but IIRC both have feature \nautodetection, which allows to use gitk and git-gui with older git (and \nof course the reverse: old gitk or git-gui with new git). I have used \ngitk from 'master' occassionaly, while having as default some older git \nversion installed from RPM (released version).\n\ngitk and git-gui development is managed independently from git, and can \nbe done using only gitk and git-gui sources.  So using submodules is a \ngood idea.  Additionally this allows to not checkout gitk and git-gui \nfor example if you don't have Tcl/Tk installed or don't use windowed \nenvironment.\n\ngitweb could also be managed as module... but currently there is no \nmaintainer who would manage gitweb.git repository...\n\n\nOn the other hand having gitk, git-gui, gitweb in git proper allow to \nuse just introduced features, and not only features available in last \nreleased version.\n\n\nBTW. if some tool is tightly coupled with git, it can use git.git \nrepository (or modification of thereof) as submodule, as shown in the \ncases of msysGit and cgit.\n\n-- \nJakub Narebski\nPoland\n"},{"id":"66735","messageId":"200801281144.51261.jnareb@gmail.com","threadId":"11754","inReplyTo":"200801270139.57830.jnareb@gmail.com","subject":"Re: Submodules use case: web development based on modular CMS","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2008-01-28T10:44:48Z","receivedAt":"2008-01-28T10:44:48Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Sun, 27 Jan 2008, Jakub Narebski wrote:\n\n> Somebody (in comments) proposed using Subversion externals; I have \n> proposed using Git submodules (subproject) support. \n\nBTW. Mercurial supports something similar by so called (unofficial) \n\"forest extension\":\n  http://www.selenic.com/mercurial/wiki/index.cgi/ForestExtension\n\nIf I understand it corectly the mechanism follow idea of manifest-like \nfile Mercurial uses, and is similar to one of the test implementations \nof submodule support before we had current \"in core\" version.\n\nI wonder if Bazaar-NG has similar extension / plugin...\n\n\nIt would be nice to have comparison of submodule / subproject support in \ndifferent version control systems...\n-- \nJakub Narebski\nPoland\n"}]}