{"thread":{"id":"12430","subject":"[RFC] git-submodule: multiple-level modules definition","startedAt":"2008-03-02T17:11:09Z","lastAt":"2008-03-04T13:27:25Z","messageCount":8,"participants":["Ping Yin","Lars Hjemli"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"70667","messageId":"46dff0320803020911m30c8e3b3w80c0c92258a45f0f@mail.gmail.com","threadId":"12430","inReplyTo":null,"subject":"[RFC] git-submodule: multiple-level modules definition","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-03-02T17:11:09Z","receivedAt":"2008-03-02T17:11:09Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"1. .gitmodules supports multiple submodule level definition\n\nThis follows suggestions from Linus and and Sven Verdoolaege etc.\nSee \"Let .git/config specify the url for submodules\"\n(http://article.gmane.org/gmane.comp.version-control.git/48939).\n\n.gitmodules with with multiple level of indirection\n------------------------------------------------------\n[submodule \"service\"]\n submodule = crawler\n submodule = search\n[submodule \"crawler\"]\n submodule = util\n submodule = imcrawter\n[submodule \"search\"]\n submodule = util\n submodule = imsearch\n[submodule \"util\"] (1)\n url = git://xyzzy/util.git\n[submodule \"imsearch\"]\n path = search/imsearch\n url = git://xyzzy/imsearch.git\n[submodule \"imcrawler\"]\n path = crawler/imcrawter\n url = git://xyzzy/imcrawter.git\n\n(1): path default to submodule name\n------------------------------------------------------\n\nTo simplify the case, submodule sections with submodule attribute should\nhave neither path attribute nor url attribute (if have, should be ignored).\n\n2. Designate modules by logical names instead of real path\n\n\"--name|-n\" options is proposed with which modules are designated by logical\nnames instead of real paths as following example shows.\n\nIdentical commands forms with/without \"--name\"\n---------------------------------------------------\n$ git submodule XXX util imcrawler (1)\n$ git submodule XXX -n crawler (2)\n$ git submodule XXX util imcrawler imsearch (3)\n$ git submodule XXX -n service (4)\n$ git submodule XXX -n crawler search (5)\n\n* XXX represent list, update or init, but not add\n* (1) and (2) are idetical conditionally (explained below)\n* (3), (4) and (5) are idetical conditionally\n---------------------------------------------------\n\nThere are still minor difference between these two forms.\n\nIn the no \"--name\" form, the path parameter may be not the real submodule\npath, and it just acts as the filter for real submodule paths.\n\nIn the \"--name\" form, name parameter must be the logical name, and the real\npaths corresponding to the logical name may be neither a submodule path nor\neven existent. How to handle such a path depends on the subcommand.\n\n 1) status: Output 0{40} as the sha1. Doing this can remind the user to\n add the path as submodule or delete the path from .gitmodules.\n 2) update: Skip that path and issue a warning\n 3) init: Also init for that path\n\nSo in the example above, commands (1) and (2) are identical only when util and\nimcrawler are already submodules.\n\n3. Add submodules in batch with \"git submodule add --name\"\n\nThe former workflow to add submodules is adding one by one with\n\"git submodule add url path\" which then modifies .gitmodules. However,\nsometimes it may be more convenient to work in the reverse way: edit\n.gitmodules first and then add submodules in batch.\n\nNow \"git submodule add --name modulename\" can help us to do that. It will\nfind all submodules corresponding to the logical name and add them in batch\nby using the path and url from .gitmodules. Of course, it will skip those\npaths which have already been submodules.\n\n4. Do we really need 'init' subcommand?\n\nI think .gitmodules should have been bound more tightly with \"git submodule\".\nNow it seems that 'update' or 'status' can still be performed without\n.gitmodules after \"git submodule init\".\n\nSo why not to read submodules info from .gitmodules directly just like what\n.gitignore nor .gitattributes have been dealt with? Just for editing them\nmore free?\n\n>From another point, correspoingding to .gitignore we have .git/info/exclude\nwhich just records some paths special to the local repository. But\n.git/info/exclude can't replace .gitignore. But why can .git/config replace\n.gitmodules which is a naive attribute of the repository?\n\nAfter introducing multi-level module definition, .gitmodules becomes\nmore naive to the repository. And it becomes more inappropriate to put\nall .gitmodules contents into .git/config since multi-level module\ndefinition shouldn't be edited outside .gitmodules.\n\n\n-- \nPing Yin\n"},{"id":"70683","messageId":"8c5c35580803021018v7590ad76kbbab2b1113e27666@mail.gmail.com","threadId":"12430","inReplyTo":"46dff0320803020911m30c8e3b3w80c0c92258a45f0f@mail.gmail.com","subject":"Re: [RFC] git-submodule: multiple-level modules definition","fromName":"Lars Hjemli","fromEmail":"hjemli@gmail.com","sentAt":"2008-03-02T18:18:45Z","receivedAt":"2008-03-02T18:18:45Z","isPatch":false,"sender":{"key":"hjemli@gmail.com","avatar":null},"body":"On Sun, Mar 2, 2008 at 6:11 PM, Ping Yin wrote:\n> 4. Do we really need 'init' subcommand?\n\nYes: .git/config tells git which submodule(s) the user is interested\nin and where to fetch updates from for these submodules, while\n.gitmodules is used to map a submodule _name_ to a submodule _path_\nand also to provide a url where submodule updates _can_ be fetched.\n\nSo the actual use of submodules is strictly a local configuration\nissue and as such it needs to be kept in .git/config. And to help the\nuser populate .git/config with submodule information we have 'git\nsubmodule init'.\n\n--\nlarsh\n"},{"id":"70720","messageId":"46dff0320803021747y6e334903pf8b5a77168bc7882@mail.gmail.com","threadId":"12430","inReplyTo":"8c5c35580803021018v7590ad76kbbab2b1113e27666@mail.gmail.com","subject":"Re: [RFC] git-submodule: multiple-level modules definition","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-03-03T01:47:06Z","receivedAt":"2008-03-03T01:47:06Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Mon, Mar 3, 2008 at 2:18 AM, Lars Hjemli wrote:\n> On Sun, Mar 2, 2008 at 6:11 PM, Ping Yin wrote:\n> > 4. Do we really need 'init' subcommand?\n>\n> Yes: .git/config tells git which submodule(s) the user is interested\n> in and where to fetch updates from for these submodules, while\n> .gitmodules is used to map a submodule _name_ to a submodule _path_\n> and also to provide a url where submodule updates _can_ be fetched.\n>\n> So the actual use of submodules is strictly a local configuration\n> issue and as such it needs to be kept in .git/config. And to help the\n> user populate .git/config with submodule information we have 'git\n> submodule init'.\nIs it always a local configuration issue? In a project with hundreds\nof submodule, will anybody modify the submodule url?\nI think .gitmodules can be the default repository configuration, and\nwhen we have local configuration issue, we can override the\nconfiguration in .git/config with the help of git init. But why always\nuse git init even when we don't have the local configuration\nrequirement?\n>\n> --\n> larsh\n>\n\n\n\n-- \nPing Yin\n"},{"id":"70747","messageId":"8c5c35580803022257n7ff0cdc3i1e72a0034d254aa4@mail.gmail.com","threadId":"12430","inReplyTo":"46dff0320803021747y6e334903pf8b5a77168bc7882@mail.gmail.com","subject":"Re: [RFC] git-submodule: multiple-level modules definition","fromName":"Lars Hjemli","fromEmail":"hjemli@gmail.com","sentAt":"2008-03-03T06:57:37Z","receivedAt":"2008-03-03T06:57:37Z","isPatch":false,"sender":{"key":"hjemli@gmail.com","avatar":null},"body":"On Mon, Mar 3, 2008 at 2:47 AM, Ping Yin wrote:\n> But why always\n> use git init even when we don't have the local configuration\n> requirement?\n\nWe need a way to tell git which of those submodules we are interested\nin, i.e. which submodules to clone/fetch during 'git submodule\nupdate'.\n\n--\nlarsh\n"},{"id":"70770","messageId":"46dff0320803030302t286c33a5h3555200c9e8840e3@mail.gmail.com","threadId":"12430","inReplyTo":"8c5c35580803022257n7ff0cdc3i1e72a0034d254aa4@mail.gmail.com","subject":"Re: [RFC] git-submodule: multiple-level modules definition","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-03-03T11:02:26Z","receivedAt":"2008-03-03T11:02:26Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Mon, Mar 3, 2008 at 2:57 PM, Lars Hjemli wrote:\n> On Mon, Mar 3, 2008 at 2:47 AM, Ping Yin wrote:\n> > But why always\n> > use git init even when we don't have the local configuration\n> > requirement?\n>\n> We need a way to tell git which of those submodules we are interested\n> in, i.e. which submodules to clone/fetch during 'git submodule\n> update'.\n>\nNow that's easy with the multiple level module definition. We can now\ngroup our modules by hierachy and designate modules we are interested\nby logical names such as crawler or search as my proposal has\nmentioned.\n\nAnd if there are not a logical module name satisfying our requirement,\nwe can define one by ourself in .git/config (if it's a special or\ntemporary requirement) or in .gitmodules (if it's a common\nrequirement).\n\nNow .git/config and .gitmodules will play the same as\n.git/info/exclude and .gitignore.\n> --\n> larsh\n>\n\n\n\n-- \nPing Yin\n"},{"id":"70773","messageId":"8c5c35580803030351l2e8d7eedq58108b54bf07dc5c@mail.gmail.com","threadId":"12430","inReplyTo":"46dff0320803030302t286c33a5h3555200c9e8840e3@mail.gmail.com","subject":"Re: [RFC] git-submodule: multiple-level modules definition","fromName":"Lars Hjemli","fromEmail":"hjemli@gmail.com","sentAt":"2008-03-03T11:51:34Z","receivedAt":"2008-03-03T11:51:34Z","isPatch":false,"sender":{"key":"hjemli@gmail.com","avatar":null},"body":"On Mon, Mar 3, 2008 at 12:02 PM, Ping Yin wrote:\n>\n> On Mon, Mar 3, 2008 at 2:57 PM, Lars Hjemli wrote:\n> > On Mon, Mar 3, 2008 at 2:47 AM, Ping Yin wrote:\n> > > But why always\n> > > use git init even when we don't have the local configuration\n> > > requirement?\n> >\n> > We need a way to tell git which of those submodules we are interested\n> > in, i.e. which submodules to clone/fetch during 'git submodule\n> > update'.\n> >\n> Now that's easy with the multiple level module definition. We can now\n> group our modules by hierachy and designate modules we are interested\n> by logical names such as crawler or search as my proposal has\n> mentioned.\n\nSorry, you've lost me here. How is 'git submodule update' going to\navoid cloning/fetching all submodules with your proposal?\n\nPS: the supposed work-flow is an initial 'git submodule init foo bar\nbaz' (i.e. register the submodules you're interested in checking out)\nfollowed by an occasional 'git submodule update' (i.e. update all the\nsubmodules you are interested in).\n\n--\nlh\n"},{"id":"70780","messageId":"46dff0320803030445o6f37e102heda4878c2d865347@mail.gmail.com","threadId":"12430","inReplyTo":"8c5c35580803030351l2e8d7eedq58108b54bf07dc5c@mail.gmail.com","subject":"Re: [RFC] git-submodule: multiple-level modules definition","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-03-03T12:45:08Z","receivedAt":"2008-03-03T12:45:08Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Mon, Mar 3, 2008 at 7:51 PM, Lars Hjemli wrote:\n> On Mon, Mar 3, 2008 at 12:02 PM, Ping Yin wrote:\n> >\n> > On Mon, Mar 3, 2008 at 2:57 PM, Lars Hjemli wrote:\n> > > On Mon, Mar 3, 2008 at 2:47 AM, Ping Yin wrote:\n> > > > But why always\n> > > > use git init even when we don't have the local configuration\n> > > > requirement?\n> > >\n> > > We need a way to tell git which of those submodules we are interested\n> > > in, i.e. which submodules to clone/fetch during 'git submodule\n> > > update'.\n> > >\n> > Now that's easy with the multiple level module definition. We can now\n> > group our modules by hierachy and designate modules we are interested\n> > by logical names such as crawler or search as my proposal has\n> > mentioned.\n>\n> Sorry, you've lost me here. How is 'git submodule update' going to\n> avoid cloning/fetching all submodules with your proposal?\n>\n> PS: the supposed work-flow is an initial 'git submodule init foo bar\n> baz' (i.e. register the submodules you're interested in checking out)\n> followed by an occasional 'git submodule update' (i.e. update all the\n> submodules you are interested in).\n>\n\nSee my first email. In following .gitmodules example, we can use \"git\nsubmodule update service\" to update all sub-submodules of submodule\n\"service\" (util, imsearch, imcrawler). We don't have to do \"git init\"\nfirst.\n\nWithout the multiple level module definition, \"git init\" is good idea,\nsince every time we want to 'git update', we needn't give so many\nmodule paths. However, if we introduce multiple level module\ndefinition, we can define the module groups we are interested such as\ncrawler, search and the toppest group service. So we just need to type\none or few module names which makes \"git init\" not so useful.\n\nAnd it's more flexible by using module name: at one time we can update\n\"cralwer\" module group, and at another time we can update \"search\"\nmodule group.\n\n\n.gitmodules example\n------------------------------------------------------\n[submodule \"service\"]\n submodule = crawler\n submodule = search\n[submodule \"crawler\"]\n submodule = util\n submodule = imcrawter\n[submodule \"search\"]\n submodule = util\n submodule = imsearch\n[submodule \"util\"] (1)\n url = git://xyzzy/util.git\n[submodule \"imsearch\"]\n path = search/imsearch\n url = git://xyzzy/imsearch.git\n[submodule \"imcrawler\"]\n path = crawler/imcrawter\n url = git://xyzzy/imcrawter.git\n------------------------------------------------------\n> --\n> lh\n>\n\n\n\n-- \nPing Yin\n"},{"id":"70939","messageId":"46dff0320803040527l681f8068n2455843b2bb23cc7@mail.gmail.com","threadId":"12430","inReplyTo":"46dff0320803020911m30c8e3b3w80c0c92258a45f0f@mail.gmail.com","subject":"Re: [RFC] git-submodule: multiple-level modules definition","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-03-04T13:27:25Z","receivedAt":"2008-03-04T13:27:25Z","isPatch":false,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Mon, Mar 3, 2008 at 1:11 AM, Ping Yin wrote:\n> 1. .gitmodules supports multiple submodule level definition\n>\n> This follows suggestions from Linus and and Sven Verdoolaege etc.\n> See \"Let .git/config specify the url for submodules\"\n> (http://article.gmane.org/gmane.comp.version-control.git/48939).\n>\n> .gitmodules with with multiple level of indirection\n> ------------------------------------------------------\n> [submodule \"service\"]\n> submodule = crawler\n> submodule = search\n> [submodule \"crawler\"]\n> submodule = util\n> submodule = imcrawter\n> [submodule \"search\"]\n> submodule = util\n> submodule = imsearch\n> [submodule \"util\"] (1)\n> url = git://xyzzy/util.git\n> [submodule \"imsearch\"]\n> path = search/imsearch\n> url = git://xyzzy/imsearch.git\n> [submodule \"imcrawler\"]\n> path = crawler/imcrawter\n> url = git://xyzzy/imcrawter.git\n>\n> (1): path default to submodule name\n> ------------------------------------------------------\n>\n> To simplify the case, submodule sections with submodule attribute should\n> have neither path attribute nor url attribute (if have, should be ignored).\n>\n> 2. Designate modules by logical names instead of real path\n>\n> \"--name|-n\" options is proposed with which modules are designated by logical\n> names instead of real paths as following example shows.\n>\n> Identical commands forms with/without \"--name\"\n> ---------------------------------------------------\n> $ git submodule XXX util imcrawler (1)\n> $ git submodule XXX -n crawler (2)\n> $ git submodule XXX util imcrawler imsearch (3)\n> $ git submodule XXX -n service (4)\n> $ git submodule XXX -n crawler search (5)\n>\n> * XXX represent list, update or init, but not add\n> * (1) and (2) are idetical conditionally (explained below)\n> * (3), (4) and (5) are idetical conditionally\n> ---------------------------------------------------\n>\n\nMaybe i should introduce more about the background.\n\nIn my company we try to keep all the codes in a unique big environment\nwhich consists of many sub projects (more than 30). These subprojects\nare hierachily organized. There are different roles which need touch\ndifferent part of these sub projects. For example, crawler deveoper\nshould touch crawler-related projects, and search developer need touch\nsearch-related projects. There are also some common modules which are\nused by most of these roles. There are also many other roles.\n\nSo for easy use, we need define one logical name for each kind of role\nand this name should correspond to all the role-related sub projects.\nFiltering submodules by path is not so useful for us because\nrole-related projects may live in many different directories.\n\n>From the management view of company, all the roles should use the\nsame big environment. And we don't want to use different branch for\ndifferent roles. So all roles will get the same .gitmodules file. They\nare not much familiar with git and can't afford to type git init path1\n... pathn each time when they want to build their develop environment.\nIt's more friendly to them if they only need to remember a logical\nname such as crawler or search.\n\nAnd some people have multiple roles. Sometime they want to work on\nsearch and sometimes on crawler. So logical name is very convenient to\nthem.\n\nAs a conclusion, my main point is\n1. Multiple level module definition is very useful since it can help\nus build hierachy of modules\n2. Designating modules by logical name which corresponds to a group of\nrole-related modules is usefull since it can help role-based\ndevelopment much.\n\n\n\n-- \nPing Yin\n"}]}