{"thread":{"id":"46712","subject":"gitmodules below root directory","startedAt":"2017-09-06T13:53:36Z","lastAt":"2017-09-07T01:26:22Z","messageCount":4,"participants":["Robert Dailey","Stefan Beller","Junio C Hamano","Jacob Keller"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"327653","messageId":"CAHd499AuoZ-89mpnVkMhxaDT44SRNT2xWE7sykcaP8n5xTRd-g@mail.gmail.com","threadId":"46712","inReplyTo":null,"subject":"gitmodules below root directory","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2017-09-06T13:53:30Z","receivedAt":"2017-09-06T13:53:36Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"The gitmodules documentation[1] states that the .gitmodules file is at\nthe root. However, it would be nice if this could be supported in any\ndirectory similar to how .gitignore works. Right now git-subrepo does\nnot support submodules inside of a subrepo[2] (I suspect subtrees\nwould have the same problem, but I did not verify). I think this is a\nlimitation of git, rather than subrepo itself. Perhaps there are\nreasons why .gitmodules must be at the root, but I at least wanted to\npoint it out and see if this could be supported.\n\n[1]: https://git-scm.com/docs/gitmodules\n[2]: https://github.com/ingydotnet/git-subrepo/issues/262\n"},{"id":"327663","messageId":"CAGZ79kaLdw0bX2gfw6AxQtQJZD9iiZGhYPJt9HwUh9T_+0T=fw@mail.gmail.com","threadId":"46712","inReplyTo":"CAHd499AuoZ-89mpnVkMhxaDT44SRNT2xWE7sykcaP8n5xTRd-g@mail.gmail.com","subject":"Re: gitmodules below root directory","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-09-06T18:35:16Z","receivedAt":"2017-09-06T18:35:26Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Sep 6, 2017 at 6:53 AM, Robert Dailey <rcdailey.lists@gmail.com> wrote:\n> The gitmodules documentation[1] states that the .gitmodules file is at\n> the root. However, it would be nice if this could be supported in any\n> directory similar to how .gitignore works. Right now git-subrepo does\n> not support submodules inside of a subrepo[2] (I suspect subtrees\n> would have the same problem, but I did not verify). I think this is a\n> limitation of git, rather than subrepo itself. Perhaps there are\n> reasons why .gitmodules must be at the root, but I at least wanted to\n> point it out and see if this could be supported.\n>\n> [1]: https://git-scm.com/docs/gitmodules\n> [2]: https://github.com/ingydotnet/git-subrepo/issues/262\n\nI agree that subtree likely suffers the same problem.\nAnd at first it seems reasonable to want to have .gitmodules\nat deeper trees supported, as that would fix subtree and subrepo\n(and others) with ease.\n\nHistorically the need to store submodule URLs were the motivation\nfor having the .gitmodules file. An absolute URL for a submodule would\nwork fine no matter where the .gitmodules file would be located.\nRelative URLs are currently defined as relative to the top level\nof the project, which we would need to inspect if the anchor\nis chosen well at the root or if we would want to allow anchoring\nthe relative URL within the tree. (This is no reason against\n.gitmodules in deep trees, just pointing out the work required).\n\nBut does the URL still make sense? For absolute URLs this is likely\nthe case, for relative URLs my bets are off. Maybe?\n\nIt turned out that people want to e.g. move, delete and re-introduce\nsubmodules, which is why the location of a submodule git directory\nwas moved to be either inside the tree (to keep supporting existing\ngit repos with submodules) as well as interned in the superproject.\n\nIn the example given in [2], the git dir of the submodule\n(\"folder B\") may be located at .git/modules/nameB as seen\nfrom the root of RepoX:\n\n    RepoX\n    + folder A\n    + folder B (submodule)\n    + .gitmodules\n    + .git # regular RepoX git dir\n       + modules/<nameB>\n\nAn important mechanism of the .gitmodules file is\nthe resolution of the \"name\" and the \"path\" of the\nsubmodule. (Given the path of a gitlink entry, where do\nI find the git repository for the submodule? vice versa is slightly\nless relevant: Given this git repository deep inside my own git\ndirectory, where is the working tree)\n\nSo in the example we'd have\n\n    RepoY\n    + RepoX (subrepo)\n      + folder A\n      + folder B (submodule)\n     +.gitmodules\n\nThe path entry in the .gitmodules file would not change via\nsubtree/subrepo merge, such that Git would need to know\nthat the actual path to the submodule is the\nconcatenation of 'path to tree in which the .gitmodules file is'\nand the given path inside the .gitmodules file. Seems doable so far.\n\nWhat about the name of a submodule? The .gitmodules file\nfollows the syntax of git config files, such that names cannot\noccur twice as the names are stored as the section name:\n\n  [submodule \"nameB\"]\n    path = \"folder B\"\n\nAnd I would think the property of having unique names\nis important, such that each submodule has its unique\nplace to put its git dir inside the superprojects\n\"$GIT_DIR/modules/\".\n\nWith multiple .gitmodules files, we would loose the\nuniqueness property. (It may not be too bad, maybe\neven a clever hack, haven't thought about it deeply,\nbut it seems ugly at first)\n\nAs said above, the name<->path resolution is\nimportant, (and shall be unique, deterministic and simple),\nso how do we do it? What about the case where we have\n\n  .gitmodules \"name\" -> dir/path\n  dir/.gitmodules \"name\" -> ./path\n\nIn this case we'd have the same mapping, but using this\nmechanism we can map multiple names at the same path,\nand we could choose to resolve a given path in different\n.gitmodules files, which is cumbersome.\n\n  anotherdir/.gitmodules \"name\" -> ../dir/path\n\nseems crazy, too.\n\nWhat about moving submodules?\nConsider the example as in [2] again:\n\n  $ git mv RepoX/folderB dir/sub\n  $ git commit -m \"move submodule\"\n  # ok fine, we can come up with a plan\n  # where to put the submodule configuration,\n  # maybe in dir/.gitmodules?\n\n  $ git rm RepoX\n  $ git commit -m \"don't need the rest of RepoX\"\n  # observation: we would not want\n  # RepoX/.gitmodules to still have impact on\n  # the submodule.\n\n  $ git revert HEAD^^ # undo the initial move\n  # we'd move the .gitmodules file back to RepoX/.\n\ntl;dr: I think this idea produces lots of interesting\ncorner cases in the data model, let's not go there\nwithout having an idea how to solve them.\n\nFrom an implementation stand point:\nThe submodule-config API could easily enhanced\nto support reading multiple .gitmodules files (in case\ntheir location is well defined, we would not want to\nwalk the whole tree recursively). This API is only\neasily accessible from within C, such that current\nimplementing this idea in git-submodule.sh would\nbe a hassle to do. Prathamesh made good progress\nin his GSoC project porting most of git-submodule.sh to\nC, though, so once that is merged I'd claim that the\nactual implementation of this idea is \"rather easy\".\n\nThanks,\nStefan\n"},{"id":"327670","messageId":"xmqqk21b4n0k.fsf@gitster.mtv.corp.google.com","threadId":"46712","inReplyTo":"CAHd499AuoZ-89mpnVkMhxaDT44SRNT2xWE7sykcaP8n5xTRd-g@mail.gmail.com","subject":"Re: gitmodules below root directory","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-09-06T19:58:51Z","receivedAt":"2017-09-06T19:58:59Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Robert Dailey <rcdailey.lists@gmail.com> writes:\n\n> The gitmodules documentation[1] states that the .gitmodules file is at\n> the root. However, it would be nice if this could be supported in any\n> directory similar to how .gitignore works.\n\nI have a mild suspicion that there would be a huge impedance\nmismatch between what gitmodules file is meant to do and the way\nignore/attribute setting is done.\n\nWhen the mechanism is primarily about expressing a few generic\ntraits that are shared by things that can be grouped by paths\n(e.g. \"all paths whose pathnames match '*.py' pattern contain text\",\n\"all paths in sub/ directory are ignored\"), it may make sense to\nspread the information across multiple .gitignore files and make the\nclosest one take precedence over the further ones.  Even though\nallowing multiple sources of information spread over the tree leads\nto end-user confusion (e.g. \"why is this path ignored?\", which\ntriggered the debugging aid \"git check-ignore\"), such a grouping by\npattern matching on paths (which is what makes \"closest file take\nprecedence\" meaningful) to assign generic traits (e.g. \"it's text\")\nmakes it worthwhile by allowing to express the rules more concisely.\n\nCompared to that, what .gitmodules file expresses is more specific\nto each submodule---no two submodules in your single superproject\nwould share the same URL, unless you are doing something quite\nunusual, for example.  Having a single file also means that updating\nis much simpler---\"git submodule add\" and other things do not have\nto choose among .gitmodules, a/.gitmodules and a/b/.gitmodules when\nthey update an entry for the submodule at path \"a/b/c\".\n\nHaving said that, I do not think the current \".gitmodules must be at\nthe top and nothing else matters\" is ideal.  A possible change that\nI suspect may make more sense is to get rid of .gitmodules file,\ninstead of spreading more of them all over the tree.\n\nThe current gitlink implementation records only the \"what commit\nfrom the subproject history is to be checked out at this path?\" and\nnothing else, by storing a single SHA-1 that happens to be the name\nof the commit object (but the superproject does not even care the\nfact that it is a commit or a random string).  We could substitute\nthat with the name of a blob object that belongs to the superproject\nhistory and records the information about the submodule at the path\n(e.g. \"which repository the upstream project recommends to clone the\nsubproject from?\", \"what commit object is to be checked out\").  \n\nWhen you see a single tree of a superproject, you need to see what\ncommit is to be checked out from the tree object and everything else\nneeds to be read from the .gitmodules file in that tree in the\ncurrent system, but it does not have to be that way.\n\n\n\n\n\n"},{"id":"327686","messageId":"CA+P7+xrbmx7JaCfxKLwKqCTXVBMd4mSg8zpxDLqRoxVAJDwhtw@mail.gmail.com","threadId":"46712","inReplyTo":"xmqqk21b4n0k.fsf@gitster.mtv.corp.google.com","subject":"Re: gitmodules below root directory","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-09-07T01:25:53Z","receivedAt":"2017-09-07T01:26:22Z","isPatch":false,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Wed, Sep 6, 2017 at 12:58 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> The current gitlink implementation records only the \"what commit\n> from the subproject history is to be checked out at this path?\" and\n> nothing else, by storing a single SHA-1 that happens to be the name\n> of the commit object (but the superproject does not even care the\n> fact that it is a commit or a random string).  We could substitute\n> that with the name of a blob object that belongs to the superproject\n> history and records the information about the submodule at the path\n> (e.g. \"which repository the upstream project recommends to clone the\n> subproject from?\", \"what commit object is to be checked out\").\n>\n> When you see a single tree of a superproject, you need to see what\n> commit is to be checked out from the tree object and everything else\n> needs to be read from the .gitmodules file in that tree in the\n> current system, but it does not have to be that way.\n>\n>\n\nIMO, this approach described here, (point the gitlink at a blob which\ndescribes the full contents, URL, etc) would make more sense. The\ntrickiest parts I think are (a) it really requires tooling to change\nthe git module vs just editing a file, and (b) we'd need to prevent\nthe blobs from getting garbage collected.\n\nI think it makes each individual submodule a bit more robust, since\nthe actual submodule pointer always points directly to the full data\nabout that submodule (it's recommended URL, it's path, etc), and\nchanges to those things *are* changes to the submodule pointer.\n\nThanks,\nJake\n"}]}