{"thread":{"id":"35648","subject":"Submodule relative URL problems","startedAt":"2014-01-13T14:02:56Z","lastAt":"2014-01-13T20:44:07Z","messageCount":4,"participants":["Lianheng Tong","Jonathan Nieder","Heiko Voigt","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"233053","messageId":"DAD6CF7D-49F2-4FB5-9EF2-EBAAF86881AA@kcl.ac.uk","threadId":"35648","inReplyTo":"DC691CA7-BE36-4FE7-895A-FE8E1FD0C080@kcl.ac.uk","subject":"Submodule relative URL problems","fromName":"Lianheng Tong","fromEmail":"lianheng.tong@kcl.ac.uk","sentAt":"2014-01-13T14:02:56Z","receivedAt":"2014-01-13T14:02:56Z","isPatch":false,"sender":{"key":"lianheng.tong@kcl.ac.uk","avatar":null},"body":"Hi All,\n\nI am currently working on a project, call it A, which contains several sub-projects B, C, D.  The B, C, D are actually experimental input and outputs for simulation runs, which I need to keep track of the daily changes.  Most of these calculations are done on workstation, but I do need to occasionally clone the repositories to other machines, and then push the updates back.\n\nI version track B, C and D with independent git repositories, and organisationally it makes sense for them to be subdirectories of A.  I would like to version control the overall project A, with B, C, D as submodules.\n\nThis is easily done on the main workstation (I will call it W1), with:\n\ncd A\ngit init\nfor ii in B C D ; do\n  git submodule ./${ii}/.git\ndone\ngit add .\ngit commit\n\nRepository A/.git on W1 does not have an origin.  All seems to work fine with this step up on W1.\n\nHOWEVER, problem starts to appear when I attempted to clone the overall repository A to another workstation W2:\n\ngit clone W1:<path to A on W1>/.git  <path to A on W2>\ncd <path to A on W2>\ngit submodule init\ngit submodule update\n\ngit then produces an error indicating that:\n\nW1:<path to A on W1>/.git/B/.git\n\nis not a valid repository (i.e. directory not found)\n\nThe extra .git has been inserted in the URL to submodule repository B, C, and D.\n\nThis can be solved by changing the .gitmodules file, replacing ./B/.git to ../B/.git,  and then git submodule sync on W2.\n\nHowever, this is far from being an optimal solution, because if .gitmodules on W1 is updated accordingly, then the submodule repository for B would point to\n\n<path to A on W1>/../B/.git\n\nwhich will be incorrect.  I could in principle define the URL for the submodule repositories as absolute paths, but keeping them relative to A has many advantages, such as I will be able to move A around, without effecting the main repositories on W1 in anyway, and I can rsync them to another disk for backup etc.\n\nI would be very grateful if someone can offer me an solution to my problem.\n\nMany thanks in advance!\n\nLianheng\n\n================================================\nLianheng Tong                                    Tel: +44 79 1758 3822  \nRoom S4.02, Strand Building           Fax: +44 20 7848 2420  \nDepartment of Physics                       lianheng.tong@kcl.ac.uk\nKings College London                                  \nStrand, London WC2R 2LS, U.K.                         \n================================================"},{"id":"233060","messageId":"20140113195518.GB18964@google.com","threadId":"35648","inReplyTo":"DAD6CF7D-49F2-4FB5-9EF2-EBAAF86881AA@kcl.ac.uk","subject":"Re: Submodule relative URL problems","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2014-01-13T19:55:18Z","receivedAt":"2014-01-13T19:55:18Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nLianheng Tong wrote:\n\n> git clone W1:<path to A on W1>/.git  <path to A on W2>\n\nInteresting.\n\nThoughts:\n\n * More typical usage is to clone from a bare repository (A.git), which\n   wouldn't have this problem.  But I think your case is worth\n   supporting, too.\n\n * What would you think of putting symlinks in A's .git directory?\n\n\tcd A/.git\n\tln -s ../B ../C ../D .\n\n * Perhaps as a special case when the superproject is foo/.git, git\n   should treat relative submodule paths as relative to foo/ instead\n   of relative to foo/.git/.  I think that would take care of your\n   case without breaking existing normal practices, though after the\n   patch is made it still wouldn't take care of people using old\n   versions of git without that patch.  What do you think?\n\nThanks,\nJonathan\n"},{"id":"233064","messageId":"20140113203118.GA2606@sandbox-ub","threadId":"35648","inReplyTo":"20140113195518.GB18964@google.com","subject":"Re: Re: Submodule relative URL problems","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2014-01-13T20:31:18Z","receivedAt":"2014-01-13T20:31:18Z","isPatch":false,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"Hi,\n\nOn Mon, Jan 13, 2014 at 11:55:18AM -0800, Jonathan Nieder wrote:\n> Lianheng Tong wrote:\n> \n> > git clone W1:<path to A on W1>/.git  <path to A on W2>\n> \n> Interesting.\n> \n> Thoughts:\n> \n>  * More typical usage is to clone from a bare repository (A.git), which\n>    wouldn't have this problem.  But I think your case is worth\n>    supporting, too.\n> \n>  * What would you think of putting symlinks in A's .git directory?\n> \n> \tcd A/.git\n> \tln -s ../B ../C ../D .\n> \n>  * Perhaps as a special case when the superproject is foo/.git, git\n>    should treat relative submodule paths as relative to foo/ instead\n>    of relative to foo/.git/.  I think that would take care of your\n>    case without breaking existing normal practices, though after the\n>    patch is made it still wouldn't take care of people using old\n>    versions of git without that patch.  What do you think?\n\nI do not fully get the repository layout, since some commands simply do\nnot work. Nevertheless I think what Lianheng Tong is running into is\nthe following:\n\n * If a superproject has *no remote* a relative submodule url is relative\n   to the *superproject itself*\n * If a superproject has *a remote* a relative submodule url is relative\n   to the *superprojects remote*\n\nThe simplest solution is: Have central bare repositories for everything\nso that every workstation has the same remote.\n\nThe second solution: Make sure both repositories have each other as a\nremote. But then you run into a chicken/egg problem when setting the two\nup.\n\nThe interpretation of relative urls was a design decision back when the\nrelative urls were introduced. I am quite sure it would produce a lot of\nfallout if we change that.\n\nIf I get your usecase wrong it would be very helpful if you could\nprovide us with a working script that creates the repository setup your\nare using.\n\nCheers Heiko\n"},{"id":"233065","messageId":"xmqqppnv6294.fsf@gitster.dls.corp.google.com","threadId":"35648","inReplyTo":"20140113195518.GB18964@google.com","subject":"Re: Submodule relative URL problems","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-01-13T20:44:07Z","receivedAt":"2014-01-13T20:44:07Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n>  * More typical usage is to clone from a bare repository (A.git), which\n>    wouldn't have this problem.  But I think your case is worth\n>    supporting, too.\n\nI think the relative URL among nested submodules was specifically\ndesigned for hosting environments that have forest of bare\nrepositories to serve as publishing or meeting points.  I personally\ndo not know where that \"worth supporting\" comes from, but if the\nchange can be done without confusing the codepaths involved, I\nwouldn't object too much ;-)\n\n>  * Perhaps as a special case when the superproject is foo/.git, git\n>    should treat relative submodule paths as relative to foo/ instead\n>    of relative to foo/.git/.  I think that would take care of your\n>    case without breaking existing normal practices, though after the\n>    patch is made it still wouldn't take care of people using old\n>    versions of git without that patch.  What do you think?\n\nIf we were to start adding special cases, it may also be sensible to\nclean up the more normal case of using subdirectories of bare\nrepositories to represent nestedness (i.e. you can have a submodule\n\"logs.git\", but not \"logs\").  We could reuse the $GIT_DIR/modules/$sm_name\nconvention somehow perhaps?\n\nAny change to implement the special case you suggest likely has to\ntouch the same place as such a change does, as both require some\nreorganization of the code that traverses the pathnames to find\nrelated repositories.\n"}]}