{"thread":{"id":"29266","subject":"[PATCH] Submodules always use a relative path to gitdir","startedAt":"2011-12-29T21:00:26Z","lastAt":"2012-01-06T18:53:20Z","messageCount":16,"participants":["Antony Male","Junio C Hamano","Fredrik Gustafsson","Phil Hord","Jens Lehmann","Nguyen Thai Ngoc Duy"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"181787","messageId":"1325192426-10103-1-git-send-email-antony.male@gmail.com","threadId":"29266","inReplyTo":null,"subject":"[PATCH] Submodules always use a relative path to gitdir","fromName":"Antony Male","fromEmail":"antony.male@gmail.com","sentAt":"2011-12-29T21:00:26Z","receivedAt":"2011-12-29T21:00:26Z","isPatch":true,"sender":{"key":"antony.male@gmail.com","avatar":"https://gravatar.com/avatar/44ebc98c2d05837be689ed72514f8119844424a9cf099e5202be4190281982f1?d=mp&s=160"},"body":"This fixes a problem where moving a git repository with checked-out\nsubmodules would cause a fatal error when commands such as 'git\nsubmodule update' were run.\n\nGit submoule clone uses git clone --separate-git-dir to checkout a\nsubmodule's git repository into <supermodule>/.git/modules, if this\nfolder does not already exist. git clone --separate-git-dir was\ndesigned for a scenario where the git repository stays in one location\nand the working copy can be moved. Therefore the .git file in the\nworking copy uses an absolute path to specify the location of the\nrepository.\n\nIn the submodules scenario, neither the git repository nor the working\ncopy will be moved relative to each other. However, the supermodule may\nbe moved, which moves both the submodule's git repository and its\nworking copy. This means that the submodule's .git file no longer\npoints to its repository, causing the error.\n\nPreviously, if git submodule clone was called when the submodule's git\nrepository already existed in <supermodule>/.git/modules, it would\nsimply re-create the submodule's .git file, using a relative path.\nThis patch uses the above mechanism to re-write the .git file after git\nclone --separate-git-dir is run, replacing the absolute path with a\nrelative one.\n\nAn alternative patch would teach git-clone an option to control whether\nan absolute or relative path is used when --separate-git-dir is passed.\n\nSigned-off-by: Antony Male <antony.male@gmail.com>\n---\n git-submodule.sh |    2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 3adab93..18eb5ff 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -159,7 +159,6 @@ module_clone()\n \tif test -d \"$gitdir\"\n \tthen\n \t\tmkdir -p \"$path\"\n-\t\techo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n \t\trm -f \"$gitdir/index\"\n \telse\n \t\tmkdir -p \"$gitdir_base\"\n@@ -171,6 +170,7 @@ module_clone()\n \t\tfi ||\n \t\tdie \"$(eval_gettext \"Clone of '\\$url' into submodule path '\\$path' failed\")\"\n \tfi\n+\techo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n }\n \n #\n-- \n1.7.8\n"},{"id":"181788","messageId":"7vsjk3vw67.fsf@alter.siamese.dyndns.org","threadId":"29266","inReplyTo":"1325192426-10103-1-git-send-email-antony.male@gmail.com","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-12-29T22:40:32Z","receivedAt":"2011-12-29T22:40:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Antony Male <antony.male@gmail.com> writes:\n\n> Git submoule clone uses git clone --separate-git-dir to checkout a\n> submodule's git repository into <supermodule>/.git/modules,...\n\nThis is misleading. The <superproject>/.git/modules/<name> is the location\nof the $GIT_DIR for the submodule <name>, not the location of its checkout\nat <superproject>/<path> that is outside <superproject>/.git/modules/\nhierarchy.\n\n> folder does not already exist. git clone --separate-git-dir was\n> designed for a scenario where the git repository stays in one location\n> and the working copy can be moved.\n\nAre you sure about this \"clone's design\"? It sounds like a revisionist\nhistory.\n\nSaying something like \"it would be nicer if it also let us use in this new\ndifferent scenario\" in the proposed commit log message is perfectly fine,\nbut my understanding is that the --separate-git-dir option and \"gitdir: \"\nsupport were designed to allow having the $GIT_DIR in a different place\nfrom the working tree that has \".git\" in it, nothing more, nothing less. I\ndo not think we meant to support moving either directory after they are\nset up. If you want to move either, you would need to (and you can, like\nyour patch does) tweak \"gitdir:\" to adjust.\n\nBy-the-way-Nit. We do not use any folders. s/folder/directory/.\n\n> In the submodules scenario, neither the git repository nor the working\n> copy will be moved relative to each other. However, the supermodule may\n> be moved,...\n\nAgain, who said that you are allowed to move the superproject directory in\nthe first place? I would understand that it might be nicer if it could be\nmoved but I haven't thought this through thoroughly yet---there may be\nother side effects from doing so, other than the relativeness of \"gitdir\".\n\n> Previously, if git submodule clone was called when the submodule's git\n> repository already existed in <supermodule>/.git/modules, it would\n> simply re-create the submodule's .git file, using a relative path.\n\n... \"to point at the existing <superproject>/.git/modules/<name>\".\n\nOverall, I think I can agree with the goal, but the tone of the proposed\ncommit log message rubs the reader in a wrong way to see clearly what this\npatch is proposing to do and where its merit lies. It is probably not a\nbig deal, and perhaps it may be just the order of explanation.\n\nI would probably explain the goal like this if I were doing this patch,\nwithout triggering any need for revisionist history bias.\n\n    Recent versions of \"git submodule\" maintain the submodule <name> at\n    <path> in the superproject using a \"separate git-dir\" mechanism. The\n    repository data for the submodule is stored in \".git/modules/<name>/\"\n    directory of the superproject, and its working tree is created at\n    \"<path>/\" directory, with \"<path>/.git\" file pointing at the\n    \".git/modules/<name>/\" directory.\n\n    This is so that we can check out an older version of the superproject\n    that does not yet have the submodule <name> anywhere without losing\n    (and later having to re-clone) the submodule repository. Removing\n    \"<path>\" won't lose \".git/modules/<name>\", and a different branch that\n    has the submodule at different location in the superproject, say\n    \"<path2>\", can create \"<path2>/\" and \".git\" in it to point at the same\n    \".git/modules/<name>\".\n\n    When instantiating such a submodule, if \".git/modules/<name>/\" does\n    not exist in the superproject, the submodule repository needs to be\n    cloned there first. Then we only need to create \"<path>\" directory,\n    point \".git/modules/<name>/\" in the superproject with \"<path>/.git\",\n    and check out the working tree.\n\n    However, the current code is not structured that way. The codepath to\n    deal with newly cloned submodules uses \"git clone --separate-git-dir\"\n    and creates \"<path>\" and \"<path>/.git\". This can make the resulting\n    submodule working tree at \"<path>\" different from the codepath for\n    existing submodules. An example of such differences is that this\n    codepath prepares \"<path>/.git\" with an absolute path, while the\n    normal codepath uses a relative path.\n\nWhen explained this way, the remedy is quite clear, and the change is more\nforward-looking, isn't it?  If we later start doing more in the codepath\nto deal with existing submodules, your patch may break without having\nextra code to cover the \"newly cloned\" case, too.\n\nI further wonder if we can get away without using separate-git-dir option\nin this codepath, though. IOW using\n\n        git clone $quiet -bare ${reference:+\"$reference\"} \"$url\" \"$gitdir\"\n\nmight be a better solution.\n\nFor example (this relates to the point I mumbled \"haven't thought this\nthrough thoroughly yet\"), doesn't the newly cloned repository have\ncore.worktree that points at the working tree that records the <path>,\nwhich would become meaningless when a commit in the superproject that\nbinds the submodule at different path <path2>?\n\n git-submodule.sh |   21 ++++++++-------------\n 1 files changed, 8 insertions(+), 13 deletions(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 3adab93..9a23e9d 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -156,21 +156,16 @@ module_clone()\n \t\t;;\n \tesac\n \n-\tif test -d \"$gitdir\"\n+\tif ! test -d \"$gitdir\"\n \tthen\n-\t\tmkdir -p \"$path\"\n-\t\techo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n-\t\trm -f \"$gitdir/index\"\n-\telse\n-\t\tmkdir -p \"$gitdir_base\"\n-\t\tif test -n \"$reference\"\n-\t\tthen\n-\t\t\tgit-clone $quiet \"$reference\" -n \"$url\" \"$path\" --separate-git-dir \"$gitdir\"\n-\t\telse\n-\t\t\tgit-clone $quiet -n \"$url\" \"$path\" --separate-git-dir \"$gitdir\"\n-\t\tfi ||\n-\t\tdie \"$(eval_gettext \"Clone of '\\$url' into submodule path '\\$path' failed\")\"\n+\t\tgit clone $quiet -n ${reference:+\"$reference\"} \\\n+\t\t\t--separate-git-dir \"$gitdir\" \"$url\" \"$path\" ||\n+\t\tdie \"$(eval_gettext \"Clone of '\\$url' for submodule '\\$name' failed\")\n \tfi\n+\n+\tmkdir -p \"$path\"\n+\techo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n+\trm -f \"$gitdir/index\"\n }\n \n #\n"},{"id":"181789","messageId":"CAJmizVZ_n9KmKWwDeLuYxBTWvndh5cTcvUbFduOtEcOPL=_WeQ@mail.gmail.com","threadId":"29266","inReplyTo":"1325192426-10103-1-git-send-email-antony.male@gmail.com","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Fredrik Gustafsson","fromEmail":"iveqy@iveqy.com","sentAt":"2011-12-29T22:48:41Z","receivedAt":"2011-12-29T22:48:41Z","isPatch":true,"sender":{"key":"iveqy@iveqy.com","avatar":"https://avatars.githubusercontent.com/u/761743?v=4"},"body":"2011/12/29 Antony Male <antony.male@gmail.com>:\n<snip>\n>                die \"$(eval_gettext \"Clone of '\\$url' into submodule path '\\$path' failed\")\"\n>        fi\n> +       echo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n\nThis will replace an already created file. Is it really the best\nsolution to create a gitfile with an absolute path and after that\nreplace it with a relative path. Why not write the relative path from\nthe beginning?\n\nThe patch also breaks two tests:\nt7406 and t5526.\n\nRegards\nFredrik\n"},{"id":"181816","messageId":"CABURp0oVWwNcYjzPoMvXrDKtDnpeLGXQhdxGnsARmBt9ZbUcHg@mail.gmail.com","threadId":"29266","inReplyTo":"1325192426-10103-1-git-send-email-antony.male@gmail.com","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2011-12-31T20:31:31Z","receivedAt":"2011-12-31T20:31:31Z","isPatch":true,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Thu, Dec 29, 2011 at 4:00 PM, Antony Male <antony.male@gmail.com> wrote:\n> This fixes a problem where moving a git repository with checked-out\n> submodules would cause a fatal error when commands such as 'git\n> submodule update' were run.\n\nThanks.  I noticed this itch when looking at git-files a few months\nago.  It bothered me, but not enough to fix it;  just enough to note\nit as a problem area to avoid in the future.\n\n> An alternative patch would teach git-clone an option to control whether\n> an absolute or relative path is used when --separate-git-dir is passed.\n\nI think I like this option better. Did you look at what it would take?\n\nPhil\n"},{"id":"181817","messageId":"CABURp0pdvf9Eo_pM2UCYUBANOJOGON6pQS-SXuCWQE=s2XNOfQ@mail.gmail.com","threadId":"29266","inReplyTo":"7vsjk3vw67.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2011-12-31T21:28:11Z","receivedAt":"2011-12-31T21:28:11Z","isPatch":true,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Thu, Dec 29, 2011 at 5:40 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Antony Male <antony.male@gmail.com> writes:\n>\n>> Git submoule clone uses git clone --separate-git-dir to checkout a\n>> submodule's git repository into <supermodule>/.git/modules,...\n>\n> This is misleading. The <superproject>/.git/modules/<name> is the location\n> of the $GIT_DIR for the submodule <name>, not the location of its checkout\n> at <superproject>/<path> that is outside <superproject>/.git/modules/\n> hierarchy.\n\nYes, so I think a simple s/checkout/clone/ should fix it.\n\n[...]\n\n>> In the submodules scenario, neither the git repository nor the working\n>> copy will be moved relative to each other. However, the supermodule may\n>> be moved,...\n>\n> Again, who said that you are allowed to move the superproject directory in\n> the first place? I would understand that it might be nicer if it could be\n> moved but I haven't thought this through thoroughly yet---there may be\n> other side effects from doing so, other than the relativeness of \"gitdir\".\n\nPreviously it was accepted practice to clone a local repo with rsync.\nThis method continues to work well even with submodules before\ngit-files became the norm.  But now it breaks because of the absolute\npaths.\n\nSimilarly, clones on network mounts and portable drives where absolute\npaths may change from time to time or machine to machine will also\nbreak now but worked before.\n\nSo, who said you were NOT allowed to move the superproject directory\ndirectory in the first place?  It seems natural that you should be\nable to do so, especially since the submodules are all contained\nwithin the superproject path.\n\n\n>> Previously, if git submodule clone was called when the submodule's git\n>> repository already existed in <supermodule>/.git/modules, it would\n>> simply re-create the submodule's .git file, using a relative path.\n>\n> ... \"to point at the existing <superproject>/.git/modules/<name>\".\n>\n> Overall, I think I can agree with the goal, but the tone of the proposed\n> commit log message rubs the reader in a wrong way to see clearly what this\n> patch is proposing to do and where its merit lies. It is probably not a\n> big deal, and perhaps it may be just the order of explanation.\n>\n> I would probably explain the goal like this if I were doing this patch,\n> without triggering any need for revisionist history bias.\n>\n>    Recent versions of \"git submodule\" maintain the submodule <name> at\n>    <path> in the superproject using a \"separate git-dir\" mechanism. The\n>    repository data for the submodule is stored in \".git/modules/<name>/\"\n>    directory of the superproject, and its working tree is created at\n>    \"<path>/\" directory, with \"<path>/.git\" file pointing at the\n>    \".git/modules/<name>/\" directory.\n>\n>    This is so that we can check out an older version of the superproject\n>    that does not yet have the submodule <name> anywhere without losing\n>    (and later having to re-clone) the submodule repository. Removing\n\nRevisionism nit: the real danger here is that you lose local commits.\n\n>    \"<path>\" won't lose \".git/modules/<name>\", and a different branch that\n>    has the submodule at different location in the superproject, say\n>    \"<path2>\", can create \"<path2>/\" and \".git\" in it to point at the same\n>    \".git/modules/<name>\".\n\nThis doesn't explain why one path is absolute and one is relative.\nBut I don't suppose this is the place for historical documentation\nanyway.\n\n>    When instantiating such a submodule, if \".git/modules/<name>/\" does\n>    not exist in the superproject, the submodule repository needs to be\n>    cloned there first. Then we only need to create \"<path>\" directory,\n>    point \".git/modules/<name>/\" in the superproject with \"<path>/.git\",\n>    and check out the working tree.\n>\n>    However, the current code is not structured that way. The codepath to\n>    deal with newly cloned submodules uses \"git clone --separate-git-dir\"\n>    and creates \"<path>\" and \"<path>/.git\". This can make the resulting\n>    submodule working tree at \"<path>\" different from the codepath for\n>    existing submodules. An example of such differences is that this\n>    codepath prepares \"<path>/.git\" with an absolute path, while the\n>    normal codepath uses a relative path.\n\nI had to read this three times before I understood it. There are some\nminor grammatical nits in it, but also the use of nearness and use of\n\"path\" and \"codepath\" to mean two unrelated things was misleading me.\nHere's my attempt to clean it up:\n\n    However, the current code is not structured that way. The code to\n    deal with newly cloned submodules is different from the code to\n    checkout a workdir for existing submodules.  The \"newly cloned\n    submodule\" code uses \"git clone --separate-git-dir\" to create\n    \"<path>\" and \"<path>/.git\". The \"existing submodules\" code\n    simply creates the \"<path>/.git\" internally, using a relative path.\n    This makes the resulting submodule working tree at \"<path>\" different\n    depending on which code is used.  An example of such differences\n    is that the \"newly cloned submodule\" code prepares \"<path>/.git\"\n    with an absolute path, while the \"existing submodules\" code\n    prepares the same file using a relative path.\n\n\n> When explained this way, the remedy is quite clear, and the change is more\n> forward-looking, isn't it?  If we later start doing more in the codepath\n> to deal with existing submodules, your patch may break without having\n> extra code to cover the \"newly cloned\" case, too.\n\n\n> I further wonder if we can get away without using separate-git-dir option\n> in this codepath, though. IOW using\n>\n>        git clone $quiet -bare ${reference:+\"$reference\"} \"$url\" \"$gitdir\"\n>\n> might be a better solution.\n\nYou may be right about this one.  I still think the addition of a\n--relative-path option to 'git-checkout --separate-work-dir' could be\nuseful and also easier to maintain/describe.\n\n> For example (this relates to the point I mumbled \"haven't thought this\n> through thoroughly yet\"), doesn't the newly cloned repository have\n> core.worktree that points at the working tree that records the <path>,\n> which would become meaningless when a commit in the superproject that\n> binds the submodule at different path <path2>?\n\nOoh, yes it does.  Maybe that should be fixed in this case too.\n\nBecause submodule cloning with a separate work-dir is a special case\nof git-files and work-dirs because we know that each is relative\n(subordinate) to the superproject path.  Therefore, I think in this\nspecial-case version of the \"separate work-dir\" scenario, we should\nuse super-project-relative paths for both cases.\n\nHow do we codify this so this functionality is reliably retained by\nfuture developers?  I think moving the code into someplace more\nexplicit would help, but I haven't looked too deeply at the code.\n\n>  git-submodule.sh |   21 ++++++++-------------\n>  1 files changed, 8 insertions(+), 13 deletions(-)\n>\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 3adab93..9a23e9d 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -156,21 +156,16 @@ module_clone()\n>                ;;\n>        esac\n>\n> -       if test -d \"$gitdir\"\n> +       if ! test -d \"$gitdir\"\n>        then\n> -               mkdir -p \"$path\"\n> -               echo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n> -               rm -f \"$gitdir/index\"\n> -       else\n> -               mkdir -p \"$gitdir_base\"\n> -               if test -n \"$reference\"\n> -               then\n> -                       git-clone $quiet \"$reference\" -n \"$url\" \"$path\" --separate-git-dir \"$gitdir\"\n> -               else\n> -                       git-clone $quiet -n \"$url\" \"$path\" --separate-git-dir \"$gitdir\"\n> -               fi ||\n> -               die \"$(eval_gettext \"Clone of '\\$url' into submodule path '\\$path' failed\")\"\n> +               git clone $quiet -n ${reference:+\"$reference\"} \\\n> +                       --separate-git-dir \"$gitdir\" \"$url\" \"$path\" ||\n> +               die \"$(eval_gettext \"Clone of '\\$url' for submodule '\\$name' failed\")\n>        fi\n> +\n> +       mkdir -p \"$path\"\n> +       echo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n> +       rm -f \"$gitdir/index\"\n>  }\n\nDoesn't this avoid creating core.worktree in the first place?  I'm ok\nwith that because I assume it's never used in the submodule scenario,\nbut I also suspect that assumption could be wrong.  Any concerns?\n\nPhil\n"},{"id":"181823","messageId":"4F007492.8010909@web.de","threadId":"29266","inReplyTo":"7vsjk3vw67.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-01-01T14:58:26Z","receivedAt":"2012-01-01T14:58:26Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 29.12.2011 23:40, schrieb Junio C Hamano:\n> Antony Male <antony.male@gmail.com> writes:\n> I further wonder if we can get away without using separate-git-dir option\n> in this codepath, though. IOW using\n> \n>         git clone $quiet -bare ${reference:+\"$reference\"} \"$url\" \"$gitdir\"\n> \n> might be a better solution.\n\nA quick test shows that using a bare repo won't fly because without the\ncore.worktree setting commands that operate on the work tree can't be\nrun anymore inside submodules (starting with the initial checkout). If\nwe could teach setup to take the directory where the gitfile was found\nas first guess for the git work tree it looks like we can make that\napproach work. I'll see if I can come up with something here ...\n\n> For example (this relates to the point I mumbled \"haven't thought this\n> through thoroughly yet\"), doesn't the newly cloned repository have\n> core.worktree that points at the working tree that records the <path>,\n> which would become meaningless when a commit in the superproject that\n> binds the submodule at different path <path2>?\n\nYes, and the core.worktree setting also contains an absolute path. So\nwe must either make that relative too and rewrite it on every \"git\nsubmodule add\" to record the possibly changed path there or make the\nbare clone work with a work tree (which sounds a bit strange ;-).\n\n>  git-submodule.sh |   21 ++++++++-------------\n>  1 files changed, 8 insertions(+), 13 deletions(-)\n> \n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 3adab93..9a23e9d 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -156,21 +156,16 @@ module_clone()\n>  \t\t;;\n>  \tesac\n>  \n> -\tif test -d \"$gitdir\"\n> +\tif ! test -d \"$gitdir\"\n>  \tthen\n> -\t\tmkdir -p \"$path\"\n> -\t\techo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n> -\t\trm -f \"$gitdir/index\"\n> -\telse\n> -\t\tmkdir -p \"$gitdir_base\"\n> -\t\tif test -n \"$reference\"\n> -\t\tthen\n> -\t\t\tgit-clone $quiet \"$reference\" -n \"$url\" \"$path\" --separate-git-dir \"$gitdir\"\n> -\t\telse\n> -\t\t\tgit-clone $quiet -n \"$url\" \"$path\" --separate-git-dir \"$gitdir\"\n> -\t\tfi ||\n> -\t\tdie \"$(eval_gettext \"Clone of '\\$url' into submodule path '\\$path' failed\")\"\n> +\t\tgit clone $quiet -n ${reference:+\"$reference\"} \\\n> +\t\t\t--separate-git-dir \"$gitdir\" \"$url\" \"$path\" ||\n> +\t\tdie \"$(eval_gettext \"Clone of '\\$url' for submodule '\\$name' failed\")\n>  \tfi\n> +\n> +\tmkdir -p \"$path\"\n> +\techo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n> +\trm -f \"$gitdir/index\"\n>  }\n>  \n>  #\n\nThat broke quite some tests for me (even though I really liked\nto get rid of that if ;-)\n\nHere is a patch that solves the first part of the absolute paths\nproblem (passes all tests; parts of the commit message shamelessly\ncopied from your proposal). Then another patch can tackle the\ncore.worktree config setting problem to make superprojects\nrelocatable gain.\n---------8<--------\nSubject: [PATCH] submodules: always use a relative path to gitdir\n\nRecent versions of \"git submodule\" maintain the submodule <name> at\n<path> in the superproject using a \"separate git-dir\" mechanism. The\nrepository data for the submodule is stored in \".git/modules/<name>/\"\ndirectory of the superproject, and its working tree is created at\n\"<path>/\" directory, with \"<path>/.git\" file pointing at the\n\".git/modules/<name>/\" directory.\n\nThis is so that we can check out an older version of the superproject\nthat does not yet have the submodule <name> anywhere without losing\n(and later having to re-clone) the submodule repository. Removing\n\"<path>\" won't lose \".git/modules/<name>\", and a different branch that\nhas the submodule at different location in the superproject, say\n\"<path2>\", can create \"<path2>/\" and \".git\" in it to point at the same\n\".git/modules/<name>\".\n\nWhen instantiating such a submodule, if \".git/modules/<name>/\" does\nnot exist in the superproject, the submodule repository needs to be\ncloned there first. Then we only need to create \"<path>\" directory,\npoint \".git/modules/<name>/\" in the superproject with \"<path>/.git\",\nand check out the working tree.\n\nHowever, the current code is not structured that way. The codepath to\ndeal with newly cloned submodules uses \"git clone --separate-git-dir\"\nand creates \"<path>\" and \"<path>/.git\". This can make the resulting\nsubmodule working tree at \"<path>\" different from the codepath for\nexisting submodules. An example of such differences is that this\ncodepath prepares \"<path>/.git\" with an absolute path, while the\nnormal codepath uses a relative path.\n\nFix the latter by always writing the relative path to the git directory\nin \"<path>/.git\". To make that work, the 'name' variable has to be set to\nthe value of the 'path' variable for newly added submodules.\n\nThis is only the first step to make superprojects movable again like they\nwere before the separate-git-dir approach was introduced. The second step\nmust be to either use a relative path in core.worktree too or to get rid\nof that setting by using a bare repo in \"./git/modules/<name>\".\n\nWhile at it also replace an if/else construct evaluating the presence\nof the 'reference' option with a single line of bash code.\n\nReported-by: Antony Male <antony.male@gmail.com>\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n git-submodule.sh |   12 +++++-------\n 1 files changed, 5 insertions(+), 7 deletions(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 3adab93..2a93c61 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -131,6 +131,7 @@ module_clone()\n \tgitdir=\n \tgitdir_base=\n \tname=$(module_name \"$path\" 2>/dev/null)\n+\ttest -n \"$name\" || name=\"$path\"\n \tbase_path=$(dirname \"$path\")\n\n \tgitdir=$(git rev-parse --git-dir)\n@@ -159,18 +160,15 @@ module_clone()\n \tif test -d \"$gitdir\"\n \tthen\n \t\tmkdir -p \"$path\"\n-\t\techo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n \t\trm -f \"$gitdir/index\"\n \telse\n \t\tmkdir -p \"$gitdir_base\"\n-\t\tif test -n \"$reference\"\n-\t\tthen\n-\t\t\tgit-clone $quiet \"$reference\" -n \"$url\" \"$path\" --separate-git-dir \"$gitdir\"\n-\t\telse\n-\t\t\tgit-clone $quiet -n \"$url\" \"$path\" --separate-git-dir \"$gitdir\"\n-\t\tfi ||\n+\t\tgit clone $quiet -n ${reference:+\"$reference\"} \\\n+\t\t\t--separate-git-dir \"$gitdir\" \"$url\" \"$path\" ||\n \t\tdie \"$(eval_gettext \"Clone of '\\$url' into submodule path '\\$path' failed\")\"\n \tfi\n+\n+\techo \"gitdir: $rel_gitdir\" >\"$path/.git\"\n }\n\n #\n-- \n1.7.8.2.303.g78a27\n"},{"id":"181869","messageId":"7vsjjwvdyl.fsf@alter.siamese.dyndns.org","threadId":"29266","inReplyTo":"4F007492.8010909@web.de","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-03T18:27:30Z","receivedAt":"2012-01-03T18:27:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann <Jens.Lehmann@web.de> writes:\n\n> Am 29.12.2011 23:40, schrieb Junio C Hamano:\n>> Antony Male <antony.male@gmail.com> writes:\n>> I further wonder if we can get away without using separate-git-dir option\n>> in this codepath, though. IOW using\n>> \n>>         git clone $quiet -bare ${reference:+\"$reference\"} \"$url\" \"$gitdir\"\n>> \n>> might be a better solution.\n>\n> A quick test shows that using a bare repo won't fly because without the\n> core.worktree setting commands that operate on the work tree can't be\n> run anymore inside submodules (starting with the initial checkout). \n\nProbably the right thing to do would be to restructure the flow as I\nsuggested, i.e.\n\n\tif we do not have it yet\n        then\n        \tgit clone --bare ...\n\tfi\n\t# now we have it, make sure they are correct\n\tgit config core.bare false\n\tgit config core.worktree $there\n        echo \"gitdir: $here\" >$there/.git\n\n> Yes, and the core.worktree setting also contains an absolute path. So\n> we must either make that relative too and rewrite it on every \"git\n> submodule add\" to record the possibly changed path there or make the\n> bare clone work with a work tree (which sounds a bit strange ;-).\n\nUpdate of core.worktree has to be done regardless of the absolute/relative\ndifferences anyway, no?\n\nThe first version of the superproject you trigger module_clone for\nsubmodule $name may happen to have it at $path, module_clone notices that\nyou do not have it, and the initial \"clone --separate-git-dir\" will set\nthe core.worktree to $superproject/$path.  Nobody will update it after\nthat, even when we check out different version of superproject that has\nthe same submodule $name at a different location in the superproject.\n"},{"id":"181871","messageId":"7vlipovd4n.fsf@alter.siamese.dyndns.org","threadId":"29266","inReplyTo":"CABURp0pdvf9Eo_pM2UCYUBANOJOGON6pQS-SXuCWQE=s2XNOfQ@mail.gmail.com","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-03T18:45:28Z","receivedAt":"2012-01-03T18:45:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phil Hord <phil.hord@gmail.com> writes:\n\n>> Again, who said that you are allowed to move the superproject directory in\n>> the first place? I would understand that it might be nicer if it could be\n>> moved but I haven't thought this through thoroughly yet---there may be\n>> other side effects from doing so, other than the relativeness of \"gitdir\".\n>\n> Previously it was accepted practice to clone a local repo with rsync.\n> This method continues to work well even with submodules before\n> git-files became the norm.  But now it breaks because of the absolute\n> paths.\n\nYou are utterly mistaken.\n\nThere are 47 million things you can do to your repository outside of the\ncontrol of git, and obviously we do not exhaustively enumerate everything\nthat ought to work (or not work). Anything that is not explicitly allowed\nin the documentation is, ehh, not allowed.\n\nMany such things may happen to work, either by accident or as a natural\nconsequence of the design. Some things needs adjustments after you do them\nwithout telling git. There is a difference between what is not allowed and\nwhat is explicitly forbidden.\n\nCopying with rsync (or cp for that matter) is one good example. Doing so\nwill cause the cached stat information in the index and the working tree\nfiles go out of sync, and diff-files will give you false differences after\nthat. You would adjust to that by running \"update-index --refresh\". So we\ndo not say \"you are allowed to cp and git will guarantee everything will\nwork as-is\", but it is not explicitly forbidden. As long as you make\nnecessary adjustments, you can keep using the copied repository.\n\n> So, who said you were NOT allowed to move the superproject directory\n> directory in the first place?\n\nSee above.\n\nAnd the extent of the design of\n\n    echo \"gitdir: $there\" >.git && git config core.worktree \"$(pwd)\"\n\nis to work with the locations of these two places as they are set up.\nMoving one or the other or both may or may not work without adjusting to\nwhat you did. If you \"mv $there $newlocation\" (the repository) behind\nGit's back, you may need to update .git to point at the new location of\nthe repository.  If you move your working tree woth \"mv\", you may need to\nupdate core.worktree to point at the new location of the working tree.\nAnd until you do so things may not work. That is why we do not explicitly\nsay \"you can move them to arbitrary places without telling git and things\nwill work\"---because that is not the case.\n\n> This doesn't explain why one path is absolute and one is relative.\n\nExactly. Because absolute/relative does not come into play as the scope of\nthe design did not include supporting \"moving\" one, the other, or both to\narbitrary places without telling git.\n"},{"id":"181883","messageId":"7vobuktv6p.fsf@alter.siamese.dyndns.org","threadId":"29266","inReplyTo":"7vlipovd4n.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-03T19:58:22Z","receivedAt":"2012-01-03T19:58:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> ...\n> And the extent of the design of\n>\n>     echo \"gitdir: $there\" >.git && git config core.worktree \"$(pwd)\"\n>\n> is to work with the locations of these two places as they are set up.\n> Moving one or the other or both may or may not work without adjusting to\n> what you did. If you \"mv $there $newlocation\" (the repository) behind\n> Git's back, you may need to update .git to point at the new location of\n> the repository.  If you move your working tree woth \"mv\", you may need to\n> update core.worktree to point at the new location of the working tree.\n> And until you do so things may not work. That is why we do not explicitly\n> say \"you can move them to arbitrary places without telling git and things\n> will work\"---because that is not the case.\n\nJust to avoid any misunderstanding, I still agree with the overall goal of\nthe original patch to allow moving the whole superproject tree, including\nits submodule repositories in its .git/modules/, and the working trees of\nitself and its submodules. It is a narrow special case with a very well\ndefined relative relationships between the working tree of submodules and\nthe repositories that control them, and having them point to each other\nwith relative paths will make any post-move adjustments unnecessary, unlike\nmore general unconstrained uses of the \"gitdir: $there\" mechanism.\n"},{"id":"181897","messageId":"4F037CBF.9010005@web.de","threadId":"29266","inReplyTo":"7vsjjwvdyl.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-01-03T22:10:07Z","receivedAt":"2012-01-03T22:10:07Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 03.01.2012 19:27, schrieb Junio C Hamano:\n> Jens Lehmann <Jens.Lehmann@web.de> writes:\n>> Am 29.12.2011 23:40, schrieb Junio C Hamano:\n>>> I further wonder if we can get away without using separate-git-dir option\n>>> in this codepath, though. IOW using\n>>>\n>>>         git clone $quiet -bare ${reference:+\"$reference\"} \"$url\" \"$gitdir\"\n>>>\n>>> might be a better solution.\n>>\n>> A quick test shows that using a bare repo won't fly because without the\n>> core.worktree setting commands that operate on the work tree can't be\n>> run anymore inside submodules (starting with the initial checkout). \n> \n> Probably the right thing to do would be to restructure the flow as I\n> suggested, i.e.\n> \n> \tif we do not have it yet\n>         then\n>         \tgit clone --bare ...\n> \tfi\n> \t# now we have it, make sure they are correct\n> \tgit config core.bare false\n\nAh, I forgot to set core.bare to false when trying this. But even then\na dozen tests fail, no matter if I set core.worktree or not. A cursory\nglance indicates problems with branches ... I'll have to dig deeper\nhere.\n\n> \tgit config core.worktree $there\n\nPlease see below.\n\n>         echo \"gitdir: $here\" >$there/.git\n> \n>> Yes, and the core.worktree setting also contains an absolute path. So\n>> we must either make that relative too and rewrite it on every \"git\n>> submodule add\" to record the possibly changed path there or make the\n>> bare clone work with a work tree (which sounds a bit strange ;-).\n> \n> Update of core.worktree has to be done regardless of the absolute/relative\n> differences anyway, no?\n\nNot if we would implement a \"if no worktree is set but we came here via\na gitfile, then take the directory the gitfile was found in as worktree\"\nheuristic. And that heuristic looks quite sane to me, as a gitfile can\nonly be found in a work tree, or am I missing something obvious here?\n"},{"id":"181898","messageId":"7vhb0csa6w.fsf@alter.siamese.dyndns.org","threadId":"29266","inReplyTo":"4F037CBF.9010005@web.de","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-03T22:17:11Z","receivedAt":"2012-01-03T22:17:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann <Jens.Lehmann@web.de> writes:\n\n> Not if we would implement a \"if no worktree is set but we came here via\n> a gitfile, then take the directory the gitfile was found in as worktree\"\n> heuristic. And that heuristic looks quite sane to me, as a gitfile can\n> only be found in a work tree, or am I missing something obvious here?\n\nLike it wouldn't work without changes to the core side?\n"},{"id":"182003","messageId":"4F0629C6.9010908@web.de","threadId":"29266","inReplyTo":"7vhb0csa6w.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-01-05T22:52:54Z","receivedAt":"2012-01-05T22:52:54Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 03.01.2012 23:17, schrieb Junio C Hamano:\n> Jens Lehmann <Jens.Lehmann@web.de> writes:\n> \n>> Not if we would implement a \"if no worktree is set but we came here via\n>> a gitfile, then take the directory the gitfile was found in as worktree\"\n>> heuristic. And that heuristic looks quite sane to me, as a gitfile can\n>> only be found in a work tree, or am I missing something obvious here?\n> \n> Like it wouldn't work without changes to the core side?\n\nI totally agree that when just talking about being able to move the\nsuperproject around that approach is more invasive than just adding\na relative core.worktree setting and is just not worth the hassle.\n\nBut I was also thinking about moving the submodule around inside the\nsuperproject. Until the gitfile was used that meant just mv'ing the\nsubmodule and changing the path in .gitmodules accordingly. Now you\nalso have to adjust the core.worktree setting and maybe also the\ngitfile content (if you move the submodule out of the directory level\nit lived in before).\n\nOne solution I can think of is to teach \"git mv\" about submodules and\nlet it do the necessary changes to .gitmodules (which seems to be a\ngood idea anyways), core.worktree and the gitfile. The manipulation of\ncore.worktree could be obsoleted by not using that setting but instead\nimplementing the heuristic I described above. And if the gitfile could\nbe taught somehow that a path in there is relative to the superprojects\nroot directory, then it would never have to be changed either, restoring\nthe behavior we had before introducing the gitfile.\n\nSo in the long run I suspect we might have to change core git anyways\nto make moving submodules easy for the user (surely \"git mv\" and maybe\nalso the setup and gitfile code). Does that make more sense?\n\nIf not I'm fine with just setting core.worktree to a relative path in\nthe git-submodule.sh script (like I did for the gitfile). And I'll look\ninto teaching \"git mv\" about submodules right after that.\n"},{"id":"182004","messageId":"7vlipllmfh.fsf@alter.siamese.dyndns.org","threadId":"29266","inReplyTo":"4F0629C6.9010908@web.de","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-06T00:11:30Z","receivedAt":"2012-01-06T00:11:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann <Jens.Lehmann@web.de> writes:\n\n> So in the long run I suspect we might have to change core git anyways\n> to make moving submodules easy for the user (surely \"git mv\" and maybe\n> also the setup and gitfile code). Does that make more sense?\n\nIf you need to change \"git mv\" anyway to help moving submodule checkout,\nthen how gitfile points into .git/modules/ hierarchy of the superproject\nbecomes an implementation detail the end users should not have to care\nabout.\n\nWhat does \"if we reached thru a gitfile, then the working tree is where\nyou found that gitfile\" really solve? The way you found that gitfile is by\ntraversing the directory hierarchy upwards from a subdirectory of a\nworking tree of a submodule, and you already know where the top of that\nworking tree is, no?\n\nAnd the heuristics would not work if somebody goes into the $GIT_DIR/ that\ngoverns the submodule as going upwards from there will not hit gitfile, so\nwe would need help from core.worktree anyway. A non-submodule setting that\nuses gitfile would need to worry about core.worktree, too, so I'd rather\navoid loading more heuristics to gitfile handling unless there is a clear\nadvantage for doing so, which I am not really seeing here.\n\nThat is not really a \"If not\" below (i.e. I am not saying it is _not_ OK.\nI am saying I don't know what the advantage of that approach is), but ...\n\n> If not I'm fine with just setting core.worktree to a relative path in\n> the git-submodule.sh script (like I did for the gitfile). And I'll look\n> into teaching \"git mv\" about submodules right after that.\n\n... teaching \"git mv\" may be a good move, I would think. I do think keeping\ncore.worktree pointing at the right directory is necessary, but I do not\nsee much point in making it a relative path, though.\n"},{"id":"182022","messageId":"CABURp0rFOFfX7eu-v6ZK07iTfXwhOne60d70GkCdOvx0k8BZkQ@mail.gmail.com","threadId":"29266","inReplyTo":"7vlipllmfh.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Phil Hord","fromEmail":"phil.hord@gmail.com","sentAt":"2012-01-06T14:26:53Z","receivedAt":"2012-01-06T14:26:53Z","isPatch":true,"sender":{"key":"phil.hord@gmail.com","avatar":"https://avatars.githubusercontent.com/u/123908?v=4"},"body":"On Thu, Jan 5, 2012 at 7:11 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Jens Lehmann <Jens.Lehmann@web.de> writes:\n>> If not I'm fine with just setting core.worktree to a relative path in\n>> the git-submodule.sh script (like I did for the gitfile). And I'll look\n>> into teaching \"git mv\" about submodules right after that.\n>\n> ... teaching \"git mv\" may be a good move, I would think. I do think keeping\n> core.worktree pointing at the right directory is necessary, but I do not\n> see much point in making it a relative path, though.\n\nI do, in the case of submodules, as already discussed.\n\nDo you see any _problem_ with making core.worktree a relative\ndirectory in the specific case of git submodules?\n\nPhil\n"},{"id":"182026","messageId":"CACsJy8Agw6aTu=odeJXEbYuWQnE228w24_baP8u2eiX2-BEpeA@mail.gmail.com","threadId":"29266","inReplyTo":"CABURp0rFOFfX7eu-v6ZK07iTfXwhOne60d70GkCdOvx0k8BZkQ@mail.gmail.com","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-01-06T15:07:09Z","receivedAt":"2012-01-06T15:07:09Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Jan 6, 2012 at 9:26 PM, Phil Hord <phil.hord@gmail.com> wrote:\n> Do you see any _problem_ with making core.worktree a relative\n> directory in the specific case of git submodules?\n\nNot a problem per se, but you should look at the comment at the top of\nt1510 to see where it is relative to. Two interesting rules:\n\n2. .git file is relative to parent directory. .git file is basically\n   symlink in disguise. The directory where .git file points to will\n   become new git_dir.\n\n3. core.worktree is relative to git_dir.\n-- \nDuy\n"},{"id":"182032","messageId":"7vboqgll27.fsf@alter.siamese.dyndns.org","threadId":"29266","inReplyTo":"CABURp0rFOFfX7eu-v6ZK07iTfXwhOne60d70GkCdOvx0k8BZkQ@mail.gmail.com","subject":"Re: [PATCH] Submodules always use a relative path to gitdir","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-01-06T18:53:20Z","receivedAt":"2012-01-06T18:53:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phil Hord <phil.hord@gmail.com> writes:\n\n> On Thu, Jan 5, 2012 at 7:11 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Jens Lehmann <Jens.Lehmann@web.de> writes:\n>>> If not I'm fine with just setting core.worktree to a relative path in\n>>> the git-submodule.sh script (like I did for the gitfile). And I'll look\n>>> into teaching \"git mv\" about submodules right after that.\n>>\n>> ... teaching \"git mv\" may be a good move, I would think. I do think keeping\n>> core.worktree pointing at the right directory is necessary, but I do not\n>> see much point in making it a relative path, though.\n>\n> I do, in the case of submodules, as already discussed.\n\nOf course you are right.\n"}]}