{"thread":{"id":"33499","subject":"[RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","startedAt":"2013-04-13T19:23:27Z","lastAt":"2013-04-17T23:01:29Z","messageCount":41,"participants":["Ramkumar Ramachandra","Junio C Hamano","Duy Nguyen","Marc Branchaud","Jeff King","Jonathan Nieder"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"214139","messageId":"1365881007-25731-1-git-send-email-artagnon@gmail.com","threadId":"33499","inReplyTo":null,"subject":"[RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-13T19:23:27Z","receivedAt":"2013-04-13T19:23:27Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"This configuration variable comes into effect when 'git clone' is\ninvoked inside an existing git repository's worktree. When set,\ninstead of cloning the given repository as-is, it relocates the gitdir\nof the repository to the path specified by this variable. This\nsetting is especially useful when working with submodules.\n\nSigned-off-by: Ramkumar Ramachandra \n---\n Okay, so this is part of my evil plan to make 'git add' DTRT wrt\n submodules, and deprecate 'git submodule add' (I have some code\n written down, but this is a prerequisite: I don't like the\n .git/modules nonsense).\n\n Unfortunately, this patch is in pathetic shape and is an RFC for\n three reasons:\n\n 1. I've used setup_git_directory_gently() at the start of\n builtin/clone.c to check if I'm inside a git directory. This\n breaks a lot of existing tests (I'm yet to understand these\n failures fully).\n\n 2. setup_git_directory_gently() has the side-effect of changing the\n current directory and calling set_git_work_tree(), both of which\n must be done away with if we want the rest of clone.c to work.\n I've hacked around the issue in a very dirty manner. What is the\n solution to this?\n\n 3. I don't know how to test the case \"clone.submoduleGitDir has no\n effect outside a git repository\", because our entire test\n environment is a git repository. Even if I remove the .git\n directory, we're still inside the soure tree's git repository.\n What do I do about this? Even if we decide that this patch is\n fundamentally unworkable, we should try to fix this issue so that\n we can verify that a plain 'git clone' works outside a git\n repository.\n\n Thanks for reading. And I'm truly sorry for making you read through\n such ugly code.\n\n Documentation/config.txt | 11 +++++++++++\n builtin/clone.c | 33 ++++++++++++++++++++++++++++++++-\n environment.c | 11 -----------\n t/t5702-clone-options.sh | 41 +++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 84 insertions(+), 12 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 3d750e0..aac26c3 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -798,6 +798,17 @@ clean.requireForce::\n \tA boolean to make git-clean do nothing unless given -f\n \tor -n. Defaults to true.\n \n+clone.submoduleGitDir::\n+\tAn absolute path on the filesystem where gitdirs of submodules\n+\tshould be stored away safely. When not set, a 'git clone'\n+\texecuted inside a git repository will do exactly what it does\n+\toutside a git repository. When set, a 'git clone' executed\n+\tinside a git repository will create the worktree in place of\n+\tthe full repository, and put the object store in a\n+\tsubdirectory of clone.submoduleGitDir, choosing the name to be\n+\tthe \"humanish\" part of the source repository (`repo.git` for\n+\t`/path/to/repo.git` and `foo.git` for `host.xz:foo/.git`).\n+\n color.branch::\n \tA boolean to enable/disable color in the output of\n \tlinkgit:git-branch[1]. May be set to `always`,\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex f9c380e..4a845a4 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -44,6 +44,7 @@ static char *option_template, *option_depth;\n static char *option_origin = NULL;\n static char *option_branch = NULL;\n static const char *real_git_dir;\n+static const char *submodule_gitdir;\n static char *option_upload_pack = \"git-upload-pack\";\n static int option_verbosity;\n static int option_progress = -1;\n@@ -707,12 +708,22 @@ static void write_refspec_config(const char* src_ref_prefix,\n \tstrbuf_release(&value);\n }\n \n+static int git_clone_config(const char *var, const char *value, void *cb)\n+{\n+\tif (!strcmp(var, \"clone.submodulegitdir\")) {\n+\t\tgit_config_string(&submodule_gitdir, var, value);\n+\t\treturn 0;\n+\t}\n+\treturn git_default_config(var, value, cb);\n+}\n+\n int cmd_clone(int argc, const char **argv, const char *prefix)\n {\n \tint is_bundle = 0, is_local;\n \tstruct stat buf;\n \tconst char *repo_name, *repo, *work_tree, *git_dir;\n-\tchar *path, *dir;\n+\tchar *path, *dir, *dest_git_dir;\n+\tchar cwd[PATH_MAX];\n \tint dest_exists;\n \tconst struct ref *refs, *remote_head;\n \tconst struct ref *remote_head_points_at;\n@@ -725,6 +736,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tconst char *src_ref_prefix = \"refs/heads/\";\n \tstruct remote *remote;\n \tint err = 0, complete_refs_before_fetch = 1;\n+\tint nongit = 1;\n \n \tstruct refspec *refspec;\n \tconst char *fetch_pattern;\n@@ -732,6 +744,14 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tjunk_pid = getpid();\n \n \tpacket_trace_identity(\"clone\");\n+\n+\t/* setup_git_directory_gently without changing directories */\n+\tgetcwd(cwd, sizeof(cwd) - 1);\n+\tsetup_git_directory_gently(&nongit);\n+\tchdir(cwd);\n+\n+\tgit_config(git_clone_config, NULL);\n+\n \targc = parse_options(argc, argv, prefix, builtin_clone_options,\n \t\t\t builtin_clone_usage, 0);\n \n@@ -785,6 +805,17 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\tdie(_(\"destination path '%s' already exists and is not \"\n \t\t\t\"an empty directory.\"), dir);\n \n+\tif (!nongit && submodule_gitdir) {\n+\t\tchar *user_path = expand_user_path(submodule_gitdir);\n+\t\tif (!user_path)\n+\t\t\tdie(_(\"Unable to expand path in clone.submoduleGitDir: %s\"), submodule_gitdir);\n+\t\tdest_git_dir = mkpathdup(\"%s/%s.git\", user_path, dir);\n+\t\tif (!stat(dest_git_dir, &buf) && !is_empty_dir(dest_git_dir))\n+\t\t\tdie(_(\"destination path '%s' already exists and is not \"\n+\t\t\t\t\t\"an empty directory.\"), dest_git_dir);\n+\t\treal_git_dir = dest_git_dir;\n+\t}\n+\n \tstrbuf_addf(&reflog_msg, \"clone: from %s\", repo);\n \n \tif (option_bare)\ndiff --git a/environment.c b/environment.c\nindex e2e75c1..9dce4c7 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -182,8 +182,6 @@ const char *strip_namespace(const char *namespaced_ref)\n \treturn namespaced_ref + namespace_len;\n }\n \n-static int git_work_tree_initialized;\n-\n /*\n * Note. This works only before you used a work tree. This was added\n * primarily to support git-clone to work in a new repository it just\n@@ -191,15 +189,6 @@ static int git_work_tree_initialized;\n */\n void set_git_work_tree(const char *new_work_tree)\n {\n-\tif (git_work_tree_initialized) {\n-\t\tnew_work_tree = real_path(new_work_tree);\n-\t\tif (strcmp(new_work_tree, work_tree))\n-\t\t\tdie(\"internal error: work tree has already been set\\n\"\n-\t\t\t \"Current worktree: %s\\nNew worktree: %s\",\n-\t\t\t work_tree, new_work_tree);\n-\t\treturn;\n-\t}\n-\tgit_work_tree_initialized = 1;\n \twork_tree = xstrdup(real_path(new_work_tree));\n }\n \ndiff --git a/t/t5702-clone-options.sh b/t/t5702-clone-options.sh\nindex 02cb024..9b845d8 100755\n--- a/t/t5702-clone-options.sh\n+++ b/t/t5702-clone-options.sh\n@@ -33,4 +33,45 @@ test_expect_success 'redirected clone -v' '\n \n '\n \n+test_expect_success 'clone.submoduleGitDir takes effect in a git repository' '\n+\tcd ~ &&\n+\trm -rf bare newrepo superproject &&\n+\tmkdir bare &&\n+\tbare_path=\"$(pwd)/bare\" &&\n+\tgit init newrepo &&\n+\t(\n+\t\tcd newrepo &&\n+\t\techo quux >foo &&\n+\t\tgit add foo &&\n+\t\tgit commit -m \"Add foo\"\n+\t) &&\n+\tgit init superproject &&\n+\tcd superproject &&\n+\ttest_config clone.submoduleGitDir \"$bare_path\" &&\n+\tgit clone ../newrepo &&\n+\ttest_path_is_file newrepo/.git &&\n+\tcd ../bare/newrepo.git &&\n+\tgit rev-parse --is-bare-repository\n+'\n+\n+test_expect_success 'clone.submoduleGitDir path get tilde-expansion' '\n+\tcd ~ &&\n+\trm -rf bare newrepo superproject &&\n+\tmkdir bare &&\n+\tgit init newrepo &&\n+\t(\n+\t\tcd newrepo &&\n+\t\techo quux >foo &&\n+\t\tgit add foo &&\n+\t\tgit commit -m \"Add foo\"\n+\t) &&\n+\tgit init superproject &&\n+\tcd superproject &&\n+\ttest_config clone.submoduleGitDir ~/bare &&\n+\tgit clone ../newrepo &&\n+\ttest_path_is_file newrepo/.git &&\n+\tcd ../bare/newrepo.git &&\n+\tgit rev-parse --is-bare-repository\n+'\n+\n test_done\n-- \n1.8.2.1.389.gcaa7d79.dirty\n"},{"id":"214246","messageId":"7vy5ck4m6b.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"1365881007-25731-1-git-send-email-artagnon@gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-15T01:28:28Z","receivedAt":"2013-04-15T01:28:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramkumar Ramachandra writes:\n\n> This configuration variable comes into effect when 'git clone' is\n> invoked inside an existing git repository's worktree. When set,\n> instead of cloning the given repository as-is, it relocates the gitdir\n> of the repository to the path specified by this variable.\n\nRelocate to where in the superproject's gitdir? Presumably you can\ndo this more than once in a given superproject, so there needs to be\na key per such a clone, no? I am guessing that you would follow the\nusual \"when adding a submodule without name, use its path as the\ninitial name\" convention, but then I would suggest it to be spelled\nout (and if you are doing it differently, that choice needs to be\nspelled out and defended).\n\n> Okay, so this is part of my evil plan to make 'git add' DTRT wrt\n> submodules,...\n\nIf the envisioned use of this is to use it as a building block of\nsomething else that is user-facing (e.g. the user says \"git add\",\nand before the command finishes, somewhere we internally run \"git\nclone\"), then would it be possible that you are better off running\nthat clone with --separate-git-dir and let it make the gitfile for\nyou?\n\nAny new configuration variable brings its own problem by forcing\nexisting users to countermand it explicitly from the command line.\nIf the --separate-git-dir would not work for your application, you\nneed a new feature and you can achieve the same by adding a new\ncommand line option (say, --submodule-git-dir), that would be more\npreferrable.\n"},{"id":"214258","messageId":"7v61zo4igg.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"7vy5ck4m6b.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-15T02:48:47Z","receivedAt":"2013-04-15T02:48:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> If the envisioned use of this is to use it as a building block of\n> something else that is user-facing (e.g. the user says \"git add\",\n> and before the command finishes, somewhere we internally run \"git\n> clone\"), then would it be possible that you are better off running\n> that clone with --separate-git-dir and let it make the gitfile for\n> you?\n\nAs you may have already guessed, in principle I am all for teaching\n\"git add\" not just to add a submodule itself (which we already do)\nbut also to record information about the submodule, without having\nto delegate it to \"git submodule\". \"git submodule add\" was meant as\nan interim measure until we figure out what kind of metainformation\nis necessary, and doing things in \"git add\" has always been a longer\nterm goal.\n\nThere are two ways to \"add\" a submodule to a superproject. You may\nbring an existing project with \"git clone\" inside the working tree\nof a superproject (which I am guessing is the use case that inspired\nthis patch), but it will leave the git dir of the submodule embedded\nin its working tree. \n\nYou could continue \"git clone\" and then teach \"git add\" (or \"git\nsubmodule add\") to relocate the embedded git directory from the\nsubmodule working tree, you could \"git clone\" with separate-git-dir\nfrom the beginning, or you could extend \"git add\", perhaps\n\n git add --url=git://up.stre.am/repository [--name=name] sub/mod/ule\n\nand do that \"git clone --separate-git-dir\" internally (which will\nmean that the end user will not run \"git clone\").\n\nAnother way ti \"add\" a submodule is to run \"git init\" to originate a\nnew project inside the working tree of a superproject. The resulting\nsubmodule working tree will have the embedded git dir, and again\n\"git add\" (or \"git submodule add\") could notice and relocate it, but\nif the extended \"git add\" wants to help that use case as well, I\nthink it is the matter of running \"git init --separate-git-dir\",\njust like \"add by cloning from elsewhere\" can do the same with the\nflag to \"git clone\".\n"},{"id":"214271","messageId":"CALkWK0=9OgRtrwnCpVOpmjHb0j38M=VQrzfFh4H=sV=dVvcV8w@mail.gmail.com","threadId":"33499","inReplyTo":"7vy5ck4m6b.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-15T07:59:45Z","receivedAt":"2013-04-15T07:59:45Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Junio C Hamano wrote:\n> Relocate to where in the superproject's gitdir? Presumably you can\n> do this more than once in a given superproject, so there needs to be\n> a key per such a clone, no? I am guessing that you would follow the\n> usual \"when adding a submodule without name, use its path as the\n> initial name\" convention, but then I would suggest it to be spelled\n> out (and if you are doing it differently, that choice needs to be\n> spelled out and defended).\n\nI probably wasn't clear enough in the commit message, but this is what\nhappens when I set clone.submoduleGitDir to ~/bare: a git clone\ngh:artagnon/clayoven inside the superproject's worktree will make\n~/bare/clayoven.git and ./clayoven corresponding to the GITDIR and the\nworktree of the newly cloned repository. If there are conflicts, it\nwill complain as usual saying that the destination path %s already\nexists, in which case the user has to choose a name for the GITDIR\n(not yet implemented) and/or the worktree path (as the final\ncommand-line argument to git clone).\n"},{"id":"214272","messageId":"CALkWK0m-X7K=WXFiiMkqZBBTBB9KC6myeN+s_xYLXfadGJCdZQ@mail.gmail.com","threadId":"33499","inReplyTo":"7v61zo4igg.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-15T08:08:32Z","receivedAt":"2013-04-15T08:08:32Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"> You could continue \"git clone\" and then teach \"git add\" (or \"git\n> submodule add\") to relocate the embedded git directory from the\n> submodule working tree, you could \"git clone\" with separate-git-dir\n> from the beginning, or you could extend \"git add\", perhaps\n>\n> git add --url=git://up.stre.am/repository [--name=name] sub/mod/ule\n>\n> and do that \"git clone --separate-git-dir\" internally (which will\n> mean that the end user will not run \"git clone\").\n\nI specifically did not go down this route, because I think it is\ngross. Where does moving a GITDIR fit into what git add's normal job\n(index manipulation) is? Tools should do one specific thing, and do\nit well: not a mixed bag of unrelated things. git clone, on the other\nhand, was always intended to have a way to point to a location for\nGITDIR and the worktree: isn't this feature very close to\n--separate-git-dir already? It is, therefore, git clone's job to\nrelocate the GITDIR. My future plan is to deny git add'ing anything\nbut a worktree-with-a-gitfile.\n"},{"id":"214274","messageId":"CALkWK0mvtRhFc0_4883ATNaYpb+kDwpV9VxeAoqJy5HxNQ6vgg@mail.gmail.com","threadId":"33499","inReplyTo":"7vy5ck4m6b.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-15T08:19:43Z","receivedAt":"2013-04-15T08:19:43Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Junio C Hamano wrote:\n> Any new configuration variable brings its own problem by forcing\n> existing users to countermand it explicitly from the command line.\n> If the --separate-git-dir would not work for your application, you\n> need a new feature and you can achieve the same by adding a new\n> command line option (say, --submodule-git-dir), that would be more\n> preferrable.\n\nI'm getting a little tired of your first instinct to oppose every new\naddition to git. (Ofcourse I understand your attitude as the\nmaintainer, but still)\n\nIt doesn't make sense as a command-line option, because it is \"magic\"\nthat kicks in only when git clone is executed inside an existing git\nworktree. The point is that the user doesn't have to remember\nanything special: a normal git clone already does the right thing\noutside a git worktree; my proposal is to make it do the right thing\ninside a git worktree as well. Although I'm not against allowing a\nuser to create a \"full clone\" inside a git repository by overriding\nclone.submoduleGitDir via a command-line option, I really cannot see\nwhy this would be anything but rare. Why would a user *want* a full\nclone inside a git worktree?\n\nAlso, naming it --submodule-git-dir can cause a lot of confusion:\n--separate-git-dir names a specific directory to put the GITDIR in,\nwhile --submodule-git-dir names a directory inside which to create\nother named directories to put GITDIRs in. Ofcourse\nclone.submoduleGitDir is a bad name too: any suggestions?\n"},{"id":"214280","messageId":"CACsJy8Cx0QA_epns2WNWjBBSG6zpXVaTebybiTRuVt+OARupAg@mail.gmail.com","threadId":"33499","inReplyTo":"CALkWK0mvtRhFc0_4883ATNaYpb+kDwpV9VxeAoqJy5HxNQ6vgg@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-04-15T09:25:08Z","receivedAt":"2013-04-15T09:25:08Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Apr 15, 2013 at 6:19 PM, Ramkumar Ramachandra\n wrote:\n> It doesn't make sense as a command-line option, because it is \"magic\"\n> that kicks in only when git clone is executed inside an existing git\n> worktree. The point is that the user doesn't have to remember\n> anything special: a normal git clone already does the right thing\n> outside a git worktree; my proposal is to make it do the right thing\n> inside a git worktree as well. Although I'm not against allowing a\n> user to create a \"full clone\" inside a git repository by overriding\n> clone.submoduleGitDir via a command-line option, I really cannot see\n> why this would be anything but rare. Why would a user *want* a full\n> clone inside a git worktree?\n\nIf a user is inside .git, I believe setup_git_directory() will also\nfind correct gitdir. In that case, we do not want magic (i.e. only do\nyour magic when you are inside worktree). Still I'd rather see no\nmagic (i.e. command line option) first. Let people try it out for a\nwhile. If people like it and find it inconvenient, magic can come\nlater. I suspect you might want more magic in other places. Maybe if\nyou hold it back until you see full picture, you'll only need a few\nnew config keys (instead of one per separate magic).\n\n> Unfortunately, this patch is in pathetic shape and is an RFC for\n> three reasons:\n>\n> 1. I've used setup_git_directory_gently() at the start of\n> builtin/clone.c to check if I'm inside a git directory. This\n> breaks a lot of existing tests (I'm yet to understand these\n> failures fully).\n>\n> 2. setup_git_directory_gently() has the side-effect of changing the\n> current directory and calling set_git_work_tree(), both of which\n> must be done away with if we want the rest of clone.c to work.\n> I've hacked around the issue in a very dirty manner. What is the\n> solution to this?\n\nJust do what scripts do: spawn a process to run rev-parse so that it\ndoes not mess up the main process. You might be able to introduce\n\"dry-run\" mode for setup_git_directory(), but that won't be easy.\n--\nDuy\n"},{"id":"214285","messageId":"7vfvys160z.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"CALkWK0mvtRhFc0_4883ATNaYpb+kDwpV9VxeAoqJy5HxNQ6vgg@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-15T09:45:32Z","receivedAt":"2013-04-15T09:45:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramkumar Ramachandra writes:\n\n> Junio C Hamano wrote:\n>> Any new configuration variable brings its own problem by forcing\n>> existing users to countermand it explicitly from the command line.\n>> If the --separate-git-dir would not work for your application, you\n>> need a new feature and you can achieve the same by adding a new\n>> command line option (say, --submodule-git-dir), that would be more\n>> preferrable.\n>\n> I'm getting a little tired of your first instinct to oppose every new\n> addition to git. (Ofcourse I understand your attitude as the\n> maintainer, but still)\n\nIt was purly about \"do not add anything that makes no sense.\" and \nnot about \"oppose all new addition.\"\n\nWhen you add a submodule with the current system, bypassing \"git\nsubmodule add\", you can either\n\n (1) \"git clone $URL here\" and then \"git add here\"; or\n (2) \"git init here\" and then \"git add here\".\n\nBecause you didn't say what you are aiming for in the grander\npicture, I thought you were \"making the UI simpler\" by making it\nunnecessary for the users to say \"git clone\" himself as a separate\nstep before doing \"git add\". In such a world, \"add\" would internally\nrun \"clone\". If that were the case (I now know it is not), then the\nconfiguration _is_ unnecessary, and it is perfectly valid to\nquestion why you thought it is needed.\n\nIf your plan is instead to keep \"git clone\" followed by \"git add\" as\nthe pattern for use case (1), teaching \"clone\" to automatically use\nthe --separate-git-dir mechanism to point at the right place inside\nthe $GIT_DIR of the superproject does make sense to help the use\ncase.\n\nBut if that is the direction you are aiming for, would it be\npossible that the same configuration variable can and should cover\nthe use case (2) as well? After all, between \"git init here\" and\n\"git add here\", the user may say (cd here && git pull $URL) and the\nexpected end result would be the same as (1), no?\n\nI do not recall the details of the codepaths involved offhand, but\nwhen you \"git clone $URL [here]\", after running \"mkdir here\", it\nwould create a $GIT_DIR for the \"here\" repository in \"here/.git\"\n(and with --separate-git-dir, it would create it elsewhere and drop\ngitfile at \"here/.git\"). When you \"git init here\", after running\n\"mkdir here\", the same thing happens.\n\nHow common are these two implementations?\n\nIf \"clone\" just calls init_db(), I would imagine that it might be\ntrivial to cover both cases by telling init_db() to pay attention to\nthe configuration, without doing much in the \"clone\" itself.\n"},{"id":"214287","messageId":"CALkWK0nPhXhv64t7tDwLudFgi7NnanVsnYQPqWhYiAp-y9Z90w@mail.gmail.com","threadId":"33499","inReplyTo":"CACsJy8Cx0QA_epns2WNWjBBSG6zpXVaTebybiTRuVt+OARupAg@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-15T09:47:52Z","receivedAt":"2013-04-15T09:47:52Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Duy Nguyen wrote:\n> If a user is inside .git, I believe setup_git_directory() will also\n> find correct gitdir. In that case, we do not want magic (i.e. only do\n> your magic when you are inside worktree). Still I'd rather see no\n> magic (i.e. command line option) first. Let people try it out for a\n> while. If people like it and find it inconvenient, magic can come\n> later. I suspect you might want more magic in other places. Maybe if\n> you hold it back until you see full picture, you'll only need a few\n> new config keys (instead of one per separate magic).\n\nGood suggestion. I'll make it a command-line option for now.\n\n> Just do what scripts do: spawn a process to run rev-parse so that it\n> does not mess up the main process. You might be able to introduce\n> \"dry-run\" mode for setup_git_directory(), but that won't be easy.\n\nOkay, thanks.\n"},{"id":"214292","messageId":"7v61zo14p8.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"CALkWK0m-X7K=WXFiiMkqZBBTBB9KC6myeN+s_xYLXfadGJCdZQ@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-15T10:14:11Z","receivedAt":"2013-04-15T10:14:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramkumar Ramachandra writes:\n\n> I specifically did not go down this route, because I think it is\n> gross. Where does moving a GITDIR fit into what git add's normal job\n> (index manipulation) is? Tools should do one specific thing, and do\n> it well: not a mixed bag of unrelated things.\n\nI see you are trying to repeat the UNIX mantra, but I do not think\nit is working.\n\nWhen we discuss \"git add\", the \"one unit of work\" is at much higher\nlevel than that of \"git update-index\". \"git add dir/\" has to do a\nlot more than \"git add file\", and \"git add symlink\" has to do quite\na different thing from \"git add file\", but to the end user, all of\nthem are about doing everything necessary to add what the user named\nto the index. \"git add submodule/\" that does whatever necessary to\nadd the submodule to the index is still doing one thing well inside\nthe same framework, and that may include moving the $GIT_DIR and\nturning it into a gitfile.\n\nNot that I am saying I prefer \"add --url=xxx\". Quite the opposite.\nI very much prefer the \"clone and then add, but clone drops the\nrepository at the right place from the beginning\" approach than \"add\nthat knows about URL only for submodules\", which is an ugly kludge.\n\nIf the user creates here/.git without gitlink with whatever means,\nit is \"git add here\"'s job, if it wants to make it a submodule and\nif it wants to make it possible to later check out another branch\nthat does not have the submodule, to stash away the repository and\nturn it into gitfile, if it is part of what is needed to add a\nsubmodule.\n\nOf course, we could start from teaching \"submodule add\" to do so,\nand then internally redirect \"git add subm\" to \"git submodule add\",\nbut that is a minor implementation detail that does not affect the\nend user experience.\n"},{"id":"214294","messageId":"CALkWK0n_vOwQkJ6BMXdE06cUJhiJdWNYq3vPDDOZHvji6FyKow@mail.gmail.com","threadId":"33499","inReplyTo":"7v61zo14p8.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-15T11:35:05Z","receivedAt":"2013-04-15T11:35:05Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Junio C Hamano wrote:\n> When we discuss \"git add\", the \"one unit of work\" is at much higher\n> level than that of \"git update-index\". \"git add dir/\" has to do a\n> lot more than \"git add file\", and \"git add symlink\" has to do quite\n> a different thing from \"git add file\", but to the end user, all of\n> them are about doing everything necessary to add what the user named\n> to the index. \"git add submodule/\" that does whatever necessary to\n> add the submodule to the index is still doing one thing well inside\n> the same framework, and that may include moving the $GIT_DIR and\n> turning it into a gitfile.\n\nYou're looking at it from an end-user point of view, while I'm looking\nat it from the implementation point of view. Here's a coarse\nsimplification of what git add does:\n\n1. Lock the index file, and grab a FILE handle to read/ write the file.\n\n2. Update active_cache. Depending on the pathspec, we might be adding\none entry or multiple entries, with different modes to the index.\nNowhere did I say that it should add exactly one entry to active_cache\nwith a predefined mode. Sure, tools that operate a lower layer of\nabstraction like update-index can be more picky about this.\n\n3. Write tree and blob objects to the database corresponding to the\nworktree entries. Files and symbolic links get blob objects, while\ndirectories get tree objects.\n\n4. Write active_cache to FILE handler we grabbed in step 1, and\nrelease the lock.\n\nWhat it does not do:\n\n1. Move random files/ directory around in the worktree.\n\n2. Mangle existing files in the worktree. (Although I know that the\n.gitmodules-mangling is coming soon, I'm not exactly elated with it\n[1])\n\n3. Write commit or tag objects to the database.\n\n4. Update random refs.\n\n5. Make coffee for the user to applaud him on the successful add.\n\nIn my opinion, with some minor exceptions, all git tools follow these\nprinciples. Briefly, branch is a refs/heads/* helper, checkout is a\nHEAD + worktree helper, fetch is a receive-pack + refs/remotes/*\nhelper, and reset is a bit of a swiss army knife that operates on HEAD\n+ index + worktree.\n\nIn general, I like git because commands don't create unnatural or\nheavy abstractions on top of these concepts. With some minor\nexceptions, all the commands are easy to understand and consistent.\n\n[1]: This is what led to my OBJ_LINK proposal.\n\n> Not that I am saying I prefer \"add --url=xxx\". Quite the opposite.\n> I very much prefer the \"clone and then add, but clone drops the\n> repository at the right place from the beginning\" approach than \"add\n> that knows about URL only for submodules\", which is an ugly kludge.\n\nI don't know why you brought up the alternative in the first place.\nWe both agree that it is git clone's job, although your reason is more\nsuperficial and mine's tied to the implementation.\n\n> If the user creates here/.git without gitlink with whatever means,\n> it is \"git add here\"'s job, if it wants to make it a submodule and\n> if it wants to make it possible to later check out another branch\n> that does not have the submodule, to stash away the repository and\n> turn it into gitfile, if it is part of what is needed to add a\n> submodule.\n\nI disagree. I think we should get a first-class tool to attach/\ndetach worktrees from a GITDIR. It can incoporate the logic from\ncontrib/workdir/git-new-workdir to optionally create a worktree with\nan independent index, HEAD, and logs/HEAD.\n\n> Of course, we could start from teaching \"submodule add\" to do so,\n> and then internally redirect \"git add subm\" to \"git submodule add\",\n> but that is a minor implementation detail that does not affect the\n> end user experience.\n\nYuck. Don't you care about the implementation, as long as it fixes\nthe end-user's problem?\n"},{"id":"214296","messageId":"CALkWK0nOW5HFrnsGwmFtmtkUc_SzwiQFw9dY6Pa2y5yRJ_OrCw@mail.gmail.com","threadId":"33499","inReplyTo":"7vfvys160z.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-15T11:48:38Z","receivedAt":"2013-04-15T11:48:38Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Junio C Hamano wrote:\n> When you add a submodule with the current system, bypassing \"git\n> submodule add\", you can either\n>\n> (1) \"git clone $URL here\" and then \"git add here\"; or\n> (2) \"git init here\" and then \"git add here\".\n>\n> Because you didn't say what you are aiming for in the grander\n> picture, I thought you were \"making the UI simpler\"\n\nIn the original email, I wrote:\n> Okay, so this is part of my evil plan to make 'git add' DTRT wrt\n> submodules, and deprecate 'git submodule add' (I have some code\n> written down, but this is a prerequisite: I don't like the\n> .git/modules nonsense).\n\nI'm not sure how you inferred \"making the UI simpler\" from that, or the tests.\n\n> by making it\n> unnecessary for the users to say \"git clone\" himself as a separate\n> step before doing \"git add\". In such a world, \"add\" would internally\n> run \"clone\". If that were the case (I now know it is not), then the\n> configuration _is_ unnecessary, and it is perfectly valid to\n> question why you thought it is needed.\n\nNo, I would _never_ propose something that ugly. Neither my code,\ntests, nor my commit message indicates that I was going in that\ndirection, so I don't know where you got the idea from.\n\n> But if that is the direction you are aiming for, would it be\n> possible that the same configuration variable can and should cover\n> the use case (2) as well? After all, between \"git init here\" and\n> \"git add here\", the user may say (cd here && git pull $URL) and the\n> expected end result would be the same as (1), no?\n\nGood point. Yes, I would definitely want that.\n\n> If \"clone\" just calls init_db(), I would imagine that it might be\n> trivial to cover both cases by telling init_db() to pay attention to\n> the configuration, without doing much in the \"clone\" itself.\n\nRight. I'll start hacking.\n"},{"id":"214312","messageId":"516C21CF.5080705@xiplink.com","threadId":"33499","inReplyTo":"CALkWK0mvtRhFc0_4883ATNaYpb+kDwpV9VxeAoqJy5HxNQ6vgg@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Marc Branchaud","fromEmail":"mbranchaud@xiplink.com","sentAt":"2013-04-15T15:50:39Z","receivedAt":"2013-04-15T15:50:39Z","isPatch":true,"sender":{"key":"mbranchaud@xiplink.com","avatar":null},"body":"In general I think it is a mistake to overload \"git clone\" with the notion of\nadding a submodule. If I want to *add* something to a repository, I'll use\nsome kind of \"add\" command. To me \"git clone\" is not the kind of verb I\nwould expect to add something to some distant-parent .git directory.\n\nInstead of mucking around with\"git clone\" I would much rather see \"git add\"\nautodetect URLs and do the submodule thing:\n\tgit add ssh://host/blammo.git\nwould clone blammo.git into ./blammo/ and set it up as a submodule inside\n$PWD's git repo. (This may benefit from \"git clone\" learning some kind of\n--separate-git-dir option, but that's irrelevant to me.)\n\nOn 13-04-15 04:19 AM, Ramkumar Ramachandra wrote:\n>\n> Why would a user *want* a full clone inside a git worktree?\n\nPlease try to be careful with your assumptions.\n\nI could have\n\t~/.git/\nto maintain revisions of various personal files, config .dotfiles, scripts in\n~/bin/ and so on.\n\nI could also have various projects' repos under ~/Code, where I do my \"real\"\nwork:\n\t~/Code/git/.git/\n\t~/Code/DayJob/.git/\n\t~/Code/project-foo/.git/\n\nNow, are these Code/* repos inside ~/.git/'s worktree or not? I'd really\nprefer them not to be. I would be especially upset to have some \"magic\" that\nautomatically adds new clones inside ~/Code/ to ~/.git/.\n\n\t\tM.\n"},{"id":"214345","messageId":"7vvc7nu1hu.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"516C21CF.5080705@xiplink.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-15T17:50:37Z","receivedAt":"2013-04-15T17:50:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marc Branchaud writes:\n\n> In general I think it is a mistake to overload \"git clone\" with the notion of\n> adding a submodule.\n\nI agree with that principle, but my understanding is that this\neffort is not about teaching \"git clone\" to create a submodule.\n\nBoth \"git clone\" and \"git init\" already know how to use a directory\nthat is outside the working tree of the newly created repository to\nstore its $GIT_DIR and point at it with .git in the working tree\nusing the gitfile mechanism (their --separate-git-dir option). My\nunderstanding is that this \"config\" is about making that option\neasier to use when you _know_ any new repository you create with\n\"git clone\" or \"git init\" inside your (toplevel super)project's\nworking tree will become its submodule, as it is more convenient to\nhave their $GIT_DIR inside the .git/modules/$name of the\nsuperproject.\n\nAfter that \"clone\" or \"init\" creates a repository, you still have to\n\"add\" if you want to make it a submodule to the toplevel.\n\n> If I want to *add* something to a repository, I'll use\n> some kind of \"add\" command. To me \"git clone\" is not the kind of verb I\n> would expect to add something to some distant-parent .git directory.\n>\n> Instead of mucking around with\"git clone\" I would much rather see \"git add\"\n> autodetect URLs and do the submodule thing:\n>\n> \tgit add ssh://host/blammo.git\n>\n> would clone blammo.git into ./blammo/ and set it up as a submodule inside\n> $PWD's git repo.\n\nI do not think the addition Ram is envisioning in the patch will\nprevent you from teaching \"add\" to do that. An implemention of such\nan addition indeed would most likely use the same --separate-git-dir\nmechanism anyway.\n"},{"id":"214346","messageId":"CALkWK0=S3=cLxJ85M-efD7fym29Y_pD5XyeBsunkiuPV=vVR4w@mail.gmail.com","threadId":"33499","inReplyTo":"516C21CF.5080705@xiplink.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-15T17:50:56Z","receivedAt":"2013-04-15T17:50:56Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Marc Branchaud wrote:\n> git add ssh://host/blammo.git\n\nHeh. And I want git add *coffee* to make me coffee.\nWhat's your gripe with git submodule add?\n\n> I could have\n> ~/.git/\n> to maintain revisions of various personal files, config .dotfiles, scripts in\n> ~/bin/ and so on.\n> [...]\n> Now, are these Code/* repos inside ~/.git/'s worktree or not?\n\nPlease don't version your entire ~, effectively shooting yourself in the face?\n\nUse a dotfiles repo and write a simple Makefile to symlink ~/.etc to\n~/dotfiles/.etc. If you're looking for a good example, see\nhttps://github.com/artagnon/dotfiles.\n"},{"id":"214349","messageId":"CALkWK0n0y6OPJvYjNeEbUx_CC58vHRRLCsmJtws+RKyv3wRTwQ@mail.gmail.com","threadId":"33499","inReplyTo":"7vvc7nu1hu.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-15T18:00:40Z","receivedAt":"2013-04-15T18:00:40Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Junio C Hamano wrote:\n> My\n> understanding is that this \"config\" is about making that option\n> easier to use when you _know_ any new repository you create with\n> \"git clone\" or \"git init\" inside your (toplevel super)project's\n> working tree will become its submodule, as it is more convenient to\n> have their $GIT_DIR inside the .git/modules/$name of the\n> superproject.\n\nRight. But I'm still worried about .git/modules/$name. Can you\nexplain why it's a better idea than having a dedicated ~/bare? In the\ncase when I have it in ~/bare, I can do many more interesting things:\nfor instance, if I cloned a repository that is actually another\nproject's submodule for instance, I don't have to re-clone it when I\nclone that superproject. What's more? I can remove submodules and\nattach a worktree to my ~/bare/repo.git and use it as a separate\nrepository easily. I can move submodules between projects. In\ncomparison, .git/modules/$name just seems like a mess.\n\n> I do not think the addition Ram is envisioning in the patch will\n> prevent you from teaching \"add\" to do that. An implemention of such\n> an addition indeed would most likely use the same --separate-git-dir\n> mechanism anyway.\n\nWell, I'm against the change in principle because add operates on\nworktree paths, not URLs. I don't want to change that arbitrarily.\n"},{"id":"214360","messageId":"516C4A52.1080908@xiplink.com","threadId":"33499","inReplyTo":"7vvc7nu1hu.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2013-04-15T18:43:30Z","receivedAt":"2013-04-15T18:43:30Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 13-04-15 01:50 PM, Junio C Hamano wrote:\n> Marc Branchaud writes:\n> \n>> In general I think it is a mistake to overload \"git clone\" with the notion of\n>> adding a submodule.\n> \n> I agree with that principle, but my understanding is that this\n> effort is not about teaching \"git clone\" to create a submodule.\n> \n> Both \"git clone\" and \"git init\" already know how to use a directory\n> that is outside the working tree of the newly created repository to\n> store its $GIT_DIR and point at it with .git in the working tree\n> using the gitfile mechanism (their --separate-git-dir option). My\n> understanding is that this \"config\" is about making that option\n> easier to use when you _know_ any new repository you create with\n> \"git clone\" or \"git init\" inside your (toplevel super)project's\n> working tree will become its submodule, as it is more convenient to\n> have their $GIT_DIR inside the .git/modules/$name of the\n> superproject.\n> \n> After that \"clone\" or \"init\" creates a repository, you still have to\n> \"add\" if you want to make it a submodule to the toplevel.\n\nTo me it makes more sense to move the .git directory when the user invokes\n\"git submodule add\" instead of creating it in an unusual place when the\nsub-repo is cloned. After all, git can't *know* that it'll be a submodule\nuntil it's submodule-added to the super-repo. Sure, the user might have set\nclone.submoduleGitDir somewhere, but users make mistakes, and this setting\nmakes it harder to clean up a mistake:\n\tgit clone foo.git\n\t# Doh! I mean to clone foof.git!\n\trm -rf foo\n\t# Gah, now there's cruft in my clone.submoduleGitDir...\n\nAll that said, the basic idea of being able to configure where \"git clone\"\nstores .git directories might be reasonable. Something like\nclone.gitDirHome. It seems like something only a git hacker would ever care\nabout, but that's no reason not to have such a config option. OTOH, I still\ndon't see a reason for it, because I don't buy the submodule-at-clone-time\nargument.\n\n\t\tM.\n"},{"id":"214357","messageId":"20130415184347.GA21170@sigill.intra.peff.net","threadId":"33499","inReplyTo":"CALkWK0n0y6OPJvYjNeEbUx_CC58vHRRLCsmJtws+RKyv3wRTwQ@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-04-15T18:43:47Z","receivedAt":"2013-04-15T18:43:47Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Apr 15, 2013 at 11:30:40PM +0530, Ramkumar Ramachandra wrote:\n\n> Junio C Hamano wrote:\n> > My\n> > understanding is that this \"config\" is about making that option\n> > easier to use when you _know_ any new repository you create with\n> > \"git clone\" or \"git init\" inside your (toplevel super)project's\n> > working tree will become its submodule, as it is more convenient to\n> > have their $GIT_DIR inside the .git/modules/$name of the\n> > superproject.\n> \n> Right. But I'm still worried about .git/modules/$name. Can you\n> explain why it's a better idea than having a dedicated ~/bare?\n\nI do not have too much deep knowledge of submodules, nor have I been\nfollowing this thread very closely, but I have not seen how ~/bare would\nhandle per-submodule information?\n\nThat is, let us imagine I do:\n\n git clone $PROJECT one && cd one && git submodule update foo\n git clone $PROJECT two && cd two && git submodule update foo\n\nThe current scheme would put the cloned modules into\none/.git/modules/foo and two/.git/modules/foo, respectively. Let us\nimagine instead that the first one writes to ~/modules/$URL (assuming\nsome sane mapping of the URL into the filesystem), and the second one\nsays \"A-ha, I already have ~/modules/$URL, so I can skip cloning it\".\n\nBut that is not the end of the story. If I do:\n\n cd one/foo &&\n hack hack hack &&\n git commit -m foo &&\n cd .. &&\n git commit -m 'updated submodule'\n\nyou would not want to see a dirty, updated submodule in project \"two\".\nYou did not touch \"two/foo\" nor advance its HEAD at all.\n\nSo there is some information that is per-clone (the objects, the remote\ntips), but there is some information that is per-submodule (where our\nlocal branches are, the index, the worktree). I can see why it is\nadvantageous to share the per-clone information between similar clones\n(because it avoids disk space and network transfer). But I do not think\nyou can escape having some form of per-submodule repo, even if it is a\nthin git-new-workdir-ish repo that points back to a parent repo for the\nclone.\n\nIs there some part of your proposal that I am missing? It seems like you\nwould still need one/.git/modules/foo for this \"thin\" repo.\n\nAnd once we separate out those concerns, I also do not see why sharing\nper-clone information needs to be related to submodules at all. If I do:\n\n git clone $URL one &&\n git clone $URL two\n\nthose can potentially be shared in the same way as two submodule repos\nthat happen to point to the same $URL. It would make sense to me to\nimprove such a shared-object setup independently, and then build the\nshared-submodule storage on top of that.\n\nAnd by the way, I am actually not sure that such a shared-object setup\nis a good idea, but only that _if_ you are going to do it with\nsubmodules, you might as well do it for all repos. In theory, it is not\nthat hard to have a big per-user object-only repository (either for all\nrepos, or for related ones). But we can do that already with \"git clone\n-s\", and people do not generally bother, because the maintenance is very\ntricky (especially dealing with reachability and pruning).\n\nI am open to the argument that solving it in a specific case\n(submodules) lets us make assumptions that simplify the problem from the\ngeneral case, but I do not offhand see how it would be any easier in\nthis case.\n"},{"id":"214359","messageId":"7va9ozsk60.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"516C4A52.1080908@xiplink.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"junio@pobox.com","sentAt":"2013-04-15T18:50:15Z","receivedAt":"2013-04-15T18:50:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marc Branchaud writes:\n\n>> After that \"clone\" or \"init\" creates a repository, you still have to\n>> \"add\" if you want to make it a submodule to the toplevel.\n>\n> To me it makes more sense to move the .git directory when the user invokes\n> \"git submodule add\" instead of creating it in an unusual place when the\n> sub-repo is cloned. After all, git can't *know* that it'll be a submodule\n> until it's submodule-added to the super-repo.\n\nIt does not relieve \"git add\" (or \"git submodulea add\") from the\nresponsibility of moving .git directory. It only reduces the need\nto do so.\n\nWhen the user says \"add\" and the repository has .git directory in\nit, \"add\" (or \"submodule add\") is still responsible for relocating\nit.\n"},{"id":"214358","messageId":"516C4BEB.7030507@xiplink.com","threadId":"33499","inReplyTo":"CALkWK0n0y6OPJvYjNeEbUx_CC58vHRRLCsmJtws+RKyv3wRTwQ@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2013-04-15T18:50:19Z","receivedAt":"2013-04-15T18:50:19Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 13-04-15 02:00 PM, Ramkumar Ramachandra wrote:\n> Junio C Hamano wrote:\n>> \n>> I do not think the addition Ram is envisioning in the patch will\n>> prevent you from teaching \"add\" to do that. An implemention of such\n>> an addition indeed would most likely use the same --separate-git-dir\n>> mechanism anyway.\n> \n> Well, I'm against the change in principle because add operates on\n> worktree paths, not URLs. I don't want to change that arbitrarily.\n\nI don't understand that statement.\n\nIf \"git add\" is all about specifying what lives under paths in the worktree,\nwhat's wrong with letting \"git add\" go beyond specifying just files?\n\nSyntax aside for the moment, I think a command like\n\tgit add git-repo-reference foo\nis perfectly natural: It specifies what is inside worktree path foo.\n\n\t\tM.\n"},{"id":"214384","messageId":"516C63DA.4080209@xiplink.com","threadId":"33499","inReplyTo":"7va9ozsk60.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2013-04-15T20:32:26Z","receivedAt":"2013-04-15T20:32:26Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 13-04-15 02:50 PM, Junio C Hamano wrote:\n> Marc Branchaud writes:\n> \n>>> After that \"clone\" or \"init\" creates a repository, you still have to\n>>> \"add\" if you want to make it a submodule to the toplevel.\n>>\n>> To me it makes more sense to move the .git directory when the user invokes\n>> \"git submodule add\" instead of creating it in an unusual place when the\n>> sub-repo is cloned. After all, git can't *know* that it'll be a submodule\n>> until it's submodule-added to the super-repo.\n> \n> It does not relieve \"git add\" (or \"git submodulea add\") from the\n> responsibility of moving .git directory. It only reduces the need\n> to do so.\n> \n> When the user says \"add\" and the repository has .git directory in\n> it, \"add\" (or \"submodule add\") is still responsible for relocating\n> it.\n\nSo it looks like the proposed change to git-clone provides no benefit to the\nsubmodule-adding machinery, which still needs to know when and how to\nrelocate .git directories.\n\nRam, assuming Junio's explanations match your intentions, if the whole\nmotivation for this change is \"to make 'git add' DTRT wrt submodules, and\ndeprecate 'git submodule add'\" then I don't think it's bringing you any\ncloser to that goal.\n\n\t\tM.\n"},{"id":"214386","messageId":"7vd2tvqzy8.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"20130415184347.GA21170@sigill.intra.peff.net","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-15T20:52:15Z","receivedAt":"2013-04-15T20:52:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> And by the way, I am actually not sure that such a shared-object setup\n> is a good idea, but only that _if_ you are going to do it with\n> submodules, you might as well do it for all repos. In theory, it is not\n> that hard to have a big per-user object-only repository (either for all\n> repos, or for related ones). But we can do that already with \"git clone\n> -s\", and people do not generally bother, because the maintenance is very\n> tricky (especially dealing with reachability and pruning).\n>\n> I am open to the argument that solving it in a specific case\n> (submodules) lets us make assumptions that simplify the problem from the\n> general case, but I do not offhand see how it would be any easier in\n> this case.\n\nNicely put.\n\nMaking it easier to manage such a shared object store by limiting\nuse cases is somewhat an intriguing idea, but those I can think of\noffhand all have to involve a use case without any rewound history,\nso being a submodules repository would not help.\n"},{"id":"214387","messageId":"7v8v4jqzru.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"516C63DA.4080209@xiplink.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-15T20:56:05Z","receivedAt":"2013-04-15T20:56:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marc Branchaud writes:\n\n> So it looks like the proposed change to git-clone provides no benefit to the\n> submodule-adding machinery, which still needs to know when and how to\n> relocate .git directories.\n>\n> Ram, assuming Junio's explanations match your intentions, if the whole\n\nThat is a huge assumption, given that I have a proven track record\nof guessing Ram's intention wrong ;-).\n\n> motivation for this change is \"to make 'git add' DTRT wrt submodules, and\n> deprecate 'git submodule add'\" then I don't think it's bringing you any\n> closer to that goal.\n"},{"id":"214421","messageId":"20130416025840.GH3262@elie.Belkin","threadId":"33499","inReplyTo":"1365881007-25731-1-git-send-email-artagnon@gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-04-16T02:58:40Z","receivedAt":"2013-04-16T02:58:40Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> When set,\n> instead of cloning the given repository as-is, it relocates the gitdir\n> of the repository to the path specified by this variable.\n\nInteresting. As the discussion downthread from this illustrated, I am\nnot convinced this is better than a subcommand of \"git submodule\" for\nthat particular purpose, yet.\n\nIs the goal to be able to, under some certain configuration, make\n\"git clone\" + \"git add\" behave like \"git submodule add\"?\n\n[...]\n> I don't like the\n> .git/modules nonsense).\n\nAs Jeff mentioned, a given repository can be a subproject of multiple\ndifferent containing projects, that use different versions of it.\nIt doesn't make sense for different directories on the filesystem to\nshare an index anyway.\n\nDo you want the subprojects to be symlinks to the One True Version\nof each project? (I can see that working ok in some workflows.) Or\ndo you want subprojects to be lightweight workdirs like\ngit-new-workdir creates, with .git/objects pointing to the project's\nOne True Object Store?\n\nThat is the part of this design that seems least well fleshed out to\nme at the moment.\n\nI quite like .git/modules/ (for some reasons that\nI've mentioned in other threads) and don't consider it nonsense, which\nmakes me assume I don't understand the goal of this patch, either.\nPlease don't take that personally.\n\nHope that helps,\nJonathan\n"},{"id":"214446","messageId":"CALkWK0nUzbt6R=raWaxxVgAthcUo7E+_FS0rPDDfumgeecHiZg@mail.gmail.com","threadId":"33499","inReplyTo":"20130415184347.GA21170@sigill.intra.peff.net","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-16T08:13:33Z","receivedAt":"2013-04-16T08:13:33Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Jeff King wrote:\n> So there is some information that is per-clone (the objects, the remote\n> tips), but there is some information that is per-submodule (where our\n> local branches are, the index, the worktree). I can see why it is\n> advantageous to share the per-clone information between similar clones\n> (because it avoids disk space and network transfer). But I do not think\n> you can escape having some form of per-submodule repo, even if it is a\n> thin git-new-workdir-ish repo that points back to a parent repo for the\n> clone.\n\nI want the flexibility to do the following:\n\n1. Do a \"simple clone\", where the clone contains the GITDIR embedded\nin the worktree. This is the most common case, and there is no reason\nto complicate it. I can optionally attach additional workdirs to this\nclone. I can also optionally relocate the GITDIR at a later date, if\nI feel the need to do so.\n\n2. Attach a worktree to any object store without having to write a\ngitfile and set core.worktree by hand. The limitation is that you\ncan't have two submodules from two different superprojects sharing the\nsame object store (since both of them are worktrees). However, for\nthe purpose of working on the submodule repository as an independent\nrepository (this is a very common case for me), I can attach a new\n\"workdir\" to the GITDIR very easily.\n\n3. Attach multiple submodules to the same object store. This will\nrequire maintaining a separate index, HEAD and logs/HEAD (aka.\nworkdir) for each additional submodule (the first one doesn't need it)\nin .git/modules of the superproject.\n\n> Is there some part of your proposal that I am missing? It seems like you\n> would still need one/.git/modules/foo for this \"thin\" repo.\n\nYou're talking about #3, while I'm still working on #2. And why do\nyou want to use a hammer again if I don't want to share the same\nobject store with multiple submodules? This .git/modules/ is\ncompletely optional, and is only required for the _second_ submodule\nonwards that I'm attaching to the same object store.\n\n> And by the way, I am actually not sure that such a shared-object setup\n> is a good idea, but only that _if_ you are going to do it with\n> submodules, you might as well do it for all repos. In theory, it is not\n> that hard to have a big per-user object-only repository (either for all\n> repos, or for related ones). But we can do that already with \"git clone\n> -s\", and people do not generally bother, because the maintenance is very\n> tricky (especially dealing with reachability and pruning).\n\nNo, no. I'm against dumping objects from all repositories into one\ngiant object store. That's a sledgehammer solution, while I'm looking\nfor control and flexibility. Moreover, it has lots of downsides, as\nyou already pointed out.\n\n> I am open to the argument that solving it in a specific case\n> (submodules) lets us make assumptions that simplify the problem from the\n> general case, but I do not offhand see how it would be any easier in\n> this case.\n\nSo my proposal is to build a new first-class tool to make\nmanipulations in #1, #2 and #3 easily possible. The first step is to\nformalize the names \"bare worktree\" (which refers to a worktree with a\ngitfile), \"worktree\" (which refers to a worktree with a GITDIR\nembedded in it), and \"workdir\" (which refers to a worktree with a\n\"thin\" GITDIR).\n\nThe reason I want to build it for submodules first is because the\nnon-submodule case (#2) is simply a reduced case of the submodule case\n(#3):\n\n- When I attempt to attach a new worktree to an existing GITDIR with a\nworktree attached, I will create a workdir instead. This simply\ninvolves creating a thin .git directory in the worktree in the\nnon-submodule case. In the submodule case, it is more complicated: I\nhave to locate the superproject's .git directory, and put it there.\n"},{"id":"214447","messageId":"CALkWK0kEQ+mCxkaqUusyaEpx350qNrJ8UPoeo7+hEVGEUbtaxQ@mail.gmail.com","threadId":"33499","inReplyTo":"516C4BEB.7030507@xiplink.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-16T08:17:02Z","receivedAt":"2013-04-16T08:17:02Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Marc Branchaud wrote:\n> If \"git add\" is all about specifying what lives under paths in the worktree,\n> what's wrong with letting \"git add\" go beyond specifying just files?\n>\n> Syntax aside for the moment, I think a command like\n> git add git-repo-reference foo\n> is perfectly natural: It specifies what is inside worktree path foo.\n\nI never said \"just files\". Files, directories, symlinks and\nsubmodules are all \"things in the worktree\", and all fine. Remote\nURLs, on the other hand, have nothing to do with the worktree.\n"},{"id":"214448","messageId":"CALkWK0=2+RY0cRSJD4pbHxPuqffDEqiwc7m0+Fzk7d8=wLvULQ@mail.gmail.com","threadId":"33499","inReplyTo":"7va9ozsk60.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-16T08:21:34Z","receivedAt":"2013-04-16T08:21:34Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Junio C Hamano wrote:\n> It does not relieve \"git add\" (or \"git submodulea add\") from the\n> responsibility of moving .git directory. It only reduces the need\n> to do so.\n>\n> When the user says \"add\" and the repository has .git directory in\n> it, \"add\" (or \"submodule add\") is still responsible for relocating\n> it.\n\nSince you're so stubborn about it, I suppose 'git add' could call a\nfunction in my \"new first-class program to attach detach\nworktrees/workdirs and relocate GITDIRs\" as a last resort (if the user\nsomehow managed to put a GITDIR in the submodule worktree despite our\nwell-designed tools). But last resort is not what we should be\ndiscussing now: we're discussing what the design should ideally be.\nAnd ideally, I think we both agree that it's best if init/clone did\nthe relocation.\n"},{"id":"214465","messageId":"CALkWK0kDgSicNejydLsH6iqj-yDYGz6CKd+kbn4EW1HxgAxsBA@mail.gmail.com","threadId":"33499","inReplyTo":"20130416025840.GH3262@elie.Belkin","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-16T08:36:36Z","receivedAt":"2013-04-16T08:36:36Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Jonathan Nieder wrote:\n> I quite like .git/modules/ (for some reasons that\n> I've mentioned in other threads) and don't consider it nonsense, which\n> makes me assume I don't understand the goal of this patch, either.\n> Please don't take that personally.\n\nThere's nothing to take personally, Jonathan. We're designing\nsoftware, and the rationale for choosing a design is never \"Jonathan\npersonally likes this particular design, so therefore we'll go with\nit\", but rather \"Ram's design is objectively superior, and therefore\nwe'll go with it\". I'll proceed with bashing .git/modules, while your\njob is to defend it:\n\n1. The path to the object store of a submodule depends upon how deeply\nit is nested in other submodules, and hence how many /modules/\ncomponents to add to the path to the project's name. Presumably, this\nis to avoid conflicts: but it's an overkill for such a simple job. In\nthe 98% case, I never have two submodules with the same name in my\nsuperproject; for the 2% case, I can live with the inconvenience of\nnaming a directory by hand, rather than putting up with this ugliness.\n\n2. This ugliness complicates implementation of add/ rm/ mv, because\neach of them will have to know about this contrived path solution.\n\n3. The paths in the gitfiles in various submodules is horribly ugly\nwith tons of ../ components. This is especially the case in deeply\nnested submodules. We can't use an absolute path, because the\nsuperproject directory can be moved anywhere in the filesystem.\n\n4. To relocate the object store and reuse it elsewhere is almost\nimpossible. What if I want to remove the submodule, but work on it\nindependently from the superproject? Re-clone?\n\nMy solution fixes all these problems, and we need\n.git/modules/.git (no path-to-submodule nonsense) only as a last\nresort: #3 (ref: my email to Peff).\n"},{"id":"214493","messageId":"516D70BF.3050006@xiplink.com","threadId":"33499","inReplyTo":"CALkWK0nUzbt6R=raWaxxVgAthcUo7E+_FS0rPDDfumgeecHiZg@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2013-04-16T15:39:43Z","receivedAt":"2013-04-16T15:39:43Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 13-04-16 04:13 AM, Ramkumar Ramachandra wrote:\n> Jeff King wrote:\n>> So there is some information that is per-clone (the objects, the remote\n>> tips), but there is some information that is per-submodule (where our\n>> local branches are, the index, the worktree). I can see why it is\n>> advantageous to share the per-clone information between similar clones\n>> (because it avoids disk space and network transfer). But I do not think\n>> you can escape having some form of per-submodule repo, even if it is a\n>> thin git-new-workdir-ish repo that points back to a parent repo for the\n>> clone.\n> \n> I want the flexibility to do the following:\n> \n> 1. Do a \"simple clone\", where the clone contains the GITDIR embedded\n> in the worktree. This is the most common case, and there is no reason\n> to complicate it. I can optionally attach additional workdirs to this\n> clone. I can also optionally relocate the GITDIR at a later date, if\n> I feel the need to do so.\n> \n> 2. Attach a worktree to any object store without having to write a\n> gitfile and set core.worktree by hand. The limitation is that you\n> can't have two submodules from two different superprojects sharing the\n> same object store (since both of them are worktrees). However, for\n> the purpose of working on the submodule repository as an independent\n> repository (this is a very common case for me), I can attach a new\n> \"workdir\" to the GITDIR very easily.\n\nDoesn't contrib/workdir/git-new-workdir do this?\n\n\t\tM.\n"},{"id":"214494","messageId":"516D723F.9070204@xiplink.com","threadId":"33499","inReplyTo":"CALkWK0kEQ+mCxkaqUusyaEpx350qNrJ8UPoeo7+hEVGEUbtaxQ@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2013-04-16T15:46:07Z","receivedAt":"2013-04-16T15:46:07Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 13-04-16 04:17 AM, Ramkumar Ramachandra wrote:\n> Marc Branchaud wrote:\n>> If \"git add\" is all about specifying what lives under paths in the worktree,\n>> what's wrong with letting \"git add\" go beyond specifying just files?\n>>\n>> Syntax aside for the moment, I think a command like\n>> git add git-repo-reference foo\n>> is perfectly natural: It specifies what is inside worktree path foo.\n> \n> I never said \"just files\". Files, directories, symlinks and\n> submodules are all \"things in the worktree\", and all fine. Remote\n> URLs, on the other hand, have nothing to do with the worktree.\n\nBut they have everything to do with submodules. You need a URL to identify a\nsubmodule. If you want a submodule in your worktree, at some point you have\nto specify the submodule's URL.\n\nI really feel like I'm missing something here. You seem to be saying that\nit's wrong to let \"git add\" interpret a URL as a submodule. Instead you seem\nto want to have some other mechanism create the files, directories and\nsymlinks that make up a submodule, so that \"git add\" can then operate with\nthe purity you desire. That's what I don't understand.\n\nAs a submodule user, I want to \"git add\" a submodule. I don't see why it's\nnecessary to have more than one command to do that. But if you're saying\nthat it's fine for \"git add\" to work this way, then I don't see the point of\nthe proposed change to \"git clone\".\n\n\t\tM.\n"},{"id":"214495","messageId":"516D7241.8050901@xiplink.com","threadId":"33499","inReplyTo":"CALkWK0=2+RY0cRSJD4pbHxPuqffDEqiwc7m0+Fzk7d8=wLvULQ@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2013-04-16T15:46:09Z","receivedAt":"2013-04-16T15:46:09Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 13-04-16 04:21 AM, Ramkumar Ramachandra wrote:\n> Junio C Hamano wrote:\n>> It does not relieve \"git add\" (or \"git submodulea add\") from the\n>> responsibility of moving .git directory. It only reduces the need\n>> to do so.\n>>\n>> When the user says \"add\" and the repository has .git directory in\n>> it, \"add\" (or \"submodule add\") is still responsible for relocating\n>> it.\n> \n> Since you're so stubborn about it, I suppose 'git add' could call a\n> function in my \"new first-class program to attach detach\n> worktrees/workdirs and relocate GITDIRs\" as a last resort (if the user\n> somehow managed to put a GITDIR in the submodule worktree despite our\n> well-designed tools). But last resort is not what we should be\n> discussing now: we're discussing what the design should ideally be.\n> And ideally, I think we both agree that it's best if init/clone did\n> the relocation.\n\nIf that's the question, then put me on the \"disagree\" side. I just don't see\nwhy that approach is \"best\", especially if the intention is \"to make 'git\nadd' DTRT wrt submodules, and deprecate 'git submodule add'\".\n\n\t\tM.\n"},{"id":"214499","messageId":"7v38uqo059.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"CALkWK0kDgSicNejydLsH6iqj-yDYGz6CKd+kbn4EW1HxgAxsBA@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-16T17:28:34Z","receivedAt":"2013-04-16T17:28:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramkumar Ramachandra writes:\n\n> My solution fixes all these problems, and we need\n> .git/modules/.git (no path-to-submodule nonsense) only as a last\n> resort: #3 (ref: my email to Peff).\n\nHave you noticed that there are distinction between submodule path\nand submodule name already in the current system, and name is\nderived from path if you do not give it when adding a submodule\nmerely as a convenience?\n\nIf some existing code uses .git/modules/.git in \"git submodule\",\nthat is a bug that needs to be fixed.\n"},{"id":"214597","messageId":"CACsJy8D-5x5HXgpr2hHUHee6jcfj3++b961sJB_aKTZC1ZS+tw@mail.gmail.com","threadId":"33499","inReplyTo":"1365881007-25731-1-git-send-email-artagnon@gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-04-17T10:22:59Z","receivedAt":"2013-04-17T10:22:59Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sun, Apr 14, 2013 at 5:23 AM, Ramkumar Ramachandra\n wrote:\n> This configuration variable comes into effect when 'git clone' is\n> invoked inside an existing git repository's worktree. When set,\n> instead of cloning the given repository as-is, it relocates the gitdir\n> of the repository to the path specified by this variable. This\n> setting is especially useful when working with submodules.\n\nWhat if I clone a repo then realize it was a mistake and remove it?\nWith current clone, a \"rm -rf\" would do. With this, I'll need to\nfigure out which subdir in the top .git contains the repo I want to\nremove. I'm not sure how \"git submodule\" handles this case though\n(i.e. total submodule ignorant speaking..)\n--\nDuy\n"},{"id":"214600","messageId":"CALkWK0kw69rMveDXpGEvV=fGxiQ7JoT_JE9ZU5cor0xD=BUbFQ@mail.gmail.com","threadId":"33499","inReplyTo":"CACsJy8D-5x5HXgpr2hHUHee6jcfj3++b961sJB_aKTZC1ZS+tw@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-17T10:53:20Z","receivedAt":"2013-04-17T10:53:20Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Duy Nguyen wrote:\n> What if I clone a repo then realize it was a mistake and remove it?\n> With current clone, a \"rm -rf\" would do. With this, I'll need to\n> figure out which subdir in the top .git contains the repo I want to\n> remove. I'm not sure how \"git submodule\" handles this case though\n> (i.e. total submodule ignorant speaking..)\n\nCurrently, submodules relocate the GITDIR of submodules to\n.git/modules. So, my proposed patch doesn't make the situation any\nworse. In fact, it improves the situation because you're guaranteed\nthat all your GITDIRs will be in ~/bare (or whatever your\ncore.submoduleGitDir is), as opposed to a complex path in .git/modules\nof your containing superproject.\n"},{"id":"214601","messageId":"CACsJy8C9mrJzmg4FjqBMAZis7WQUpyhNH7TMTLbebWQE124YMg@mail.gmail.com","threadId":"33499","inReplyTo":"CALkWK0kw69rMveDXpGEvV=fGxiQ7JoT_JE9ZU5cor0xD=BUbFQ@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-04-17T10:59:05Z","receivedAt":"2013-04-17T10:59:05Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Apr 17, 2013 at 8:53 PM, Ramkumar Ramachandra\n wrote:\n> Duy Nguyen wrote:\n>> What if I clone a repo then realize it was a mistake and remove it?\n>> With current clone, a \"rm -rf\" would do. With this, I'll need to\n>> figure out which subdir in the top .git contains the repo I want to\n>> remove. I'm not sure how \"git submodule\" handles this case though\n>> (i.e. total submodule ignorant speaking..)\n>\n> Currently, submodules relocate the GITDIR of submodules to\n> .git/modules. So, my proposed patch doesn't make the situation any\n> worse. In fact, it improves the situation because you're guaranteed\n> that all your GITDIRs will be in ~/bare (or whatever your\n> core.submoduleGitDir is), as opposed to a complex path in .git/modules\n> of your containing superproject.\n\nNo, submodule code does not change \"git clone\". If I'm not mistaken,\nsubmodule will not kick in until you type \"git submodule something\".\nIf I turn clone.submoduleGitDir on, how can I undo my mistake in a\nuser friendly way?\n--\nDuy\n"},{"id":"214603","messageId":"CALkWK0nLamX1XKcg2t7VWJTPuFhX+ctEGE=4sjSSd7JqMmGzPA@mail.gmail.com","threadId":"33499","inReplyTo":"CACsJy8C9mrJzmg4FjqBMAZis7WQUpyhNH7TMTLbebWQE124YMg@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-17T11:13:10Z","receivedAt":"2013-04-17T11:13:10Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Duy Nguyen wrote:\n> No, submodule code does not change \"git clone\". If I'm not mistaken,\n> submodule will not kick in until you type \"git submodule something\".\n> If I turn clone.submoduleGitDir on, how can I undo my mistake in a\n> user friendly way?\n\nSo, if you currently want to add a submodule, you have to 'git\nsubmodule add', which runs clone internally apart from other things.\nHow do you undo this mistake?\n\nWhat I'm essentially proposing is to give the job of cloning back to\nclone, and the job of adding back to add, instead of creating an\nunnatural abstraction over them using 'git submodule add'. The point\nbeing: why would you ever _want_ to clone inside a worktree unless you\nintend to add a submodule? In other words, you intent for running a\n'git clone' inside a worktree is exactly the same as your intent for\nrunning a 'git submodule add' inside a worktree. Ofcourse, if you\nhave a fringe case where that was _not_ your intent, we'll provide a\ncommand-line switch to turn off the relocation for that clone.\n"},{"id":"214605","messageId":"CACsJy8DxspNbopJbSsvcCZZwMFees1JVV_iV5r7dXRJTngzmFA@mail.gmail.com","threadId":"33499","inReplyTo":"CALkWK0nLamX1XKcg2t7VWJTPuFhX+ctEGE=4sjSSd7JqMmGzPA@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-04-17T11:36:35Z","receivedAt":"2013-04-17T11:36:35Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Apr 17, 2013 at 9:13 PM, Ramkumar Ramachandra\n wrote:\n> Duy Nguyen wrote:\n>> No, submodule code does not change \"git clone\". If I'm not mistaken,\n>> submodule will not kick in until you type \"git submodule something\".\n>> If I turn clone.submoduleGitDir on, how can I undo my mistake in a\n>> user friendly way?\n>\n> So, if you currently want to add a submodule, you have to 'git\n> submodule add', which runs clone internally apart from other things.\n> How do you undo this mistake?\n\nWell, it has \"submodule\" in the command line. My first reaction would\nbe looking for \"git submodule rm\" or something.\n\n> What I'm essentially proposing is to give the job of cloning back to\n> clone, and the job of adding back to add, instead of creating an\n> unnatural abstraction over them using 'git submodule add'. The point\n> being: why would you ever _want_ to clone inside a worktree unless you\n> intend to add a submodule? In other words, you intent for running a\n> 'git clone' inside a worktree is exactly the same as your intent for\n> running a 'git submodule add' inside a worktree. Ofcourse, if you\n> have a fringe case where that was _not_ your intent, we'll provide a\n> command-line switch to turn off the relocation for that clone.\n\nNo, the point is people make mistakes. What do we do in that case? Or\nwill you introduce yet another \"gc\" command for clean up ~/bare?\n--\nDuy\n"},{"id":"214610","messageId":"CALkWK0kchO-cKuh1vd=aziZa5CA8w81aEecUKqhazp_Y7pOrkw@mail.gmail.com","threadId":"33499","inReplyTo":"CACsJy8DxspNbopJbSsvcCZZwMFees1JVV_iV5r7dXRJTngzmFA@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2013-04-17T15:02:25Z","receivedAt":"2013-04-17T15:02:25Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"Duy Nguyen wrote:\n> Well, it has \"submodule\" in the command line. My first reaction would\n> be looking for \"git submodule rm\" or something.\n\nNo, 'git submodule rm' cannot remove the corresponding GITDIR. What\nif there are other branches that refer to the submodule? What if you\nwant to remove it from this branch and add it to another branch?\n\n> No, the point is people make mistakes. What do we do in that case? Or\n> will you introduce yet another \"gc\" command for clean up ~/bare?\n\nSo, people don't make mistakes when they use 'git submodule add', but\ndo make mistakes when using 'git clone'? How has the problem\n_changed_ with my patch? It's reasonable to point it out as an\nexisting problem, and ask for it to be fixed independent of this\ndiscussion, but that is not what you are doing.\n\ngit cannot read your mind to determine if you made a mistake, if\nthat's what you're asking. No, a gc equivalent won't work either (and\nthere's nothing in the current submodule world), because it is\nimpossible to determine if a workdir is attached to that GITDIR\nsomewhere on your filesystem.\n\nYou'll have to do _something_ to say that you don't want that GITDIR\nanymore. It's reasonable to request tooling to help with this task,\nbut your request is entirely different.\n"},{"id":"214616","messageId":"20130417154807.GA3499@elie.Belkin","threadId":"33499","inReplyTo":"CALkWK0kDgSicNejydLsH6iqj-yDYGz6CKd+kbn4EW1HxgAxsBA@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-04-17T15:48:07Z","receivedAt":"2013-04-17T15:48:07Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ramkumar Ramachandra wrote:\n\n> 2. This ugliness complicates implementation of add/ rm/ mv, because\n> each of them will have to know about this contrived path solution.\n\nWhy is that? Can't they look at the gitfile or call some helper\n(that happens to be part of the same binary)?\n"},{"id":"214622","messageId":"7vfvypf54k.fsf@alter.siamese.dyndns.org","threadId":"33499","inReplyTo":"CACsJy8DxspNbopJbSsvcCZZwMFees1JVV_iV5r7dXRJTngzmFA@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-04-17T17:18:03Z","receivedAt":"2013-04-17T17:18:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen writes:\n\n> No, the point is people make mistakes. What do we do in that case? Or\n> will you introduce yet another \"gc\" command for clean up ~/bare?\n\nI do not know if it will be a \"gc\", but we would need a way for the\nuser to say \"I no longer need the repository for this submodule kept\nlocally here (I may have to re-clone when I check out a version that\nneeds the submodule)\" to free up the .git/modules/ directories\nin the superproject. We might want to allow \"submodule deinit\" to\nalso ask for it, but \"deinit\" will not be the only occasion the user\nmight want it.\n\nIt is already a problem that needs to be addressed in the current setup.\n"},{"id":"214650","messageId":"CACsJy8AZK4iG4FsM=2wVTggyABBQUeeqb5_3qkWfuAqp0QhKUA@mail.gmail.com","threadId":"33499","inReplyTo":"CALkWK0kchO-cKuh1vd=aziZa5CA8w81aEecUKqhazp_Y7pOrkw@mail.gmail.com","subject":"Re: [RFC/PATCH] clone: introduce clone.submoduleGitDir to relocate $GITDIR","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-04-17T23:01:29Z","receivedAt":"2013-04-17T23:01:29Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Apr 18, 2013 at 1:02 AM, Ramkumar Ramachandra\n wrote:\n>> No, the point is people make mistakes. What do we do in that case? Or\n>> will you introduce yet another \"gc\" command for clean up ~/bare?\n>\n> So, people don't make mistakes when they use 'git submodule add', but\n> do make mistakes when using 'git clone'? How has the problem\n> _changed_ with my patch? It's reasonable to point it out as an\n> existing problem, and ask for it to be fixed independent of this\n> discussion, but that is not what you are doing.\n\nIt's the magic in git-clone that changes its behavior that I want to\naddress. I know you agree to go with a command line option. But I\nthink in the end there will be a switch hidden somewhere in config to\nmake things smooth, unless you make this mode the default (*). With\nnormal mode, \"rm -rf repo\" is enough, with the new submodule mode, it\nleaves some garbage behind that the user may not be aware about. Maybe\nthis is something that should be addressed anyway even for .gitmodules\nmode like Junio said. But I wonder if there are any other traps that\ncome with the config switch.\n\n(*) I don't think you can make the new mode the default though. There\nare repos in repos in the field that are not managed by \"git\nsubmodule\". Switching the default will disrupt those setups. Some\ndeprecation cycles might help, I don't know.\n--\nDuy\n"}]}