{"thread":{"id":"42882","subject":"Current state of Git worktree used with submodules?","startedAt":"2016-07-19T20:59:53Z","lastAt":"2016-08-03T21:48:24Z","messageCount":31,"participants":["Lars Schneider","Duy Nguyen","Nguyễn Thái Ngọc Duy","Stefan Beller","Jens Lehmann","Junio C Hamano","Max Kirillov","Jakub Narębski"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"291768","messageId":"005DA57F-8976-43A1-B833-D5EFADC75BEF@gmail.com","threadId":"42882","inReplyTo":null,"subject":"Current state of Git worktree used with submodules?","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2016-07-19T20:59:59Z","receivedAt":"2016-07-19T20:59:53Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"Hi,\n\nsome time ago Michael wrote in a blog post [1]:\n\"It's not recommended to use git worktree with a repository that contains submodules.\"\n\nPlus \"Documentation/git-worktree.txt\" states:\n\"Multiple checkout in general is still experimental, and the support\nfor submodules is incomplete. It is NOT recommended to make multiple\ncheckouts of a superproject.\"\n\nI wonder about the current state of this limitation. Is the statement still valid? \nIf yes, do you know if someone is working on this? If nobody is working on this, do\nyou have some pointers for me what the main problems are?\n\nThank you,\nLars\n\n\n[1] https://github.com/blog/2042-git-2-5-including-multiple-worktrees-and-triangular-workflows"},{"id":"291787","messageId":"CACsJy8ADRWNL3FR2TtWShviT4Lc4m1xaY8VOPP26Foyq+_A-3g@mail.gmail.com","threadId":"42882","inReplyTo":"005DA57F-8976-43A1-B833-D5EFADC75BEF@gmail.com","subject":"Re: Current state of Git worktree used with submodules?","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-20T04:14:56Z","receivedAt":"2016-07-20T04:16:12Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Jul 19, 2016 at 10:59 PM, Lars Schneider\n<larsxschneider@gmail.com> wrote:\n> Hi,\n>\n> some time ago Michael wrote in a blog post [1]:\n> \"It's not recommended to use git worktree with a repository that contains submodules.\"\n>\n> Plus \"Documentation/git-worktree.txt\" states:\n> \"Multiple checkout in general is still experimental, and the support\n> for submodules is incomplete. It is NOT recommended to make multiple\n> checkouts of a superproject.\"\n>\n> I wonder about the current state of this limitation. Is the statement still valid?\n\nYes.\n\n> If yes, do you know if someone is working on this? If nobody is working on this, do\n> you have some pointers for me what the main problems are?\n\nThe blocker is config file being shared (you do not want to share\ncore.worktree and submodule.*). I made some progress last weekend,\njust needed to add some tests to see if submodule works as expected\nand will post the series soon. Then you can take over if you want ;)\n\nNote that I only try to make submodules work with multi worktree, not\nwork optimally. A more ambitious goal is sharing submodule repos, so\nyou can keep disk usage down...\n\n> [1] https://github.com/blog/2042-git-2-5-including-multiple-worktrees-and-triangular-workflows\n-- \nDuy\n"},{"id":"291835","messageId":"20160720172419.25473-1-pclouds@gmail.com","threadId":"42882","inReplyTo":"CACsJy8ADRWNL3FR2TtWShviT4Lc4m1xaY8VOPP26Foyq+_A-3g@mail.gmail.com","subject":"[PATCH v4 0/4] Split .git/config in multiple worktree setup","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-20T17:24:15Z","receivedAt":"2016-07-20T17:25:09Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Jul 20, 2016 at 6:14 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n>> If yes, do you know if someone is working on this? If nobody is working on this, do\n>> you have some pointers for me what the main problems are?\n>\n> The blocker is config file being shared (you do not want to share\n> core.worktree and submodule.*). I made some progress last weekend,\n> just needed to add some tests to see if submodule works as expected\n> and will post the series soon. Then you can take over if you want ;)\n\nHere it is. I'm going to describe some more for new people. Let's\nstart with the problem, then the high level solution and finally\nwhat's done in submodule. These are separated by ^----$\n\nMultipl worktrees in its current form share the config file. The only\nexceptions are core.bare and core.worktree which will be applied for\nthe main worktree only, if present in the config file.\n\nThis does not make submodules happy because submodules use\ncore.worktree to link back to the real repos stored inside .git dir of\nthe super modules. This is not so bad right now because the\nsubmodule's worktree would be \"main\" worktree and core.worktree sticks\nto it. If one day \"git submodule add\" initialize the worktree as a\nlinked worktree, problem arises.\n\nThe second problem is more real. When you initialize submodules,\nsubmodule info is stored in the supermodule's config file, which is\nshared. So the secondary worktree will not be able to initialize its\nown submodules (and may be confused by the existing submodule.*\nsection).\n\n----\n\nSo we need to split the config file into two logical parts: a shared\npart and a worktree-specific one. This makes everybody happy even\nthough it's not easy.\n\nWhat this series does is adding \"git config --worktree\" which writes\nto the worktree-specific part, while \"git config\" writes to the shared\npart. \"git config\" as a read operation will read the shared part\nfirst, then the worktree specific part.\n\nNow. In current multiple worktrees setup, both the shared and private\nparts are in the same file, \"config\". And \"git config --worktree\" will\nrefuse to work if you have more than one worktree. For it to work with\nmultiple worktree, you need to enable extensions.worktreeConfig (in\nconfig file).\n\nThis extension designates the file \"config.worktree\" as storage for\nthe private part. It can be .git/config.worktree for main worktree, or\n.git/worktrees/xxx/config.worktree for linked ones.\n\nBefore enabling extensions.worktreeConfig (or soon after it), you need\nto move core.bare and core.worktree to .git/config.worktree because\nthe exceptions above are gone. If they are present in \"config\" file,\nthey are _shared_ (and hell follows)\n\nIf you have followed through the first four iterations, v4 [1] has\nvery close design. The main difference is: in v4, \"config\" is\nper-worktree and the shared part is split away, hidden in\n.git/common/config. This leads to the migration and backward\ncompatibility problems.\n\nThe new design is free of that because \"config\" remains shared while\nthe private is hidden away. The risk is \"git config\" by default writes\nto the shared part. Accidentally sharing something may be more\ndangerous than accidentally _not_ sharing something like v3 [1] (which\ndefaults to per-worktree). I've thought about this and I'm willing to\ntake the new direction, bettting that 90% of the time people want to\nshare, so it's a rare problem.\n\n----\n\nWith all that in place, what does a command have to do to take\nadvantage of it?\n\n- Whenever you need to write a per-worktree config (you decide it!),\n  use \"git config --worktree\". That's it. You don't really need to\n  care where it ends up to. This is what 3/4 is, for submodule.\n\n- Avoid \"git config /path/to/.git/config\" because that may or may not\n  be the right place (there's also config.worktree now). Builtin\n  commands have this worse because if you look at _another_ repo, then\n  you may need to go through repo detection and stuff before you can\n  read its config. I just fall back to running \"git config\" in 2/4.\n\nSo that's it. It seems to be running ok. But I know very little about\nsubmodules to test it properly.\n\nThe only problem left that I have to work out is config deletion. I\nsuppose we could follow the chain backward again: try to delete in\nper-worktree config file first. If not found, try again in the shared\nconfig file...\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/281906/focus=284803\n-- \n2.9.1.566.gbd532d4\n\n"},{"id":"291836","messageId":"20160720172419.25473-2-pclouds@gmail.com","threadId":"42882","inReplyTo":"20160720172419.25473-1-pclouds@gmail.com","subject":"[PATCH v4 1/4] worktree: add per-worktree config files","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-20T17:24:16Z","receivedAt":"2016-07-20T17:25:14Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"A new repo extension is added, worktreeConfig. When it is present:\n\n - Repository config reading by default includes $GIT_DIR/config _and_\n   $GIT_DIR/config.worktree. \"config\" file remains shared in multiple\n   worktree setup.\n\n - The special treatment for core.bare and core.worktree, to stay\n   effective only in main worktree, is gone. These config files are\n   supposed to be in config.worktree.\n\nThis extension is most useful in multiple worktree setup because you\nnow have an option to store per-worktree config (which is either\n.git/config.worktree for main worktree, or\n.git/worktrees/xx/config.worktree for linked ones).\n\nThis extension can be used in single worktree mode, even though it's\npretty much useless (but this can happen after you remove all linked\nworktrees and move back to single worktree).\n\n\"git config\" reads from both \"config\" and \"config.worktree\" by default\n(i.e. without either --user, --file...) when this extension is\npresent. Default writes still go to \"config\", not \"config.worktree\". A\nnew option --worktree is added for that (*).\n\nSince a new repo extension is introduced, existing git binaries should\nrefuse to access to the repo (both from main and linked worktrees). So\nthey will not misread the config file (i.e. skip the config.worktree\npart). They may still accidentally write to the config file anyway if\nthey use with \"git config --file <path>\".\n\nThis design places a bet on the assumption that the majority of config\nvariables are shared so it is the default mode. A safer move would be\ndefault writes go to per-worktree file, so that accidental changes are\nisolated.\n\n(*) \"git config --worktree\" points back to \"config\" file when this\n    extension is not present so that it works in any setup.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n Documentation/config.txt               | 11 ++++-\n Documentation/git-config.txt           | 26 ++++++++----\n Documentation/git-worktree.txt         | 31 ++++++++++++++\n Documentation/gitrepository-layout.txt |  8 ++++\n builtin/config.c                       | 18 +++++++-\n cache.h                                |  2 +\n config.c                               |  7 ++++\n environment.c                          |  1 +\n setup.c                                |  5 ++-\n t/t2028-worktree-config.sh (new +x)    | 77 ++++++++++++++++++++++++++++++++++\n 10 files changed, 175 insertions(+), 11 deletions(-)\n create mode 100755 t/t2028-worktree-config.sh\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 16dc22d..7d64da0 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2,8 +2,9 @@ CONFIGURATION FILE\n ------------------\n \n The Git configuration file contains a number of variables that affect\n-the Git commands' behavior. The `.git/config` file in each repository\n-is used to store the configuration for that repository, and\n+the Git commands' behavior. The files `.git/config` and optionally\n+`config.worktree` (see `extensions.worktreeConfig` below) are each\n+repository is used to store the configuration for that repository, and\n `$HOME/.gitconfig` is used to store a per-user configuration as\n fallback values for the `.git/config` file. The file `/etc/gitconfig`\n can be used to store a system-wide default configuration.\n@@ -264,6 +265,12 @@ advice.*::\n \t\tshow directions on how to proceed from the current state.\n --\n \n+extensions.worktreeConfig::\n+\tIf set, by default \"git config\" reads from both \"config\" and\n+\t\"config.worktree\" file in that order. In multiple working\n+\tdirectory mode, \"config\" file is shared while\n+\t\"config.worktree\" is per-working directory.\n+\n core.fileMode::\n \tTells Git if the executable bit of files in the working tree\n \tis to be honored.\ndiff --git a/Documentation/git-config.txt b/Documentation/git-config.txt\nindex f163113..9dfdb6a 100644\n--- a/Documentation/git-config.txt\n+++ b/Documentation/git-config.txt\n@@ -47,13 +47,15 @@ checks or transformations are performed on the value.\n \n When reading, the values are read from the system, global and\n repository local configuration files by default, and options\n-`--system`, `--global`, `--local` and `--file <filename>` can be\n-used to tell the command to read from only that location (see <<FILES>>).\n+`--system`, `--global`, `--local`, `--worktree` and\n+`--file <filename>` can be used to tell the command to read from only\n+that location (see <<FILES>>).\n \n When writing, the new value is written to the repository local\n configuration file by default, and options `--system`, `--global`,\n-`--file <filename>` can be used to tell the command to write to\n-that location (you can say `--local` but that is the default).\n+`--worktree`, `--file <filename>` can be used to tell the command to\n+write to that location (you can say `--local` but that is the\n+default).\n \n This command will fail with non-zero status upon error.  Some exit\n codes are:\n@@ -133,6 +135,11 @@ from all available files.\n +\n See also <<FILES>>.\n \n+--worktree::\n+\tSimilar to `--local` except that `.git/config.worktree` is\n+\tread from or written to if `extensions.worktreeConfig` is\n+\tpresent. If not it's the same as `--local`.\n+\n -f config-file::\n --file config-file::\n \tUse the given config file instead of the one specified by GIT_CONFIG.\n@@ -253,6 +260,10 @@ $XDG_CONFIG_HOME/git/config::\n $GIT_DIR/config::\n \tRepository specific configuration file.\n \n+$GIT_DIR/config.worktree::\n+\tThis is optional and is only searched when\n+\t`extensions.worktreeConfig` is present in $GIT_DIR/config.\n+\n If no further options are given, all reading options will read all of these\n files that are available. If the global or the system-wide configuration\n file are not available they will be ignored. If the repository configuration\n@@ -268,9 +279,10 @@ configuration file. Note that this also affects options like `--replace-all`\n and `--unset`. *'git config' will only ever change one file at a time*.\n \n You can override these rules either by command-line options or by environment\n-variables. The `--global` and the `--system` options will limit the file used\n-to the global or system-wide file respectively. The `GIT_CONFIG` environment\n-variable has a similar effect, but you can specify any filename you want.\n+variables. The `--global`, `--system` and `--worktree` options will limit\n+the file used to the global, system-wide or per-worktree file respectively.\n+The `GIT_CONFIG` environment variable has a similar effect, but you\n+can specify any filename you want.\n \n \n ENVIRONMENT\ndiff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt\nindex 7c4cfb0..41350db 100644\n--- a/Documentation/git-worktree.txt\n+++ b/Documentation/git-worktree.txt\n@@ -111,6 +111,37 @@ OPTIONS\n --expire <time>::\n \tWith `prune`, only expire unused working trees older than <time>.\n \n+CONFIGURATION FILE\n+------------------\n+By default, the repository \"config\" file is shared across all working\n+directories. If the config variables `core.bare` or `core.worktree`\n+are already present in the config file, they will be applied to the\n+main working directory only.\n+\n+In order to have configuration specific to working directories, you\n+can turn on \"worktreeConfig\" extension, e.g.:\n+\n+------------\n+$ git config extensions.worktreeConfig true\n+------------\n+\n+In this mode, specific configuration stays in the path pointed by `git\n+rev-parse --git-path config.worktree`. You can add or update\n+configuration in this file with `git config --worktree`. Git before\n+version XXX will refuse to access repositories with this extension.\n+\n+Note that in this file, the exception for `core.bare` and\n+core.worktree` is gone. If you have them before, you need to move them\n+to the config.worktree of the main working directory. You may also\n+take this opportunity to move other configuration that you do not want\n+to share to all working directories:\n+\n+ - `core.worktree` and `core.bare` should never be shared\n+\n+ - `core.sparseCheckout` is recommended per working directory, unless\n+   you are sure you always use sparse checkout for all working\n+   directories.\n+\n DETAILS\n -------\n Each linked working tree has a private sub-directory in the repository's\ndiff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt\nindex 577ee84..6cfdb4c 100644\n--- a/Documentation/gitrepository-layout.txt\n+++ b/Documentation/gitrepository-layout.txt\n@@ -143,6 +143,11 @@ config::\n \tif $GIT_COMMON_DIR is set and \"$GIT_COMMON_DIR/config\" will be\n \tused instead.\n \n+config.worktree::\n+\tWorking directory specific configuration file for the main\n+\tworking directory in multiple working directory setup (see\n+\tlinkgit:git-worktree[1]).\n+\n branches::\n \tA slightly deprecated way to store shorthands to be used\n \tto specify a URL to 'git fetch', 'git pull' and 'git push'.\n@@ -276,6 +281,9 @@ worktrees/<id>/link::\n \tfile. It is used to detect if the linked repository is\n \tmanually removed.\n \n+worktrees/<id>/config.worktree::\n+\tWorking directory specific configuration file.\n+\n SEE ALSO\n --------\n linkgit:git-init[1],\ndiff --git a/builtin/config.c b/builtin/config.c\nindex 1d7c6ef..535707c 100644\n--- a/builtin/config.c\n+++ b/builtin/config.c\n@@ -4,6 +4,7 @@\n #include \"parse-options.h\"\n #include \"urlmatch.h\"\n #include \"quote.h\"\n+#include \"worktree.h\"\n \n static const char *const builtin_config_usage[] = {\n \tN_(\"git config [<options>]\"),\n@@ -23,6 +24,7 @@ static char key_delim = ' ';\n static char term = '\\n';\n \n static int use_global_config, use_system_config, use_local_config;\n+static int use_worktree_config;\n static struct git_config_source given_config_source;\n static int actions, types;\n static const char *get_color_slot, *get_colorbool_slot;\n@@ -57,6 +59,7 @@ static struct option builtin_config_options[] = {\n \tOPT_BOOL(0, \"global\", &use_global_config, N_(\"use global config file\")),\n \tOPT_BOOL(0, \"system\", &use_system_config, N_(\"use system config file\")),\n \tOPT_BOOL(0, \"local\", &use_local_config, N_(\"use repository config file\")),\n+\tOPT_BOOL(0, \"worktree\", &use_worktree_config, N_(\"use per-worktree config file\")),\n \tOPT_STRING('f', \"file\", &given_config_source.file, N_(\"file\"), N_(\"use given config file\")),\n \tOPT_STRING(0, \"blob\", &given_config_source.blob, N_(\"blob-id\"), N_(\"read config from given blob object\")),\n \tOPT_GROUP(N_(\"Action\")),\n@@ -491,6 +494,7 @@ int cmd_config(int argc, const char **argv, const char *prefix)\n \t\t\t     PARSE_OPT_STOP_AT_NON_OPTION);\n \n \tif (use_global_config + use_system_config + use_local_config +\n+\t    use_worktree_config +\n \t    !!given_config_source.file + !!given_config_source.blob > 1) {\n \t\terror(\"only one config file at a time.\");\n \t\tusage_with_options(builtin_config_usage, builtin_config_options);\n@@ -525,7 +529,19 @@ int cmd_config(int argc, const char **argv, const char *prefix)\n \t\tgiven_config_source.file = git_etc_gitconfig();\n \telse if (use_local_config)\n \t\tgiven_config_source.file = git_pathdup(\"config\");\n-\telse if (given_config_source.file) {\n+\telse if (use_worktree_config) {\n+\t\tif (repository_format_worktree_config)\n+\t\t\tgiven_config_source.file = git_pathdup(\"config.worktree\");\n+\t\telse {\n+\t\t\tstruct worktree **worktrees = get_worktrees();\n+\t\t\tif (worktrees[0] && worktrees[1])\n+\t\t\t\tdie(_(\"Per-worktree configuration requires extensions.worktreeConfig\\n\"\n+\t\t\t\t      \"Please read section CONFIGURATION in `git help worktree` before\\n\"\n+\t\t\t\t      \"enabling it.\"));\n+\t\t\tfree_worktrees(worktrees);\n+\t\t\tgiven_config_source.file = git_pathdup(\"config\");\n+\t\t}\n+\t} else if (given_config_source.file) {\n \t\tif (!is_absolute_path(given_config_source.file) && prefix)\n \t\t\tgiven_config_source.file =\n \t\t\t\txstrdup(prefix_filename(prefix,\ndiff --git a/cache.h b/cache.h\nindex f1dc289..606500e 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -757,10 +757,12 @@ extern int grafts_replace_parents;\n #define GIT_REPO_VERSION 0\n #define GIT_REPO_VERSION_READ 1\n extern int repository_format_precious_objects;\n+extern int repository_format_worktree_config;\n \n struct repository_format {\n \tint version;\n \tint precious_objects;\n+\tint worktree_config;\n \tint is_bare;\n \tchar *work_tree;\n \tstruct string_list unknown_extensions;\ndiff --git a/config.c b/config.c\nindex bea937e..99ff6be 100644\n--- a/config.c\n+++ b/config.c\n@@ -1254,6 +1254,13 @@ static int do_git_config_sequence(config_fn_t fn, void *data)\n \tif (repo_config && !access_or_die(repo_config, R_OK, 0))\n \t\tret += git_config_from_file(fn, repo_config, data);\n \n+\tif (repository_format_worktree_config) {\n+\t\tchar *path = git_pathdup(\"config.worktree\");\n+\t\tif (!access_or_die(path, R_OK, 0))\n+\t\t\tret += git_config_from_file(fn, path, data);\n+\t\tfree(path);\n+\t}\n+\n \tcurrent_parsing_scope = CONFIG_SCOPE_CMDLINE;\n \tif (git_config_from_parameters(fn, data) < 0)\n \t\tdie(_(\"unable to parse command-line config\"));\ndiff --git a/environment.c b/environment.c\nindex ca72464..b4d56ef 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -26,6 +26,7 @@ int warn_ambiguous_refs = 1;\n int warn_on_object_refname_ambiguity = 1;\n int ref_paranoia = -1;\n int repository_format_precious_objects;\n+int repository_format_worktree_config;\n const char *git_commit_encoding;\n const char *git_log_output_encoding;\n const char *apply_default_whitespace;\ndiff --git a/setup.c b/setup.c\nindex 6d0e0c9..75c784f 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -389,6 +389,8 @@ static int check_repo_format(const char *var, const char *value, void *vdata)\n \t\t\t;\n \t\telse if (!strcmp(ext, \"preciousobjects\"))\n \t\t\tdata->precious_objects = git_config_bool(var, value);\n+\t\telse if (!strcmp(ext, \"worktreeconfig\"))\n+\t\t\tdata->worktree_config = git_config_bool(var, value);\n \t\telse\n \t\t\tstring_list_append(&data->unknown_extensions, ext);\n \t} else if (strcmp(var, \"core.bare\") == 0) {\n@@ -432,8 +434,9 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)\n \t}\n \n \trepository_format_precious_objects = candidate.precious_objects;\n+\trepository_format_worktree_config = candidate.worktree_config;\n \tstring_list_clear(&candidate.unknown_extensions, 0);\n-\tif (!has_common) {\n+\tif (!has_common || repository_format_worktree_config) {\n \t\tif (candidate.is_bare != -1) {\n \t\t\tis_bare_repository_cfg = candidate.is_bare;\n \t\t\tif (is_bare_repository_cfg == 1)\ndiff --git a/t/t2028-worktree-config.sh b/t/t2028-worktree-config.sh\nnew file mode 100755\nindex 0000000..34067df\n--- /dev/null\n+++ b/t/t2028-worktree-config.sh\n@@ -0,0 +1,77 @@\n+#!/bin/sh\n+\n+test_description=\"config file in multi worktree\"\n+\n+. ./test-lib.sh\n+\n+cmp_config() {\n+\tif [ \"$1\" = \"-C\" ]; then\n+\t\tshift &&\n+\t\tGD=\"-C $1\" &&\n+\t\tshift\n+\telse\n+\t\tGD=\n+\tfi &&\n+\techo \"$1\" >expected &&\n+\tshift &&\n+\tgit $GD config \"$@\" >actual &&\n+\ttest_cmp expected actual\n+}\n+\n+test_expect_success 'setup' '\n+\ttest_commit start &&\n+\tgit config --worktree per.worktree is-ok &&\n+\tgit worktree add wt1 &&\n+\tgit worktree add wt2 &&\n+\ttest_must_fail git config --worktree per.worktree is-not-ok &&\n+\tgit config extensions.worktreeConfig true\n+'\n+\n+test_expect_success 'config is shared as before' '\n+\tgit config this.is shared &&\n+\tcmp_config shared this.is &&\n+\tcmp_config -C wt1 shared this.is &&\n+\tcmp_config -C wt2 shared this.is\n+'\n+\n+test_expect_success 'config is shared (set from another worktree)' '\n+\tgit -C wt1 config that.is also-shared &&\n+\tcmp_config also-shared that.is &&\n+\tcmp_config -C wt1 also-shared that.is &&\n+\tcmp_config -C wt2 also-shared that.is\n+'\n+\n+test_expect_success 'config private to main worktree' '\n+\tgit config --worktree this.is for-main &&\n+\tcmp_config for-main this.is &&\n+\tcmp_config -C wt1 shared this.is &&\n+\tcmp_config -C wt2 shared this.is\n+'\n+\n+test_expect_success 'config private to linked worktree' '\n+\tgit -C wt1 config --worktree this.is for-wt1 &&\n+\tcmp_config for-main this.is &&\n+\tcmp_config -C wt1 for-wt1 this.is &&\n+\tcmp_config -C wt2 shared this.is\n+'\n+\n+test_expect_success 'core.bare no longer for main only' '\n+\tgit config core.bare true &&\n+\tcmp_config true core.bare &&\n+\tcmp_config -C wt1 true core.bare &&\n+\tcmp_config -C wt2 true core.bare &&\n+\tgit config --unset core.bare\n+'\n+\n+test_expect_success 'config.worktree no longer read without extension' '\n+\tgit config --unset extensions.worktreeConfig &&\n+\tcmp_config shared this.is &&\n+\tcmp_config -C wt1 shared this.is &&\n+\tcmp_config -C wt2 shared this.is\n+'\n+\n+test_expect_success 'config --worktree fails in multi worktree without extension' '\n+\ttest_must_fail git config --worktree foo.bar true\n+'\n+\n+test_done\n-- \n2.9.1.566.gbd532d4\n\n"},{"id":"291837","messageId":"20160720172419.25473-5-pclouds@gmail.com","threadId":"42882","inReplyTo":"20160720172419.25473-1-pclouds@gmail.com","subject":"[PATCH v4 4/4] t2029: some really basic tests for submodules in multi worktree","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-20T17:24:19Z","receivedAt":"2016-07-20T17:25:21Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n t/t2029-worktree-submodule.sh (new +x) | 166 +++++++++++++++++++++++++++++++++\n 1 file changed, 166 insertions(+)\n create mode 100755 t/t2029-worktree-submodule.sh\n\ndiff --git a/t/t2029-worktree-submodule.sh b/t/t2029-worktree-submodule.sh\nnew file mode 100755\nindex 0000000..f96fa50\n--- /dev/null\n+++ b/t/t2029-worktree-submodule.sh\n@@ -0,0 +1,166 @@\n+#!/bin/sh\n+\n+test_description='submodule with multiple worktrees'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\tgit config extensions.worktreeConfig true &&\n+\t>t &&\n+\tgit add t &&\n+\tgit commit -m initial &&\n+\tgit branch initial\n+'\n+\n+test_expect_success 'setup - repository in init subdirectory' '\n+\tmkdir init &&\n+\t(\n+\t\tcd init &&\n+\t\tgit init &&\n+\t\tgit config extensions.worktreeConfig true &&\n+\t\techo a >a &&\n+\t\tgit add a &&\n+\t\tgit commit -m \"submodule commit 1\" &&\n+\t\tgit tag -a -m \"rev-1\" rev-1\n+\t)\n+'\n+\n+test_expect_success 'setup - commit with gitlink' '\n+\techo a >a &&\n+\techo z >z &&\n+\tgit add a init z &&\n+\tgit commit -m \"super commit 1\"\n+'\n+\n+test_expect_success 'setup - hide init subdirectory' '\n+\tmv init .subrepo\n+'\n+\n+test_expect_success 'setup - repository to add submodules to' '\n+\tgit init addtest &&\n+\tgit -C addtest config extensions.worktreeConfig true &&\n+\tgit init addtest-ignore &&\n+\tgit -C addtest-ignore config extensions.worktreeConfig true\n+'\n+\n+# The 'submodule add' tests need some repository to add as a submodule.\n+# The trash directory is a good one as any. We need to canonicalize\n+# the name, though, as some tests compare it to the absolute path git\n+# generates, which will expand symbolic links.\n+submodurl=$(pwd -P)\n+\n+listbranches() {\n+\tgit for-each-ref --format='%(refname)' 'refs/heads/*'\n+}\n+\n+inspect() {\n+\tdir=$1 &&\n+\tdotdot=\"${2:-..}\" &&\n+\n+\t(\n+\t\tcd \"$dir\" &&\n+\t\tlistbranches >\"$dotdot/heads\" &&\n+\t\t{ git symbolic-ref HEAD || :; } >\"$dotdot/head\" &&\n+\t\tgit rev-parse HEAD >\"$dotdot/head-sha1\" &&\n+\t\tgit update-index --refresh &&\n+\t\tgit diff-files --exit-code &&\n+\t\tgit clean -n -d -x >\"$dotdot/untracked\"\n+\t)\n+}\n+\n+test_expect_success 'submodule add' '\n+\techo \"refs/heads/master\" >expect &&\n+\t>empty &&\n+\n+\t(\n+\t\tcd addtest &&\n+\t\tgit submodule add -q \"$submodurl\" submod >actual &&\n+\t\ttest_must_be_empty actual &&\n+\t\techo \"gitdir: ../.git/modules/submod\" >expect &&\n+\t\ttest_cmp expect submod/.git &&\n+\t\t(\n+\t\t\tcd submod &&\n+\t\t\tgit config core.worktree >actual &&\n+\t\t\techo \"../../../submod\" >expect &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\trm -f actual expect\n+\t\t) &&\n+\t\tgit submodule init\n+\t) &&\n+\n+\trm -f heads head untracked &&\n+\tinspect addtest/submod ../.. &&\n+\ttest_cmp expect heads &&\n+\ttest_cmp expect head &&\n+\ttest_cmp empty untracked\n+'\n+\n+test_expect_success 'submodule.* in supermodule is per-worktree' '\n+\t(\n+\t\tcd addtest &&\n+\t\tgit config -f .git/config.worktree submodule.submod.url >actual &&\n+\t\techo \"$submodurl\" >expect &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success 'turn submodule to multiworktree' '\n+\t(\n+\t\tcd addtest/.git/modules/submod &&\n+\t\tCORE_WT=\"$(git config core.worktree)\" &&\n+\t\tgit config -f config.worktree core.worktree \"$CORE_WT\" &&\n+\t\tgit config --unset core.worktree &&\n+\t\tgit config extensions.worktreeConfig true &&\n+\t\tgit config core.worktree >actual &&\n+\t\techo \"$CORE_WT\" >expect &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success 'new worktree in submodule' '\n+\t(\n+\t\tcd addtest/submod &&\n+\t\tgit worktree add submod-elsewhere &&\n+\t\tcd submod-elsewhere &&\n+\t\ttest_must_fail git config core.worktree\n+\t)\n+'\n+\n+test_expect_success 'new worktree in supermodule' '\n+\t(\n+\t\tcd addtest &&\n+\t\tgit commit -m initial &&\n+\t\tgit worktree add super-elsewhere &&\n+\t\tcd super-elsewhere &&\n+\t\ttest_must_fail git config submodule.submode\n+\t)\n+'\n+\n+test_expect_success 'submodule add in the second worktree' '\n+\t(\n+\t\tcd addtest/super-elsewhere &&\n+\t\tgit submodule add -q \"$submodurl\" submod2 >actual &&\n+\t\ttest_must_be_empty actual &&\n+\t\techo \"gitdir: ../../.git/worktrees/super-elsewhere/modules/submod2\" >expect &&\n+\t\ttest_cmp expect submod2/.git &&\n+\t\t(\n+\t\t\tcd submod2 &&\n+\t\t\tgit config core.worktree >actual &&\n+\t\t\techo \"../../../../../super-elsewhere/submod2\" >expect &&\n+\t\t\ttest_cmp expect actual &&\n+\t\t\trm -f actual expect\n+\t\t) &&\n+\t\tgit submodule init\n+\t)\n+'\n+\n+test_expect_success 'submodule.* in supermodule is per-worktree' '\n+\t(\n+\t\tcd addtest/super-elsewhere &&\n+\t\tgit config -f ../.git/worktrees/super-elsewhere/config.worktree submodule.submod2.url >actual &&\n+\t\techo \"$submodurl\" >expect &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_done\n-- \n2.9.1.566.gbd532d4\n\n"},{"id":"291838","messageId":"20160720172419.25473-4-pclouds@gmail.com","threadId":"42882","inReplyTo":"20160720172419.25473-1-pclouds@gmail.com","subject":"[PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-20T17:24:18Z","receivedAt":"2016-07-20T17:25:24Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n Documentation/git-worktree.txt | 8 ++++++++\n git-submodule.sh               | 8 ++++----\n 2 files changed, 12 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt\nindex 41350db..2a5661d 100644\n--- a/Documentation/git-worktree.txt\n+++ b/Documentation/git-worktree.txt\n@@ -142,6 +142,14 @@ to share to all working directories:\n    you are sure you always use sparse checkout for all working\n    directories.\n \n+ - `submodule.*` in current state should not be shared because the\n+   information is tied to a particular version of .gitmodules in a\n+   working directory.\n+\n+ - `remote.*` added by submodules may be per working directory as\n+   well, unless you are sure remotes from all possible submodules in\n+   history are consistent.\n+\n DETAILS\n -------\n Each linked working tree has a private sub-directory in the repository's\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 4ec7546..7b576f5 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -261,7 +261,7 @@ or you are unsure what this means choose another name with the '--name' option.\"\n \t\t\tesac\n \t\t) || die \"$(eval_gettext \"Unable to checkout submodule '\\$sm_path'\")\"\n \tfi\n-\tgit config submodule.\"$sm_name\".url \"$realrepo\"\n+\tgit config --worktree submodule.\"$sm_name\".url \"$realrepo\"\n \n \tgit add $force \"$sm_path\" ||\n \tdie \"$(eval_gettext \"Failed to add submodule '\\$sm_path'\")\"\n@@ -461,7 +461,7 @@ Submodule work tree '\\$displaypath' contains a .git directory\n \t\t\t# Remove the whole section so we have a clean state when\n \t\t\t# the user later decides to init this submodule again\n \t\t\turl=$(git config submodule.\"$name\".url)\n-\t\t\tgit config --remove-section submodule.\"$name\" 2>/dev/null &&\n+\t\t\tgit config --worktree --remove-section submodule.\"$name\" 2>/dev/null &&\n \t\t\tsay \"$(eval_gettext \"Submodule '\\$name' (\\$url) unregistered for path '\\$displaypath'\")\"\n \t\tfi\n \tdone\n@@ -1106,7 +1106,7 @@ cmd_sync()\n \t\tthen\n \t\t\tdisplaypath=$(git submodule--helper relative-path \"$prefix$sm_path\" \"$wt_prefix\")\n \t\t\tsay \"$(eval_gettext \"Synchronizing submodule url for '\\$displaypath'\")\"\n-\t\t\tgit config submodule.\"$name\".url \"$super_config_url\"\n+\t\t\tgit config --worktree submodule.\"$name\".url \"$super_config_url\"\n \n \t\t\tif test -e \"$sm_path\"/.git\n \t\t\tthen\n@@ -1114,7 +1114,7 @@ cmd_sync()\n \t\t\t\tsanitize_submodule_env\n \t\t\t\tcd \"$sm_path\"\n \t\t\t\tremote=$(get_default_remote)\n-\t\t\t\tgit config remote.\"$remote\".url \"$sub_origin_url\"\n+\t\t\t\tgit config --worktree remote.\"$remote\".url \"$sub_origin_url\"\n \n \t\t\t\tif test -n \"$recursive\"\n \t\t\t\tthen\n-- \n2.9.1.566.gbd532d4\n\n"},{"id":"291839","messageId":"20160720172419.25473-3-pclouds@gmail.com","threadId":"42882","inReplyTo":"20160720172419.25473-1-pclouds@gmail.com","subject":"[PATCH v4 2/4] submodule: update core.worktree using git-config","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-20T17:24:17Z","receivedAt":"2016-07-20T17:25:25Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"To access a separate repository, the first step should be read its\nconfig file to determine if this repository layout is supported or\nnot, or if we understand all repo extensions, of it is a linked\nworktree. Only then should know where to update the config file.\n\nUnfortunately, our C code base is not ready for doing all that in the\nsame process. The repo detection is not meant to be used for peeking\nin other repository, and config code would read config.worktree that\nis in _current_ $GIT_DIR.\n\nFor now, let's spawn a new process and let all that done separately.\n\nPS. submodule-helper also updates core.worktree. But in there, we\ncreate a new clone, we know what is the initial repository layout, so\nwe know we can simply update \"config\" file without risks.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n---\n submodule.c | 16 +++++++++++-----\n 1 file changed, 11 insertions(+), 5 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex abc2ac2..b912871 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1128,7 +1128,9 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)\n {\n \tstruct strbuf file_name = STRBUF_INIT;\n \tstruct strbuf rel_path = STRBUF_INIT;\n+\tstruct strbuf path = STRBUF_INIT;\n \tconst char *real_work_tree = xstrdup(real_path(work_tree));\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n \n \t/* Update gitfile */\n \tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n@@ -1136,13 +1138,17 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)\n \t\t   relative_path(git_dir, real_work_tree, &rel_path));\n \n \t/* Update core.worktree setting */\n-\tstrbuf_reset(&file_name);\n-\tstrbuf_addf(&file_name, \"%s/config\", git_dir);\n-\tgit_config_set_in_file(file_name.buf, \"core.worktree\",\n-\t\t\t       relative_path(real_work_tree, git_dir,\n-\t\t\t\t\t     &rel_path));\n+\tstrbuf_addstr(&path, relative_path(real_work_tree, git_dir,\n+\t\t\t\t\t   &rel_path));\n+\tcp.git_cmd = 1;\n+\targv_array_pushl(&cp.args, \"-C\", work_tree, NULL);\n+\targv_array_pushl(&cp.args, \"--work-tree\", \".\", NULL);\n+\targv_array_pushl(&cp.args, \"config\", \"core.worktree\", path.buf, NULL);\n+\tif (run_command(&cp) < 0)\n+\t\tdie(_(\"failed to update core.worktree for %s\"), git_dir);\n \n \tstrbuf_release(&file_name);\n+\tstrbuf_release(&path);\n \tstrbuf_release(&rel_path);\n \tfree((void *)real_work_tree);\n }\n-- \n2.9.1.566.gbd532d4\n\n"},{"id":"291885","messageId":"CAGZ79kZg-E8p1WW8j5ghOC=EJU++Dy++esv=vVRt8iuOYsrNpQ@mail.gmail.com","threadId":"42882","inReplyTo":"20160720172419.25473-3-pclouds@gmail.com","subject":"Re: [PATCH v4 2/4] submodule: update core.worktree using git-config","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-20T22:04:48Z","receivedAt":"2016-07-20T22:04:54Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jul 20, 2016 at 10:24 AM, Nguyễn Thái Ngọc Duy\n<pclouds@gmail.com> wrote:\n> To access a separate repository, the first step should be read its\n> config file to determine if this repository layout is supported or\n> not, or if we understand all repo extensions, of it is a linked\n> worktree. Only then should know where to update the config file.\n>\n> Unfortunately, our C code base is not ready for doing all that in the\n> same process. The repo detection is not meant to be used for peeking\n> in other repository, and config code would read config.worktree that\n> is in _current_ $GIT_DIR.\n>\n> For now, let's spawn a new process and let all that done separately.\n>\n> PS. submodule-helper also updates core.worktree. But in there, we\n> create a new clone, we know what is the initial repository layout, so\n> we know we can simply update \"config\" file without risks.\n\nRight, the submodule--helper update_clone should not be required to convert.\n\n>\n> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n> ---\n>  submodule.c | 16 +++++++++++-----\n>  1 file changed, 11 insertions(+), 5 deletions(-)\n>\n> diff --git a/submodule.c b/submodule.c\n> index abc2ac2..b912871 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -1128,7 +1128,9 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)\n>  {\n>         struct strbuf file_name = STRBUF_INIT;\n>         struct strbuf rel_path = STRBUF_INIT;\n> +       struct strbuf path = STRBUF_INIT;\n>         const char *real_work_tree = xstrdup(real_path(work_tree));\n> +       struct child_process cp = CHILD_PROCESS_INIT;\n>\n>         /* Update gitfile */\n>         strbuf_addf(&file_name, \"%s/.git\", work_tree);\n> @@ -1136,13 +1138,17 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)\n>                    relative_path(git_dir, real_work_tree, &rel_path));\n>\n>         /* Update core.worktree setting */\n> -       strbuf_reset(&file_name);\n> -       strbuf_addf(&file_name, \"%s/config\", git_dir);\n> -       git_config_set_in_file(file_name.buf, \"core.worktree\",\n> -                              relative_path(real_work_tree, git_dir,\n> -                                            &rel_path));\n> +       strbuf_addstr(&path, relative_path(real_work_tree, git_dir,\n> +                                          &rel_path));\n> +       cp.git_cmd = 1;\n> +       argv_array_pushl(&cp.args, \"-C\", work_tree, NULL);\n> +       argv_array_pushl(&cp.args, \"--work-tree\", \".\", NULL);\n> +       argv_array_pushl(&cp.args, \"config\", \"core.worktree\", path.buf, NULL);\n> +       if (run_command(&cp) < 0)\n> +               die(_(\"failed to update core.worktree for %s\"), git_dir);\n\nDo we need to make this conditional on the extensions.worktreeConfig\nvariable, though? When I just run\n\n    git config --worktree . foo bar\nfatal: Per-worktree configuration requires extensions.worktreeConfig\nPlease read section CONFIGURATION in `git help worktree` before\nenabling it.\n\nwhich would trigger the failure here?\n\n>\n>         strbuf_release(&file_name);\n> +       strbuf_release(&path);\n>         strbuf_release(&rel_path);\n>         free((void *)real_work_tree);\n>  }\n> --\n> 2.9.1.566.gbd532d4\n>\n"},{"id":"291894","messageId":"CAGZ79kZB8U+ERNeYpZ-i7Ldip7xbz0ND53g4bzMkzFC3pnyv+w@mail.gmail.com","threadId":"42882","inReplyTo":"20160720172419.25473-4-pclouds@gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-20T23:22:29Z","receivedAt":"2016-07-20T23:22:36Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jul 20, 2016 at 10:24 AM, Nguyễn Thái Ngọc Duy\n<pclouds@gmail.com> wrote:\n> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n> ---\n>  Documentation/git-worktree.txt | 8 ++++++++\n>  git-submodule.sh               | 8 ++++----\n>  2 files changed, 12 insertions(+), 4 deletions(-)\n>\n> diff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt\n> index 41350db..2a5661d 100644\n> --- a/Documentation/git-worktree.txt\n> +++ b/Documentation/git-worktree.txt\n> @@ -142,6 +142,14 @@ to share to all working directories:\n>     you are sure you always use sparse checkout for all working\n>     directories.\n>\n> + - `submodule.*` in current state should not be shared because the\n> +   information is tied to a particular version of .gitmodules in a\n> +   working directory.\n\nWhile the submodule.* settings are copied from the .gitmodules file initially,\nthey can be changed in the config later. (That was actually the whole\npoint of it,\nso you can change the submodule remotes URL without having to change history.)\n\nAnd I would think that most submodule related settings (such as remote URL,\nname, path, even depth recommendation) should be the same for all worktrees,\nand a different value for one worktree is a carefully crafted\nexception by the user.\n\nSo while the .gitmodules file can diverge in the work trees I do not\nthink that the\nactual remotes for the submodules in the different worktrees differ\nthough. The change\nof the .gitmodule files may be because you checked out an old commit, that\nhas outdated information on where to get the submodule from.\n\n> +\n> + - `remote.*` added by submodules may be per working directory as\n> +   well, unless you are sure remotes from all possible submodules in\n> +   history are consistent.\n> +\n>  DETAILS\n>  -------\n>  Each linked working tree has a private sub-directory in the repository's\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 4ec7546..7b576f5 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -261,7 +261,7 @@ or you are unsure what this means choose another name with the '--name' option.\"\n>                         esac\n>                 ) || die \"$(eval_gettext \"Unable to checkout submodule '\\$sm_path'\")\"\n>         fi\n> -       git config submodule.\"$sm_name\".url \"$realrepo\"\n> +       git config --worktree submodule.\"$sm_name\".url \"$realrepo\"\n\nThis is in cmd_add. Actually I think this should be --not-worktree\n(i.e. --local)\nas when you add a submodule in one worktree, and then in another,\nyou may want to have the same URL. However if another worktree\nalready configured it you want to keep the option.\nso rather:\n\n  if git config  submodule.\"$sm_name\".url then\n      # it exists, do nothing\n  else\n    # it does not exist\n    git config --local ...\n\n>\n>         git add $force \"$sm_path\" ||\n>         die \"$(eval_gettext \"Failed to add submodule '\\$sm_path'\")\"\n> @@ -461,7 +461,7 @@ Submodule work tree '\\$displaypath' contains a .git directory\n>                         # Remove the whole section so we have a clean state when\n>                         # the user later decides to init this submodule again\n>                         url=$(git config submodule.\"$name\".url)\n> -                       git config --remove-section submodule.\"$name\" 2>/dev/null &&\n> +                       git config --worktree --remove-section submodule.\"$name\" 2>/dev/null &&\n>                         say \"$(eval_gettext \"Submodule '\\$name' (\\$url) unregistered for path '\\$displaypath'\")\"\n\nThis is in cmd_deinit, which is documented as:\n           Unregister the given submodules, i.e. remove the whole\n           submodule.$name section from .git/config together with their work\n           tree. Further calls to git submodule update, git submodule foreach\n           and git submodule sync will skip any unregistered submodules until\n           they are initialized again, so use this command if you don’t want\n           to have a local checkout of the submodule in your work tree\n           anymore. If you really want to remove a submodule from the\n           repository and commit that use git-rm(1) instead.\n\nSo one might wonder if the unregister should be a global unregister\nor a worktree specific unregister.\n\nFrom a users POV there are:\n* non existent submodules (no gitlink recorded, no config set,\n  no repo in place)\n* not initialized submodules (gitlink is recorded, no config set,\n  and an empty repo is put in the working tree as a place holder).\n* initialized submodules (gitlink is recorded, the config\n  submodule .<name>.url is copied from the .gitmodules file to .git/config.\n  an empty dir in the working tree as a place holder)\n  A user may change the configuration before the next step as the url in\n  the .gitmodules file may be wrong and the user doesn't want to\n  rewrite history\n* existing submodules (gitlink is recorded, the config option is set\n  and instead of an empty placeholder dir, we actually have a git\n  repo there.)\n* matching submodules (the recorded git link matches\n  the actual checked out state of the repo!, config option and repo exist)\n\nI made up these terms for these 5 states and they don't appear in any\ndocumentation, but I think that is the exhaustive list of what a submodule\nshould be, when using the git-submodule command properly.\n\nThere can be more things though. As we have three indicators (existence of\ngitlink, config option and repo), we can have up to 8 states, I left some out in\nthe above listing.\n\n* gitlink recorded, no config set, repo is there and maybe even matches the\n  recorded git link.\n   What a strange thing! git treats that as a \"not initialized\" from above.\n* no gitlink exists, no config exists, but a repo exists:\n  That's just an embedded repo, never touch it!\n* no gitlink, no repo, but a config exists: just a stray old config\nlaying around,\n  ignore it\n* no gitlink, but a config and a repo exists: \"A deleted submodule\", use\n  rm -rf to purge it.\n\n--------\nThe above was a wall of text to make myself aware of the pitfalls of submodules.\n---\n\nA discussion with Jonathan Nieder in office ensued and we came to the following\nconclusions:\n* Anything except the \"existence in the working tree\" shall be shared,\ni.e. is a repository\n  specific, not working tree specific setting.\n\nCurrently we use the submodule.\"<name>\".url config option to determine\nthe existence of\na submodule (init vs non init; if init -> git submodule update will\n(maybe fetch) and checkout\naccordingly).\n\nAs an intermediate step forward we could do:\n* introduce a submodule.\"<name>\".existsInWorktree = [true/false]\nsetting, that decouples\n  the check for existence from the url being present. That is the only\noption per worktree, all\n  other submodule.\"<name>\".* options are shared if not overwritten\nmanually with care\n\n* another approach would be to have a\nsubmodule.includeInWorktree=<pathspec> option\n  which is similar, but has slightly different naming.\n\nLooking back at origin/sb/submodule-default-path (which is in pu for a\nlong time now),\nwhich allowed a configuration submodule.defaultUpdatePath <pathspec>,\nthat could be\nused with `git submodule update --init-default-path`.\n\n----\nSo maybe we want to drop that series and first talk about a migration plan from\nthe current state to a world where we have the existence depending not\non the url\nparameter, but a boolean variable submodule.<name>.<good_name>.\nDepending on <good_name> a submodule would be ignored or tried to checkout\nin e.g. `submodule update`\n\n---\nIf we want to move the current behavior of submodules forward, we\nwould want to have\nanything but the url as shared variables and then use the url variable\nas a per-worktree\nexistence flag.\n\nThanks,\nStefan\n"},{"id":"291939","messageId":"CAGZ79kZqHpnCKbcjqPUUtc2jS_HM4FAkyprd3L+y8-z_eVa7cg@mail.gmail.com","threadId":"42882","inReplyTo":"CAGZ79kZB8U+ERNeYpZ-i7Ldip7xbz0ND53g4bzMkzFC3pnyv+w@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-22T00:37:37Z","receivedAt":"2016-07-22T00:37:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"FYI: I started working on a series that decouples existence of a\nsubmodule from the URL\nas a preparatory series to this one. Then we can have the same URL in\nall working trees, but\nthe existence is configured differently for each working tree.\n\nI'll try to send it out tomorrow.\n\nThanks,\nStefan\n"},{"id":"291941","messageId":"64e9e8fc-50b3-98d8-fca8-6a70028c6398@web.de","threadId":"42882","inReplyTo":"CAGZ79kZB8U+ERNeYpZ-i7Ldip7xbz0ND53g4bzMkzFC3pnyv+w@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2016-07-22T07:32:20Z","receivedAt":"2016-07-22T07:32:56Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 21.07.2016 um 01:22 schrieb Stefan Beller:\n> So maybe we want to drop that series and first talk about a migration plan from\n> the current state to a world where we have the existence depending not\n> on the url\n> parameter, but a boolean variable submodule.<name>.<good_name>.\n> Depending on <good_name> a submodule would be ignored or tried to checkout\n> in e.g. `submodule update`\n\nWhoa, that's a very intrusive change with a ton of compatibility\nproblems waiting to happen. Maybe its simpler to make \"git submodule\nsync\" aware of worktrees and error out with an \"you cannot use\nsubmodules with different URLs in a worktree scenario\" error when\nthe URL is going to change? That should make most use cases work\nwhile avoiding the problematic ones.\n\n> If we want to move the current behavior of submodules forward, we\n> would want to have\n> anything but the url as shared variables and then use the url variable\n> as a per-worktree\n> existence flag.\n\nWithout having though deeply about all submodule variables, I see\nthem as worktree specific. E.g. \"update=none\" is used on our CI-\nServer to avoid the disk space cost on some checkouts of a certain\nsuperproject while using \"update=checkout\" on others where their\ncontent is needed.\n"},{"id":"291969","messageId":"CAGZ79kbMbW9Aex92cFj0oVWMBC0F2z9JDm9QdAO4BQPSSMhDNg@mail.gmail.com","threadId":"42882","inReplyTo":"64e9e8fc-50b3-98d8-fca8-6a70028c6398@web.de","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-22T16:07:12Z","receivedAt":"2016-07-22T16:07:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jul 22, 2016 at 12:32 AM, Jens Lehmann <Jens.Lehmann@web.de> wrote:\n> Am 21.07.2016 um 01:22 schrieb Stefan Beller:\n>>\n>> So maybe we want to drop that series and first talk about a migration plan\n>> from\n>> the current state to a world where we have the existence depending not\n>> on the url\n>> parameter, but a boolean variable submodule.<name>.<good_name>.\n>> Depending on <good_name> a submodule would be ignored or tried to checkout\n>> in e.g. `submodule update`\n>\n>\n> Whoa, that's a very intrusive change with a ton of compatibility\n> problems waiting to happen. Maybe its simpler to make \"git submodule\n> sync\" aware of worktrees and error out with an \"you cannot use\n> submodules with different URLs in a worktree scenario\" error when\n> the URL is going to change? That should make most use cases work\n> while avoiding the problematic ones.\n\nI think fixing sync alone is just a drop of water on the oven.\nActually I can think of scenarios that have different URLs for different\nworktrees (think of the automatic CI thing that should only fetch from\nan internal server, whereas the dev-checkout fetches from upstream)\nActually each config variable (including the update strategy as you\nmention below, but also the depth, branch, path) may be different in\none work tree.\n\nI do not want to forbid the existence of different settings (URLs)\nper worktree. Rather I think a different setting is a user decision,\nhence they will want to run \"git config --worktree ...\"\n\nAnd one of the unfortunate things is the coupling of existence of a\nsubmodule and the URL. If that were to be decoupled, you could do\na \"git config --worktree submodule.<name>.exists true\" (or it is wrapped\nfancily in \"git submodule init\") and the URL would not have to be copied\nfrom the .gitmodules file.\n\nI agree that this is a breaking change, which is why I'd guard it with\na config option such that the user can make the choice if they want\nto go with the old behavior or the new behavior.\n\n\n>\n>> If we want to move the current behavior of submodules forward, we\n>> would want to have\n>> anything but the url as shared variables and then use the url variable\n>> as a per-worktree\n>> existence flag.\n>\n>\n> Without having though deeply about all submodule variables, I see\n> them as worktree specific. E.g. \"update=none\" is used on our CI-\n> Server to avoid the disk space cost on some checkouts of a certain\n> superproject while using \"update=checkout\" on others where their\n> content is needed.\n\nBut this is a conscious user choice, so you would have configured\nthat on a per-worktree basis anyway?\ni.e. it seems to me as if \"update=checkout\" is a default that is good\nfor all but one worktree, so why would you want to configure that n times\ninstead of just once as default?\nThe non default behavior is then overwritten in the specific worktree.\n"},{"id":"291976","messageId":"xmqqmvl9boju.fsf@gitster.mtv.corp.google.com","threadId":"42882","inReplyTo":"CAGZ79kZB8U+ERNeYpZ-i7Ldip7xbz0ND53g4bzMkzFC3pnyv+w@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-22T16:55:17Z","receivedAt":"2016-07-22T16:55:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> From a users POV there are:\n> * non existent submodules (no gitlink recorded, no config set,\n>   no repo in place)\n> * not initialized submodules (gitlink is recorded, no config set,\n>   and an empty repo is put in the working tree as a place holder).\n\nThis is no different from what you later call \"embedded\".  The only\ndifference is that embedded thing hasn't seen its initial commit.\n\n> * initialized submodules (gitlink is recorded, the config\n>   submodule .<name>.url is copied from the .gitmodules file to .git/config.\n>   an empty dir in the working tree as a place holder)\n>   A user may change the configuration before the next step as the url in\n>   the .gitmodules file may be wrong and the user doesn't want to\n>   rewrite history\n\ni.e. what \"submodule init\" gives you.\n\n> * existing submodules (gitlink is recorded, the config option is set\n>   and instead of an empty placeholder dir, we actually have a git\n>   repo there.)\n\ni.e. what \"submodule update\" after \"submodule init\" gives you.\n\n> * matching submodules (the recorded git link matches\n>   the actual checked out state of the repo!, config option and repo exist)\n\nIs this any different from \"existing\" case for the purpose of\ndiscussing the interaction between a submodule (and its checkout)\nand having possibly multiple worktrees of its superproject?\n\nI agree that when a top-level superproject has multiple worktrees\nthese multiple worktrees may want to have the same submodule in\ndifferent states, but I'd imagine that they want to share the same\nphysical repository (i.e. $GIT_DIR/modules/$name of the primary\nworktree of the superproject)---is everybody involved in the\ndiscussion share this assumption?\n\nAssuming that everybody is on the same page, that means \"do we have\nthe repository for that submodule, and if so where in our local\nfilesystem?\" is a bit of information shared across the worktrees of\nthe superproject.  And the \"name\" used to identify the submodule is\nalso shared across these worktrees of the superproject, as it is\nmeant to be a unique (within the superproject) identifier for that\n\"other\" project it uses, no matter where in the superproject's\nworking tree (note: this is \"working tree\", not \"worktree\") it would\nbe checked out, and where the upstream URL to get further updates to\nthe submodule is (i.e. that URL may change over time if they relocate,\nor it may even change when the user who works on the superproject\ndecides to use a different mirror).\n\nWhat can be different between the instantiation of the same\nsubmodule in these multiple worktrees, and how they should be\nrecorded?\n\n * submodule.$name.URL?  I am not sure if we want to have different\n   \"upstreams\" depending on the worktree of the superproject.  While\n   there is no fundamental reason to forbid it, having to maintain a\n   single local repository for a submodule would mean they would\n   need to be treated as separate \"remotes\" in the submodule\n   repository.\n\n * submodule.$name.path of course can be different depending on\n   which commit of the superproject is checked out in the worktree,\n   as the superproject may move the submodule binding site across\n   its versions.\n\n * submodule.$name.update, submodule.$name.ignore,\n   submodule.$name.branch, etc. would need to be all different among\n   worktrees of the superproject, as that is the whole point of\n   being able to work on separate branches of the superproject in\n   separate worktrees.\n\nSomewhere in this discussion thread, you present the conclusion of\nyour discussion with Jonathan Nieder that there needs a separate\n\"should the submodule directory be populated?\" bit, which currently\nis tied to submodule.$name.URL in $GIT_DIR/config.  I tend to agree\nthat knowing where you get other people's update of that submodule\nrepository should come from and wanting to have/keep a checkout of\nthat submodule in the working tree of a particular worktree are two\ndifferent things, so such a separate bit would be needed, and that\nwould belong to per-worktree configuration.\n\n\n"},{"id":"291978","messageId":"CACsJy8CSnmnzDMGpMvvkhWRfJvp1L+pfOZ=eYp5JF0GWNH6D0Q@mail.gmail.com","threadId":"42882","inReplyTo":"CAGZ79kZB8U+ERNeYpZ-i7Ldip7xbz0ND53g4bzMkzFC3pnyv+w@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-22T17:09:01Z","receivedAt":"2016-07-22T17:09:36Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Jul 21, 2016 at 1:22 AM, Stefan Beller <sbeller@google.com> wrote:\n> On Wed, Jul 20, 2016 at 10:24 AM, Nguyễn Thái Ngọc Duy\n> <pclouds@gmail.com> wrote:\n>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n>> ---\n>>  Documentation/git-worktree.txt | 8 ++++++++\n>>  git-submodule.sh               | 8 ++++----\n>>  2 files changed, 12 insertions(+), 4 deletions(-)\n>>\n>> diff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt\n>> index 41350db..2a5661d 100644\n>> --- a/Documentation/git-worktree.txt\n>> +++ b/Documentation/git-worktree.txt\n>> @@ -142,6 +142,14 @@ to share to all working directories:\n>>     you are sure you always use sparse checkout for all working\n>>     directories.\n>>\n>> + - `submodule.*` in current state should not be shared because the\n>> +   information is tied to a particular version of .gitmodules in a\n>> +   working directory.\n>\n> While the submodule.* settings are copied from the .gitmodules file initially,\n> they can be changed in the config later. (That was actually the whole\n> point of it,\n> so you can change the submodule remotes URL without having to change history.)\n>\n> And I would think that most submodule related settings (such as remote URL,\n> name, path, even depth recommendation) should be the same for all worktrees,\n> and a different value for one worktree is a carefully crafted\n> exception by the user.\n>\n> So while the .gitmodules file can diverge in the work trees I do not\n> think that the\n> actual remotes for the submodules in the different worktrees differ\n> though. The change\n> of the .gitmodule files may be because you checked out an old commit, that\n> has outdated information on where to get the submodule from.\n\nI just quickly glanced through the rest of this mail because, as a\nsubmodule ignorant, it's just mumbo jumbo to me. But what I see here\nis, there may be problems if we choose to share some submodule info,\nbut I haven't seen any good thing from sharing any submodule info at\nall.\n\nI can imagine long term you may want to just clone a submodule repo\nonce and share it across worktrees that use it, so maybe it's just me\nnot seeing things and this may be a step towards that.\n\nAnyway, I assume some people will be working on the submodule side.\nAnd because I have not heard any bad thing about the new config\ndesign, I'm going to drop submodule patches from this series and focus\non polishing config stuff.\n-- \nDuy\n"},{"id":"291981","messageId":"CACsJy8Dyw4DefzPj2oy0SYZZC0TTjVCD+p5Ued265VogG_eNSw@mail.gmail.com","threadId":"42882","inReplyTo":"CAGZ79kZg-E8p1WW8j5ghOC=EJU++Dy++esv=vVRt8iuOYsrNpQ@mail.gmail.com","subject":"Re: [PATCH v4 2/4] submodule: update core.worktree using git-config","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-22T17:15:39Z","receivedAt":"2016-07-22T17:16:14Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Jul 21, 2016 at 12:04 AM, Stefan Beller <sbeller@google.com> wrote:\n>> diff --git a/submodule.c b/submodule.c\n>> index abc2ac2..b912871 100644\n>> --- a/submodule.c\n>> +++ b/submodule.c\n>> @@ -1128,7 +1128,9 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)\n>>  {\n>>         struct strbuf file_name = STRBUF_INIT;\n>>         struct strbuf rel_path = STRBUF_INIT;\n>> +       struct strbuf path = STRBUF_INIT;\n>>         const char *real_work_tree = xstrdup(real_path(work_tree));\n>> +       struct child_process cp = CHILD_PROCESS_INIT;\n>>\n>>         /* Update gitfile */\n>>         strbuf_addf(&file_name, \"%s/.git\", work_tree);\n>> @@ -1136,13 +1138,17 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)\n>>                    relative_path(git_dir, real_work_tree, &rel_path));\n>>\n>>         /* Update core.worktree setting */\n>> -       strbuf_reset(&file_name);\n>> -       strbuf_addf(&file_name, \"%s/config\", git_dir);\n>> -       git_config_set_in_file(file_name.buf, \"core.worktree\",\n>> -                              relative_path(real_work_tree, git_dir,\n>> -                                            &rel_path));\n>> +       strbuf_addstr(&path, relative_path(real_work_tree, git_dir,\n>> +                                          &rel_path));\n>> +       cp.git_cmd = 1;\n>> +       argv_array_pushl(&cp.args, \"-C\", work_tree, NULL);\n>> +       argv_array_pushl(&cp.args, \"--work-tree\", \".\", NULL);\n>> +       argv_array_pushl(&cp.args, \"config\", \"core.worktree\", path.buf, NULL);\n>> +       if (run_command(&cp) < 0)\n>> +               die(_(\"failed to update core.worktree for %s\"), git_dir);\n>\n> Do we need to make this conditional on the extensions.worktreeConfig\n> variable, though? When I just run\n>\n>     git config --worktree . foo bar\n> fatal: Per-worktree configuration requires extensions.worktreeConfig\n> Please read section CONFIGURATION in `git help worktree` before\n> enabling it.\n>\n> which would trigger the failure here?\n\nIt was intended, but I was probably just paranoid. The thinking back\nthen was, you are switching from \"share whole config\" to \"not share\nsomething\". This should not be taken lightly and you should examine\nyour config file and decide what to share, before making the switch.\nIt's dangerous!\n\nBut then, if everything has been shared before (assuming there are\nmore than one worktree) and you are probably happy with it (or you\nwould have made done something to unshare), so it's probably good to\nkeep on sharing. Which means we can set extensions.worktreeConfig\nautomatically here (when \"git config --worktree\" is used) instead of\ndying. We would need to move core.bare and core.worktree to main\nworktree, but that's manageable.\n\nSo in short, you would not see this message in this context in future again.\n-- \nDuy\n"},{"id":"291983","messageId":"CAGZ79ka-isR4DL7ZqOp8cXE1bmUOnd33yu=pZZHaqNmPWH3PYQ@mail.gmail.com","threadId":"42882","inReplyTo":"CACsJy8CSnmnzDMGpMvvkhWRfJvp1L+pfOZ=eYp5JF0GWNH6D0Q@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-22T17:25:42Z","receivedAt":"2016-07-22T17:25:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jul 22, 2016 at 10:09 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n>\n> I just quickly glanced through the rest of this mail because, as a\n> submodule ignorant, it's just mumbo jumbo to me. But what I see here\n> is, there may be problems if we choose to share some submodule info,\n> but I haven't seen any good thing from sharing any submodule info at\n> all.\n\nOkay. :(\n\nI assume the sharing is beneficial. (As a work-tree ignorant) I thought\nwe had this main work tree, which also holds the repository, whereas\nthe other working trees have a light weight implementation (e.g. just\na .git file pointing back to the main working tree/git dir).\n\nSo in a way my mental model is more like the config sharing here:\nYou can configure things in ~/.gitconfig for example that have effects\non more than one repo. Similarly you would want to configure things\nin one repo, that has effect on more than one working tree?\n\nAnd my assumption was to have the repository specific parts be shared,\nwhereas the working tree specific things should not be shared.\n\nBy working tree specific I strongly mean:\n\n* existence in the working tree\n* the checked out sha1\n* submodule.$name.path\n\nBy repository specific I strongly mean:\n\n* the submodule URL\n\nI am not sure about:\n\n* submodule.$name.update, submodule.$name.ignore,\n   submodule.$name.branch,\n  These have to be able to be different across working trees, but do we\n  require them to be set for each working tree individually?  I thought a\n  repo wide setup with defaults may be ok?\n\n>\n> I can imagine long term you may want to just clone a submodule repo\n> once and share it across worktrees that use it, so maybe it's just me\n> not seeing things and this may be a step towards that.\n\nJust as Junio put it:\n> I agree that when a top-level superproject has multiple worktrees\n> these multiple worktrees may want to have the same submodule in\n> different states, but I'd imagine that they want to share the same\n> physical repository (i.e. $GIT_DIR/modules/$name of the primary\n> worktree of the superproject)---is everybody involved in the\n> discussion share this assumption?\n\nI agree with that as well.\n\n>\n> Anyway, I assume some people will be working on the submodule side.\n\nOnce the discussion comes to a rough agreement, I'll give it a shot.\n\n> And because I have not heard any bad thing about the new config\n> design, I'm going to drop submodule patches from this series and focus\n> on polishing config stuff.\n\nOh, sorry for not focusing on that part. The design of git config --worktree\nis sound IMO.\n\n> --\n> Duy\n"},{"id":"291984","messageId":"CAGZ79kbH=ywi7sXUz5KKyRqo-Eg4RF3W9pf53rzKE-oz5-PW1Q@mail.gmail.com","threadId":"42882","inReplyTo":"xmqqmvl9boju.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-22T17:40:44Z","receivedAt":"2016-07-22T17:40:50Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jul 22, 2016 at 9:55 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> From a users POV there are:\n>> * non existent submodules (no gitlink recorded, no config set,\n>>   no repo in place)\n>> * not initialized submodules (gitlink is recorded, no config set,\n>>   and an empty repo is put in the working tree as a place holder).\n\nI meant empty directory, not empty repo.\n\n>\n> This is no different from what you later call \"embedded\".  The only\n> difference is that embedded thing hasn't seen its initial commit.\n\nThat did not occur to me.\nThe \"not initialized\" is what you'd get via\n\n    git clone --no-recurse repo-with-submodules\n\nwhereas the \"embedded\" could come from\n\n   git clone <repo with no submodules> tmp\n   cd tmp && git clone <another repo, maybe unrelated>\n\n>\n>> * initialized submodules (gitlink is recorded, the config\n>>   submodule .<name>.url is copied from the .gitmodules file to .git/config.\n>>   an empty dir in the working tree as a place holder)\n>>   A user may change the configuration before the next step as the url in\n>>   the .gitmodules file may be wrong and the user doesn't want to\n>>   rewrite history\n>\n> i.e. what \"submodule init\" gives you.\n\nRight.\n\n>\n>> * existing submodules (gitlink is recorded, the config option is set\n>>   and instead of an empty placeholder dir, we actually have a git\n>>   repo there.)\n>\n> i.e. what \"submodule update\" after \"submodule init\" gives you.\n\nRight.\n\n>\n>> * matching submodules (the recorded git link matches\n>>   the actual checked out state of the repo!, config option and repo exist)\n>\n> Is this any different from \"existing\" case for the purpose of\n> discussing the interaction between a submodule (and its checkout)\n> and having possibly multiple worktrees of its superproject?\n\nI don't think so.\n\n>\n> I agree that when a top-level superproject has multiple worktrees\n> these multiple worktrees may want to have the same submodule in\n> different states, but I'd imagine that they want to share the same\n> physical repository (i.e. $GIT_DIR/modules/$name of the primary\n> worktree of the superproject)---is everybody involved in the\n> discussion share this assumption?\n\nAt least me agrees.\n\n>\n> Assuming that everybody is on the same page, that means \"do we have\n> the repository for that submodule, and if so where in our local\n> filesystem?\" is a bit of information shared across the worktrees of\n> the superproject.  And the \"name\" used to identify the submodule is\n> also shared across these worktrees of the superproject, as it is\n> meant to be a unique (within the superproject) identifier for that\n> \"other\" project it uses, no matter where in the superproject's\n> working tree (note: this is \"working tree\", not \"worktree\") it would\n> be checked out, and where the upstream URL to get further updates to\n> the submodule is (i.e. that URL may change over time if they relocate,\n> or it may even change when the user who works on the superproject\n> decides to use a different mirror).\n\nI agree.\n\n>\n> What can be different between the instantiation of the same\n> submodule in these multiple worktrees, and how they should be\n> recorded?\n>\n>  * submodule.$name.URL?  I am not sure if we want to have different\n>    \"upstreams\" depending on the worktree of the superproject.  While\n>    there is no fundamental reason to forbid it, having to maintain a\n>    single local repository for a submodule would mean they would\n>    need to be treated as separate \"remotes\" in the submodule\n>    repository.\n\nYou can only have a remote if the the submodule repo exists already.\nI guess that can be made a requirement.\n\nSo setting up the worktrees and submodule URLs in the config and\nthen doing the clone of said submodule is maybe not encouraged.\n\n>\n>  * submodule.$name.path of course can be different depending on\n>    which commit of the superproject is checked out in the worktree,\n>    as the superproject may move the submodule binding site across\n>    its versions.\n\nRight.\n\n>\n>  * submodule.$name.update, submodule.$name.ignore,\n>    submodule.$name.branch, etc. would need to be all different among\n>    worktrees of the superproject, as that is the whole point of\n>    being able to work on separate branches of the superproject in\n>    separate worktrees.\n\nWhat do you mean by \"would need\". The ability to be different or rather\nthe veto of an 'inheritance' of defaults from the repository configuration?\n\n>\n> Somewhere in this discussion thread, you present the conclusion of\n> your discussion with Jonathan Nieder that there needs a separate\n> \"should the submodule directory be populated?\" bit, which currently\n> is tied to submodule.$name.URL in $GIT_DIR/config.\n\nI'll try to get the discussion back on list and whenever Jonathan starts talking\noff list, I'll poke him with a stick.\n\n>  I tend to agree\n> that knowing where you get other people's update of that submodule\n> repository should come from and wanting to have/keep a checkout of\n> that submodule in the working tree of a particular worktree are two\n> different things, so such a separate bit would be needed, and that\n> would belong to per-worktree configuration.\n>\n\nOkay. How would you disentangle these two things?\n"},{"id":"291985","messageId":"CACsJy8DKEV3FNmb1vWinRvb-FHSO_VftG7RevQ3TOFhP-Dm0cw@mail.gmail.com","threadId":"42882","inReplyTo":"CAGZ79ka-isR4DL7ZqOp8cXE1bmUOnd33yu=pZZHaqNmPWH3PYQ@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-22T17:42:25Z","receivedAt":"2016-07-22T17:42:59Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Jul 22, 2016 at 7:25 PM, Stefan Beller <sbeller@google.com> wrote:\n> On Fri, Jul 22, 2016 at 10:09 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n>>\n>> I just quickly glanced through the rest of this mail because, as a\n>> submodule ignorant, it's just mumbo jumbo to me. But what I see here\n>> is, there may be problems if we choose to share some submodule info,\n>> but I haven't seen any good thing from sharing any submodule info at\n>> all.\n>\n> Okay. :(\n\nDidn't mean to make you feel sad :)\n\n\n> I assume the sharing is beneficial. (As a work-tree ignorant) I thought\n> we had this main work tree, which also holds the repository, whereas\n> the other working trees have a light weight implementation (e.g. just\n> a .git file pointing back to the main working tree/git dir).\n\nThe main worktree is special for historical reason. But from the user\npoint of view (and even developer's at a certain level) they should be\ntreated equally. Think of it like cloning the same repo multiple\ntimes. Only now you save disk space because there's only one object\ndatabase.\n\n> So in a way my mental model is more like the config sharing here\n> You can configure things in ~/.gitconfig for example that have effects\n> on more than one repo. Similarly you would want to configure things\n> in one repo, that has effect on more than one working tree?\n>\n> And my assumption was to have the repository specific parts be shared,\n> whereas the working tree specific things should not be shared.\n\nI think that's a good assumption. Although I would rather be not\nsharing by default and let the user initiate it when they want to\nshare something. Like ~/..gitconfig, we never write anything there\nunless the user asks us to explicitly (with git config --user).\nAccidental share could have negative effect.\n\n>> I can imagine long term you may want to just clone a submodule repo\n>> once and share it across worktrees that use it, so maybe it's just me\n>> not seeing things and this may be a step towards that.\n>\n> Just as Junio put it:\n>> I agree that when a top-level superproject has multiple worktrees\n>> these multiple worktrees may want to have the same submodule in\n>> different states, but I'd imagine that they want to share the same\n>> physical repository (i.e. $GIT_DIR/modules/$name of the primary\n>> worktree of the superproject)---is everybody involved in the\n>> discussion share this assumption?\n>\n> I agree with that as well.\n\nYeah. We have a long way to go though. As I see it, you may need ref\nnamespace as well (so they look like separate clones), which has never\nbeen used on the client side before. Either that or odb alternates...\n\n>> And because I have not heard any bad thing about the new config\n>> design, I'm going to drop submodule patches from this series and focus\n>> on polishing config stuff.\n>\n> Oh, sorry for not focusing on that part. The design of git config --worktree\n> is sound IMO.\n\nThis makes me happy (I know other people can still find flaws in it,\nand I'm ok with that). This config split thing has been wrecking my\nbrain for a long time, find the the \"right\" way to do with minimum\nimpacts :)\n-- \nDuy\n"},{"id":"292096","messageId":"xmqqwpk97mbn.fsf@gitster.mtv.corp.google.com","threadId":"42882","inReplyTo":"CAGZ79kbH=ywi7sXUz5KKyRqo-Eg4RF3W9pf53rzKE-oz5-PW1Q@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-07-25T15:46:04Z","receivedAt":"2016-07-25T15:46:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> On Fri, Jul 22, 2016 at 9:55 AM, Junio C Hamano <gitster@pobox.com> wrote:\n>>\n>>  * submodule.$name.update, submodule.$name.ignore,\n>>    submodule.$name.branch, etc. would need to be all different among\n>>    worktrees of the superproject, as that is the whole point of\n>>    being able to work on separate branches of the superproject in\n>>    separate worktrees.\n>\n> What do you mean by \"would need\". The ability to be different or rather\n> the veto of an 'inheritance' of defaults from the repository configuration?\n\nThey have to be able to represent different settings per worktree\nthat checks out different branches/commits of superproject.  They\nmay happen to be set the same, but they do not have to be.\n\nIs what I meant.\n"},{"id":"292164","messageId":"CAGZ79kYbmoKPAPMVkTUycSKVtT6HLK-Y_eGXSX+z69G3+udR8Q@mail.gmail.com","threadId":"42882","inReplyTo":"CACsJy8DKEV3FNmb1vWinRvb-FHSO_VftG7RevQ3TOFhP-Dm0cw@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-25T23:25:23Z","receivedAt":"2016-07-25T23:25:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jul 22, 2016 at 10:42 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n> On Fri, Jul 22, 2016 at 7:25 PM, Stefan Beller <sbeller@google.com> wrote:\n>> On Fri, Jul 22, 2016 at 10:09 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n>>>\n>>> I just quickly glanced through the rest of this mail because, as a\n>>> submodule ignorant, it's just mumbo jumbo to me. But what I see here\n>>> is, there may be problems if we choose to share some submodule info,\n>>> but I haven't seen any good thing from sharing any submodule info at\n>>> all.\n>>\n>> Okay. :(\n>\n> Didn't mean to make you feel sad :)\n\nI was using the :( a bit carelessly here. I was quite surprised that you\n\"haven't seen any good thing from sharing any submodule info at all.\"\n\nSo what is the design philosophy in worktrees? How much independence does\none working tree have?\n\nSo here is what I did:\n *  s/git submodule init/git submodule update --init/\n * added a test_pause to the last test on the last line\n * Then:\n\n$ find . |grep da5e6058\n./addtest/.git/modules/submod/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n./addtest/.git/worktrees/super-elsewhere/modules/submod/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n./addtest/.git/worktrees/super-elsewhere/modules/submod2/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n./.git/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n\nThe last entry is the \"upstream\" for the addtest clone, so that is fine.\nHowever inside the ./addtest/ (and its worktrees, which currently are\nembedded in there?) we only want to have one object store for a given\nsubmodule?\n\n>\n>> I assume the sharing is beneficial. (As a work-tree ignorant) I thought\n>> we had this main work tree, which also holds the repository, whereas\n>> the other working trees have a light weight implementation (e.g. just\n>> a .git file pointing back to the main working tree/git dir).\n>\n> The main worktree is special for historical reason. But from the user\n> point of view (and even developer's at a certain level) they should be\n> treated equally. Think of it like cloning the same repo multiple\n> times. Only now you save disk space because there's only one object\n> database.\n\nThat's what we want for submodules too, see above?\n\n>\n>> So in a way my mental model is more like the config sharing here\n>> You can configure things in ~/.gitconfig for example that have effects\n>> on more than one repo. Similarly you would want to configure things\n>> in one repo, that has effect on more than one working tree?\n>>\n>> And my assumption was to have the repository specific parts be shared,\n>> whereas the working tree specific things should not be shared.\n>\n> I think that's a good assumption. Although I would rather be not\n> sharing by default and let the user initiate it when they want to\n> share something. Like ~/..gitconfig, we never write anything there\n> unless the user asks us to explicitly (with git config --user).\n> Accidental share could have negative effect.\n\nOkay, got it.\n\n>\n>>> I can imagine long term you may want to just clone a submodule repo\n>>> once and share it across worktrees that use it, so maybe it's just me\n>>> not seeing things and this may be a step towards that.\n>>\n>> Just as Junio put it:\n>>> I agree that when a top-level superproject has multiple worktrees\n>>> these multiple worktrees may want to have the same submodule in\n>>> different states, but I'd imagine that they want to share the same\n>>> physical repository (i.e. $GIT_DIR/modules/$name of the primary\n>>> worktree of the superproject)---is everybody involved in the\n>>> discussion share this assumption?\n>>\n>> I agree with that as well.\n>\n> Yeah. We have a long way to go though. As I see it, you may need ref\n> namespace as well (so they look like separate clones), which has never\n> been used on the client side before. Either that or odb alternates...\n>\n>>> And because I have not heard any bad thing about the new config\n>>> design, I'm going to drop submodule patches from this series and focus\n>>> on polishing config stuff.\n>>\n>> Oh, sorry for not focusing on that part. The design of git config --worktree\n>> is sound IMO.\n>\n> This makes me happy (I know other people can still find flaws in it,\n> and I'm ok with that). This config split thing has been wrecking my\n> brain for a long time, find the the \"right\" way to do with minimum\n> impacts :)\n\nAfter playing with this series a bit more, I actually like the UI as it is an\neasy mental model \"submodules behave completely independent\".\n\nHowever in 3/4 you said:\n\n+ - `submodule.*` in current state should not be shared because the\n+   information is tied to a particular version of .gitmodules in a\n+   working directory.\n\nThis is already a problem with say different branches/versions.\nThat has been solved by duplicating that information to .git/config\nas a required step. (I don't like that approach, as it is super confusing\nIMHO)\n\n+\n+ - `remote.*` added by submodules may be per working directory as\n+   well, unless you are sure remotes from all possible submodules in\n+   history are consistent.\n+\n\nSame as above.\n\nI planned to have a series out last Friday, but it took longer than expected\nas it was unclear how to best proceed and I ran into problems solving\n\"trivial issues\".\n\nI am back to the drawing board for the submodule side of things,\nbut I guess this series could be used once we figure out how to\nhave just one object database for a submodule.\n\nThanks,\nStefan\n\n> --\n> Duy\n"},{"id":"292169","messageId":"CAGZ79kYET=z-b+U-JN+H5jkRTGHR0oMdTfUZPMRJx50aH-idbw@mail.gmail.com","threadId":"42882","inReplyTo":"20160720172419.25473-2-pclouds@gmail.com","subject":"Re: [PATCH v4 1/4] worktree: add per-worktree config files","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-26T00:59:03Z","receivedAt":"2016-07-26T00:59:13Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jul 20, 2016 at 10:24 AM, Nguyễn Thái Ngọc Duy\n<pclouds@gmail.com> wrote:\n> A new repo extension is added, worktreeConfig. When it is present:\n>\n>  - Repository config reading by default includes $GIT_DIR/config _and_\n>    $GIT_DIR/config.worktree. \"config\" file remains shared in multiple\n>    worktree setup.\n>\n>  - The special treatment for core.bare and core.worktree, to stay\n>    effective only in main worktree, is gone. These config files are\n>    supposed to be in config.worktree.\n>\n> This extension is most useful in multiple worktree setup because you\n> now have an option to store per-worktree config (which is either\n> .git/config.worktree for main worktree, or\n> .git/worktrees/xx/config.worktree for linked ones).\n>\n> This extension can be used in single worktree mode, even though it's\n> pretty much useless (but this can happen after you remove all linked\n> worktrees and move back to single worktree).\n>\n> \"git config\" reads from both \"config\" and \"config.worktree\" by default\n> (i.e. without either --user, --file...) when this extension is\n> present. Default writes still go to \"config\", not \"config.worktree\". A\n> new option --worktree is added for that (*).\n>\n> Since a new repo extension is introduced, existing git binaries should\n> refuse to access to the repo (both from main and linked worktrees). So\n> they will not misread the config file (i.e. skip the config.worktree\n> part). They may still accidentally write to the config file anyway if\n> they use with \"git config --file <path>\".\n>\n> This design places a bet on the assumption that the majority of config\n> variables are shared so it is the default mode. A safer move would be\n> default writes go to per-worktree file, so that accidental changes are\n> isolated.\n>\n> (*) \"git config --worktree\" points back to \"config\" file when this\n>     extension is not present so that it works in any setup.\n>\n> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>\n\nI like the user facing design, but how am I supposed to use it internally?\n\nSay I want to read a value preferably from the worktree I'd do a\n    /*\n     * maybe I don't even have to set it to 1 as\n     * the user is supposed to do that?\n     */\n    repository_format_worktree_config = 1;\n    git_config_get_{string,bool,int} (... as usual ...)\n\nand if I want to read the value globally I would set the variable to 0\nand read? (I would need to restore it, so I'll have a temporary variable\nto keep the original value of repository_format_worktree_config)\n\nThanks,\nStefan\n\n\n> ---\n>  Documentation/config.txt               | 11 ++++-\n>  Documentation/git-config.txt           | 26 ++++++++----\n>  Documentation/git-worktree.txt         | 31 ++++++++++++++\n>  Documentation/gitrepository-layout.txt |  8 ++++\n>  builtin/config.c                       | 18 +++++++-\n>  cache.h                                |  2 +\n>  config.c                               |  7 ++++\n>  environment.c                          |  1 +\n>  setup.c                                |  5 ++-\n>  t/t2028-worktree-config.sh (new +x)    | 77 ++++++++++++++++++++++++++++++++++\n>  10 files changed, 175 insertions(+), 11 deletions(-)\n>  create mode 100755 t/t2028-worktree-config.sh\n>\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 16dc22d..7d64da0 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -2,8 +2,9 @@ CONFIGURATION FILE\n>  ------------------\n>\n>  The Git configuration file contains a number of variables that affect\n> -the Git commands' behavior. The `.git/config` file in each repository\n> -is used to store the configuration for that repository, and\n> +the Git commands' behavior. The files `.git/config` and optionally\n> +`config.worktree` (see `extensions.worktreeConfig` below) are each\n> +repository is used to store the configuration for that repository, and\n>  `$HOME/.gitconfig` is used to store a per-user configuration as\n>  fallback values for the `.git/config` file. The file `/etc/gitconfig`\n>  can be used to store a system-wide default configuration.\n> @@ -264,6 +265,12 @@ advice.*::\n>                 show directions on how to proceed from the current state.\n>  --\n>\n> +extensions.worktreeConfig::\n> +       If set, by default \"git config\" reads from both \"config\" and\n> +       \"config.worktree\" file in that order. In multiple working\n> +       directory mode, \"config\" file is shared while\n> +       \"config.worktree\" is per-working directory.\n> +\n>  core.fileMode::\n>         Tells Git if the executable bit of files in the working tree\n>         is to be honored.\n> diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt\n> index f163113..9dfdb6a 100644\n> --- a/Documentation/git-config.txt\n> +++ b/Documentation/git-config.txt\n> @@ -47,13 +47,15 @@ checks or transformations are performed on the value.\n>\n>  When reading, the values are read from the system, global and\n>  repository local configuration files by default, and options\n> -`--system`, `--global`, `--local` and `--file <filename>` can be\n> -used to tell the command to read from only that location (see <<FILES>>).\n> +`--system`, `--global`, `--local`, `--worktree` and\n> +`--file <filename>` can be used to tell the command to read from only\n> +that location (see <<FILES>>).\n>\n>  When writing, the new value is written to the repository local\n>  configuration file by default, and options `--system`, `--global`,\n> -`--file <filename>` can be used to tell the command to write to\n> -that location (you can say `--local` but that is the default).\n> +`--worktree`, `--file <filename>` can be used to tell the command to\n> +write to that location (you can say `--local` but that is the\n> +default).\n>\n>  This command will fail with non-zero status upon error.  Some exit\n>  codes are:\n> @@ -133,6 +135,11 @@ from all available files.\n>  +\n>  See also <<FILES>>.\n>\n> +--worktree::\n> +       Similar to `--local` except that `.git/config.worktree` is\n> +       read from or written to if `extensions.worktreeConfig` is\n> +       present. If not it's the same as `--local`.\n> +\n>  -f config-file::\n>  --file config-file::\n>         Use the given config file instead of the one specified by GIT_CONFIG.\n> @@ -253,6 +260,10 @@ $XDG_CONFIG_HOME/git/config::\n>  $GIT_DIR/config::\n>         Repository specific configuration file.\n>\n> +$GIT_DIR/config.worktree::\n> +       This is optional and is only searched when\n> +       `extensions.worktreeConfig` is present in $GIT_DIR/config.\n> +\n>  If no further options are given, all reading options will read all of these\n>  files that are available. If the global or the system-wide configuration\n>  file are not available they will be ignored. If the repository configuration\n> @@ -268,9 +279,10 @@ configuration file. Note that this also affects options like `--replace-all`\n>  and `--unset`. *'git config' will only ever change one file at a time*.\n>\n>  You can override these rules either by command-line options or by environment\n> -variables. The `--global` and the `--system` options will limit the file used\n> -to the global or system-wide file respectively. The `GIT_CONFIG` environment\n> -variable has a similar effect, but you can specify any filename you want.\n> +variables. The `--global`, `--system` and `--worktree` options will limit\n> +the file used to the global, system-wide or per-worktree file respectively.\n> +The `GIT_CONFIG` environment variable has a similar effect, but you\n> +can specify any filename you want.\n>\n>\n>  ENVIRONMENT\n> diff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt\n> index 7c4cfb0..41350db 100644\n> --- a/Documentation/git-worktree.txt\n> +++ b/Documentation/git-worktree.txt\n> @@ -111,6 +111,37 @@ OPTIONS\n>  --expire <time>::\n>         With `prune`, only expire unused working trees older than <time>.\n>\n> +CONFIGURATION FILE\n> +------------------\n> +By default, the repository \"config\" file is shared across all working\n> +directories. If the config variables `core.bare` or `core.worktree`\n> +are already present in the config file, they will be applied to the\n> +main working directory only.\n> +\n> +In order to have configuration specific to working directories, you\n> +can turn on \"worktreeConfig\" extension, e.g.:\n> +\n> +------------\n> +$ git config extensions.worktreeConfig true\n> +------------\n> +\n> +In this mode, specific configuration stays in the path pointed by `git\n> +rev-parse --git-path config.worktree`. You can add or update\n> +configuration in this file with `git config --worktree`. Git before\n> +version XXX will refuse to access repositories with this extension.\n> +\n> +Note that in this file, the exception for `core.bare` and\n> +core.worktree` is gone. If you have them before, you need to move them\n> +to the config.worktree of the main working directory. You may also\n> +take this opportunity to move other configuration that you do not want\n> +to share to all working directories:\n> +\n> + - `core.worktree` and `core.bare` should never be shared\n> +\n> + - `core.sparseCheckout` is recommended per working directory, unless\n> +   you are sure you always use sparse checkout for all working\n> +   directories.\n> +\n>  DETAILS\n>  -------\n>  Each linked working tree has a private sub-directory in the repository's\n> diff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt\n> index 577ee84..6cfdb4c 100644\n> --- a/Documentation/gitrepository-layout.txt\n> +++ b/Documentation/gitrepository-layout.txt\n> @@ -143,6 +143,11 @@ config::\n>         if $GIT_COMMON_DIR is set and \"$GIT_COMMON_DIR/config\" will be\n>         used instead.\n>\n> +config.worktree::\n> +       Working directory specific configuration file for the main\n> +       working directory in multiple working directory setup (see\n> +       linkgit:git-worktree[1]).\n> +\n>  branches::\n>         A slightly deprecated way to store shorthands to be used\n>         to specify a URL to 'git fetch', 'git pull' and 'git push'.\n> @@ -276,6 +281,9 @@ worktrees/<id>/link::\n>         file. It is used to detect if the linked repository is\n>         manually removed.\n>\n> +worktrees/<id>/config.worktree::\n> +       Working directory specific configuration file.\n> +\n>  SEE ALSO\n>  --------\n>  linkgit:git-init[1],\n> diff --git a/builtin/config.c b/builtin/config.c\n> index 1d7c6ef..535707c 100644\n> --- a/builtin/config.c\n> +++ b/builtin/config.c\n> @@ -4,6 +4,7 @@\n>  #include \"parse-options.h\"\n>  #include \"urlmatch.h\"\n>  #include \"quote.h\"\n> +#include \"worktree.h\"\n>\n>  static const char *const builtin_config_usage[] = {\n>         N_(\"git config [<options>]\"),\n> @@ -23,6 +24,7 @@ static char key_delim = ' ';\n>  static char term = '\\n';\n>\n>  static int use_global_config, use_system_config, use_local_config;\n> +static int use_worktree_config;\n>  static struct git_config_source given_config_source;\n>  static int actions, types;\n>  static const char *get_color_slot, *get_colorbool_slot;\n> @@ -57,6 +59,7 @@ static struct option builtin_config_options[] = {\n>         OPT_BOOL(0, \"global\", &use_global_config, N_(\"use global config file\")),\n>         OPT_BOOL(0, \"system\", &use_system_config, N_(\"use system config file\")),\n>         OPT_BOOL(0, \"local\", &use_local_config, N_(\"use repository config file\")),\n> +       OPT_BOOL(0, \"worktree\", &use_worktree_config, N_(\"use per-worktree config file\")),\n>         OPT_STRING('f', \"file\", &given_config_source.file, N_(\"file\"), N_(\"use given config file\")),\n>         OPT_STRING(0, \"blob\", &given_config_source.blob, N_(\"blob-id\"), N_(\"read config from given blob object\")),\n>         OPT_GROUP(N_(\"Action\")),\n> @@ -491,6 +494,7 @@ int cmd_config(int argc, const char **argv, const char *prefix)\n>                              PARSE_OPT_STOP_AT_NON_OPTION);\n>\n>         if (use_global_config + use_system_config + use_local_config +\n> +           use_worktree_config +\n>             !!given_config_source.file + !!given_config_source.blob > 1) {\n>                 error(\"only one config file at a time.\");\n>                 usage_with_options(builtin_config_usage, builtin_config_options);\n> @@ -525,7 +529,19 @@ int cmd_config(int argc, const char **argv, const char *prefix)\n>                 given_config_source.file = git_etc_gitconfig();\n>         else if (use_local_config)\n>                 given_config_source.file = git_pathdup(\"config\");\n> -       else if (given_config_source.file) {\n> +       else if (use_worktree_config) {\n> +               if (repository_format_worktree_config)\n> +                       given_config_source.file = git_pathdup(\"config.worktree\");\n> +               else {\n> +                       struct worktree **worktrees = get_worktrees();\n> +                       if (worktrees[0] && worktrees[1])\n> +                               die(_(\"Per-worktree configuration requires extensions.worktreeConfig\\n\"\n> +                                     \"Please read section CONFIGURATION in `git help worktree` before\\n\"\n> +                                     \"enabling it.\"));\n> +                       free_worktrees(worktrees);\n> +                       given_config_source.file = git_pathdup(\"config\");\n> +               }\n> +       } else if (given_config_source.file) {\n>                 if (!is_absolute_path(given_config_source.file) && prefix)\n>                         given_config_source.file =\n>                                 xstrdup(prefix_filename(prefix,\n> diff --git a/cache.h b/cache.h\n> index f1dc289..606500e 100644\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -757,10 +757,12 @@ extern int grafts_replace_parents;\n>  #define GIT_REPO_VERSION 0\n>  #define GIT_REPO_VERSION_READ 1\n>  extern int repository_format_precious_objects;\n> +extern int repository_format_worktree_config;\n>\n>  struct repository_format {\n>         int version;\n>         int precious_objects;\n> +       int worktree_config;\n>         int is_bare;\n>         char *work_tree;\n>         struct string_list unknown_extensions;\n> diff --git a/config.c b/config.c\n> index bea937e..99ff6be 100644\n> --- a/config.c\n> +++ b/config.c\n> @@ -1254,6 +1254,13 @@ static int do_git_config_sequence(config_fn_t fn, void *data)\n>         if (repo_config && !access_or_die(repo_config, R_OK, 0))\n>                 ret += git_config_from_file(fn, repo_config, data);\n>\n> +       if (repository_format_worktree_config) {\n> +               char *path = git_pathdup(\"config.worktree\");\n> +               if (!access_or_die(path, R_OK, 0))\n> +                       ret += git_config_from_file(fn, path, data);\n> +               free(path);\n> +       }\n> +\n>         current_parsing_scope = CONFIG_SCOPE_CMDLINE;\n>         if (git_config_from_parameters(fn, data) < 0)\n>                 die(_(\"unable to parse command-line config\"));\n> diff --git a/environment.c b/environment.c\n> index ca72464..b4d56ef 100644\n> --- a/environment.c\n> +++ b/environment.c\n> @@ -26,6 +26,7 @@ int warn_ambiguous_refs = 1;\n>  int warn_on_object_refname_ambiguity = 1;\n>  int ref_paranoia = -1;\n>  int repository_format_precious_objects;\n> +int repository_format_worktree_config;\n>  const char *git_commit_encoding;\n>  const char *git_log_output_encoding;\n>  const char *apply_default_whitespace;\n> diff --git a/setup.c b/setup.c\n> index 6d0e0c9..75c784f 100644\n> --- a/setup.c\n> +++ b/setup.c\n> @@ -389,6 +389,8 @@ static int check_repo_format(const char *var, const char *value, void *vdata)\n>                         ;\n>                 else if (!strcmp(ext, \"preciousobjects\"))\n>                         data->precious_objects = git_config_bool(var, value);\n> +               else if (!strcmp(ext, \"worktreeconfig\"))\n> +                       data->worktree_config = git_config_bool(var, value);\n>                 else\n>                         string_list_append(&data->unknown_extensions, ext);\n>         } else if (strcmp(var, \"core.bare\") == 0) {\n> @@ -432,8 +434,9 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)\n>         }\n>\n>         repository_format_precious_objects = candidate.precious_objects;\n> +       repository_format_worktree_config = candidate.worktree_config;\n>         string_list_clear(&candidate.unknown_extensions, 0);\n> -       if (!has_common) {\n> +       if (!has_common || repository_format_worktree_config) {\n>                 if (candidate.is_bare != -1) {\n>                         is_bare_repository_cfg = candidate.is_bare;\n>                         if (is_bare_repository_cfg == 1)\n> diff --git a/t/t2028-worktree-config.sh b/t/t2028-worktree-config.sh\n> new file mode 100755\n> index 0000000..34067df\n> --- /dev/null\n> +++ b/t/t2028-worktree-config.sh\n> @@ -0,0 +1,77 @@\n> +#!/bin/sh\n> +\n> +test_description=\"config file in multi worktree\"\n> +\n> +. ./test-lib.sh\n> +\n> +cmp_config() {\n> +       if [ \"$1\" = \"-C\" ]; then\n> +               shift &&\n> +               GD=\"-C $1\" &&\n> +               shift\n> +       else\n> +               GD=\n> +       fi &&\n> +       echo \"$1\" >expected &&\n> +       shift &&\n> +       git $GD config \"$@\" >actual &&\n> +       test_cmp expected actual\n> +}\n> +\n> +test_expect_success 'setup' '\n> +       test_commit start &&\n> +       git config --worktree per.worktree is-ok &&\n> +       git worktree add wt1 &&\n> +       git worktree add wt2 &&\n> +       test_must_fail git config --worktree per.worktree is-not-ok &&\n> +       git config extensions.worktreeConfig true\n> +'\n> +\n> +test_expect_success 'config is shared as before' '\n> +       git config this.is shared &&\n> +       cmp_config shared this.is &&\n> +       cmp_config -C wt1 shared this.is &&\n> +       cmp_config -C wt2 shared this.is\n> +'\n> +\n> +test_expect_success 'config is shared (set from another worktree)' '\n> +       git -C wt1 config that.is also-shared &&\n> +       cmp_config also-shared that.is &&\n> +       cmp_config -C wt1 also-shared that.is &&\n> +       cmp_config -C wt2 also-shared that.is\n> +'\n> +\n> +test_expect_success 'config private to main worktree' '\n> +       git config --worktree this.is for-main &&\n> +       cmp_config for-main this.is &&\n> +       cmp_config -C wt1 shared this.is &&\n> +       cmp_config -C wt2 shared this.is\n> +'\n> +\n> +test_expect_success 'config private to linked worktree' '\n> +       git -C wt1 config --worktree this.is for-wt1 &&\n> +       cmp_config for-main this.is &&\n> +       cmp_config -C wt1 for-wt1 this.is &&\n> +       cmp_config -C wt2 shared this.is\n> +'\n> +\n> +test_expect_success 'core.bare no longer for main only' '\n> +       git config core.bare true &&\n> +       cmp_config true core.bare &&\n> +       cmp_config -C wt1 true core.bare &&\n> +       cmp_config -C wt2 true core.bare &&\n> +       git config --unset core.bare\n> +'\n> +\n> +test_expect_success 'config.worktree no longer read without extension' '\n> +       git config --unset extensions.worktreeConfig &&\n> +       cmp_config shared this.is &&\n> +       cmp_config -C wt1 shared this.is &&\n> +       cmp_config -C wt2 shared this.is\n> +'\n> +\n> +test_expect_success 'config --worktree fails in multi worktree without extension' '\n> +       test_must_fail git config --worktree foo.bar true\n> +'\n> +\n> +test_done\n> --\n> 2.9.1.566.gbd532d4\n>\n"},{"id":"292212","messageId":"CACsJy8CUxLG5KL-u6hxCjVaydi=bXZ8Qr8RZwKPW16o_cfWVcA@mail.gmail.com","threadId":"42882","inReplyTo":"CAGZ79kYET=z-b+U-JN+H5jkRTGHR0oMdTfUZPMRJx50aH-idbw@mail.gmail.com","subject":"Re: [PATCH v4 1/4] worktree: add per-worktree config files","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-26T15:04:12Z","receivedAt":"2016-07-26T15:04:51Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Jul 26, 2016 at 2:59 AM, Stefan Beller <sbeller@google.com> wrote:\n> I like the user facing design, but how am I supposed to use it internally?\n>\n> Say I want to read a value preferably from the worktree I'd do a\n>     /*\n>      * maybe I don't even have to set it to 1 as\n>      * the user is supposed to do that?\n>      */\n>     repository_format_worktree_config = 1;\n>     git_config_get_{string,bool,int} (... as usual ...)\n>\n> and if I want to read the value globally I would set the variable to 0\n> and read? (I would need to restore it, so I'll have a temporary variable\n> to keep the original value of repository_format_worktree_config)\n\nI would understand if you need an api to write to worktree config or\nthe shared one. But choosing to _read_ from a specific source sounds\nwrong. The common rule should apply everywhere: read from worktree\nfirst, if not found, read again from shared config. Why do you need\nthis?\n-- \nDuy\n"},{"id":"292239","messageId":"CACsJy8DgeSOh-RScmcrSwy7PgeQXwA2R6w9mRmHzuWR4djg=4w@mail.gmail.com","threadId":"42882","inReplyTo":"CAGZ79kYbmoKPAPMVkTUycSKVtT6HLK-Y_eGXSX+z69G3+udR8Q@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-26T17:20:40Z","receivedAt":"2016-07-26T17:21:16Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Jul 26, 2016 at 1:25 AM, Stefan Beller <sbeller@google.com> wrote:\n> So what is the design philosophy in worktrees? How much independence does\n> one working tree have?\n\ngit-worktree started out as an alternative for git-stash: hmm.. i need\nto make some changes in another branch, okay let's leave this worktree\n(with all its messy stuff) as-is, create another worktree, make those\nchanges, then delete the worktree and go back here. There's already\nanother way of doing that without git-stash: you clone the repo, fix\nyour stuff, push back and delete the new repo.\n\nI know I have not really answered your questions. But I think it gives\nan idea what are the typical use cases for multiple worktrees. How\nmuch independence would need to be decided case-by-case, I think.\n\n> So here is what I did:\n>  *  s/git submodule init/git submodule update --init/\n>  * added a test_pause to the last test on the last line\n>  * Then:\n>\n> $ find . |grep da5e6058\n> ./addtest/.git/modules/submod/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n> ./addtest/.git/worktrees/super-elsewhere/modules/submod/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n> ./addtest/.git/worktrees/super-elsewhere/modules/submod2/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n> ./.git/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n>\n> The last entry is the \"upstream\" for the addtest clone, so that is fine.\n> However inside the ./addtest/ (and its worktrees, which currently are\n> embedded in there?) we only want to have one object store for a given\n> submodule?\n\nHow to store stuff in .git is the implementation details that the user\ndoes not care about. As long as we keep the behavior the same (they\ncan still \"git submodule init\" and stuff in the new worktree), sharing\nthe same object store makes sense (pros: lower disk consumption, cons:\nnone).\n\n\n> After playing with this series a bit more, I actually like the UI as it is an\n> easy mental model \"submodules behave completely independent\".\n>\n> However in 3/4 you said:\n>\n> + - `submodule.*` in current state should not be shared because the\n> +   information is tied to a particular version of .gitmodules in a\n> +   working directory.\n>\n> This is already a problem with say different branches/versions.\n> That has been solved by duplicating that information to .git/config\n> as a required step. (I don't like that approach, as it is super confusing\n> IMHO)\n\nHmm.. I didn't realize this. But then I have never given much thought\nabout submodules, probably because I have an alternative solution for\nit (or some of its use cases) anyway :)\n\nOK so it's already a problem. But if we keep sharing submodule stuff\nin .git/config, there's a _new_ problem: when you \"submodule init\" a\nworktree, .git/config is now tailored for the current worktree, when\nyou move back to the previous worktree, you need to \"submodule init\"\nagain. So moving to multiple worktrees setup changes how the user uses\nsubmodule, not good in my opinion.\n\nIf you have a grand plan to make submodule work at switching branches\n(without reinit) and if it happens to work the same way when we have\nmultiple worktrees, great.\n\n> I am back to the drawing board for the submodule side of things,\n> but I guess this series could be used once we figure out how to\n> have just one object database for a submodule.\n\nI would leave this out for now. Let's make submodule work with\nmultiple worktrees first (and see how the users react to this). Then\nwe can try to share object database. Object database and refs are tied\nclosely together so you may run into other problems soon.\n-- \nDuy\n"},{"id":"292243","messageId":"CAGZ79kYGj7q=SQyHvFdmXasJppTVw56xSBMiSERdx22B+A68gQ@mail.gmail.com","threadId":"42882","inReplyTo":"CACsJy8DgeSOh-RScmcrSwy7PgeQXwA2R6w9mRmHzuWR4djg=4w@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-26T18:15:10Z","receivedAt":"2016-07-26T18:23:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Jul 26, 2016 at 10:20 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n> On Tue, Jul 26, 2016 at 1:25 AM, Stefan Beller <sbeller@google.com> wrote:\n>> So what is the design philosophy in worktrees? How much independence does\n>> one working tree have?\n>\n> git-worktree started out as an alternative for git-stash: hmm.. i need\n> to make some changes in another branch, okay let's leave this worktree\n> (with all its messy stuff) as-is, create another worktree, make those\n> changes, then delete the worktree and go back here. There's already\n> another way of doing that without git-stash: you clone the repo, fix\n> your stuff, push back and delete the new repo.\n>\n> I know I have not really answered your questions. But I think it gives\n> an idea what are the typical use cases for multiple worktrees. How\n> much independence would need to be decided case-by-case, I think.\n\nThanks!\n\n\n>\n>> So here is what I did:\n>>  *  s/git submodule init/git submodule update --init/\n>>  * added a test_pause to the last test on the last line\n>>  * Then:\n>>\n>> $ find . |grep da5e6058\n>> ./addtest/.git/modules/submod/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n>> ./addtest/.git/worktrees/super-elsewhere/modules/submod/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n>> ./addtest/.git/worktrees/super-elsewhere/modules/submod2/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n>> ./.git/objects/08/da5e6058267d6be703ae058d173ce38ed53066\n>>\n>> The last entry is the \"upstream\" for the addtest clone, so that is fine.\n>> However inside the ./addtest/ (and its worktrees, which currently are\n>> embedded in there?) we only want to have one object store for a given\n>> submodule?\n>\n> How to store stuff in .git is the implementation details that the user\n> does not care about.\n\nThey do unfortunately. :(\nSome teams here are trying to migrate from the repo[1] tool to submodules,\nand they usually have large code bases. (e.g. The Android Open Source\nProject[2], put into a superproject has a .git dir size of 17G. The\n17G are partitioned as follows:\n\n.../.git$ du --max-depth=1 -h\n    44K ./hooks\n    32K ./refs\n    36K ./logs\n    17G ./modules\n    4.0K ./branches\n    8.0K ./info\n    4.7M ./objects\n    17G .\n\ni.e. roughly all in submodules.\n\nSo our users do care about both what is on disk, as well\nas what goes over the wire (network traffic).\n\nMy sudden interest in worktrees came up when I learned the\n`--reference` flag for submodule operations is broken for\nour use case, and instead of fixing the `--reference` flag,\nI think the worktree approach is generally saner (i.e. with the\nreferences you may have nasty gc issues IIUC, but in the\nworktree world gc knows about all the working trees, detached\nheads and branches.)\n\n[1] https://source.android.com/source/developing.html\n[2] https://android.googlesource.com/\n\n> As long as we keep the behavior the same (they\n> can still \"git submodule init\" and stuff in the new worktree), sharing\n> the same object store makes sense (pros: lower disk consumption, cons:\n> none).\n\nSo I think the current workflow for submodules\nmay need some redesign anyway as the submodule\ncommands were designed with a strict \"one working\ntree only\" assumption.\n\nSubmodule URLs  are stored in 3 places:\n A) In the .gitmodules file of the superproject\n B) In the option submodule.<name>.URL in the superproject\n C) In the remote.origin.URL in the submodule\n\nA) is a recommendation from the superproject to make life\nof downstream easier to find and setup the whole thing.\nYou can ignore that if you want, though generally a caring\nupstream provides good URLs here.\n\nC) is where we actually fetch from (and hope it has all\nthe sha1s that are recorded as gitlinks in the superproejct)\n\nB) seems like a hack to enable the workflow as below:\n\nCurrent workflow for handling submodule URLs:\n 1) Clone the superproject\n 2) Run git submodule init on desired submodules\n 3) Inspect .git/config to see if any submodule URL needs adaption\n 4) Run git submodule update to obtain the submodules from\n    the configured place\n 5) In case of superproject adapting the URL\n    -> git submodule sync, which overwrites the submodule.<name>.URL in the\n    superprojects .git/config as well as configuring the\nremote.\"$remote\".url in the submodule\n 6) In case of users desire to change the URL\n    -> No one command to solve it; possible workaround: edit\n    .gitmodules and git submodule sync, or configure  the submodule.<name>.URL\n    in the superprojects .git/config as well as configuring the\nremote.\"$remote\".url in\n    the submodule separately. Although just changing the submodules remote works\n    just as well (until you remove and re-clone the submodule)\n\nOne could imagine another workflow:\n 1) clone the superproject, which creates empty repositories for the\n    submodules\n (2) from the prior workflow is gone\n 3) instead of inspecting .git/config you can directly manipulate the\n    remote.$remote.url configuration in the submodule.\n 4) Run git submodule update to obtain the submodules from\n    the configured place\n\nThe current workflow is setup that way because historically you had\nthe submodules .git dir inside the submodule, which would be gone\nif you deleted a submodule. So if you later checkout an earlier version'\nthat had a submodule, you are missing the objects and more importantly\nconfiguration where to get them from.\n\nThis is now fixed by keeping the actual submodules git dir inside\nthe superprojects git dir.\n\n\n>\n>\n>> After playing with this series a bit more, I actually like the UI as it is an\n>> easy mental model \"submodules behave completely independent\".\n>>\n>> However in 3/4 you said:\n>>\n>> + - `submodule.*` in current state should not be shared because the\n>> +   information is tied to a particular version of .gitmodules in a\n>> +   working directory.\n>>\n>> This is already a problem with say different branches/versions.\n>> That has been solved by duplicating that information to .git/config\n>> as a required step. (I don't like that approach, as it is super confusing\n>> IMHO)\n>\n> Hmm.. I didn't realize this. But then I have never given much thought\n> about submodules, probably because I have an alternative solution for\n> it (or some of its use cases) anyway :)\n\nWhat is that?\n\n>\n> OK so it's already a problem. But if we keep sharing submodule stuff\n> in .git/config, there's a _new_ problem: when you \"submodule init\" a\n> worktree, .git/config is now tailored for the current worktree, when\n> you move back to the previous worktree, you need to \"submodule init\"\n> again.\n\n\"Moving back\" sounds like you use the worktree feature for short lived\nthings only. (e.g. in the man page you refer to the hot fix your boss wants\nyou to make urgently).\n\nI thought the worktree feature is more useful for long term \"branches\",\ne.g. I have one worktree of git now that tracks origin/master so I can\nuse that to \"make install\" to repair my local broken version of git.\n\n(I could have a worktree \"continuous integration\", where I only run tests\nin. I could have one worktree for Documentation changes only.)\n\nThis long lived stuff probably doesn't make sense for the a single\nrepository, but in combination with submodules (which is another way\nto approach the \"sparse/narrow\" desire of a large project), I think\nthat makes sense, because the \"continuous integration\" shares a lot\nof submodules with my \"regular everyday hacking\" or the \"I need to\ntest my colleague work now\" worktree.\n\n> So moving to multiple worktrees setup changes how the user uses\n> submodule, not good in my opinion.\n\nBecause the submodule user API is built on the strong assumption\nof \"one working tree only\", we have to at least slightly adapt.\n\nSo instead of cloning a submodule in a worktree we could just\nsetup a submodule worktree as well there?\n(i.e. that saves us network as well as disk)\n\n>\n> If you have a grand plan to make submodule work at switching branches\n> (without reinit) and if it happens to work the same way when we have\n> multiple worktrees, great.\n\nEh, I am still working on the master plan. ;)\nThe insights on how worktrees handles stuff helps me shape it though. :)\n\nIf you switch a branch (or to any sha1), the submodule currently stays\n\"as-is\" and may be updated using \"submodule update\", which goes through\nthe list of existing (checked out) submodules and checks them out to the\nsha1 pointed to by the superprojects gitlink.\n\n>\n>> I am back to the drawing board for the submodule side of things,\n>> but I guess this series could be used once we figure out how to\n>> have just one object database for a submodule.\n>\n> I would leave this out for now. Let's make submodule work with\n> multiple worktrees first (and see how the users react to this). Then\n> we can try to share object database. Object database and refs are tied\n> closely together so you may run into other problems soon.\n\nI see. The normal for submodules is to be in detached HEAD though.\n\nThe user can of course checkout branches or things in there, but\nthe \"submodule update\" operations do not go to a branch for you.\n\n\n----\nAnother (slightly offtopic) observation on the similarity of worktree\nand submodules: There is no good way implemented to remove one.\n\nFor submodules there is deinit both removes the working tree as well\nas the configuration indicating the existence (Note: the git dir still exists\nfor the submodule). Though that sounds like what we need to save us\nnetwork traffic the next time we need the submodule. Although going\nthrough the code I need to test that a bit more later today to see how\nfail safe it is.\nOn the submodule side, it often gets confusing what you want to remove\n(local checkout of the submodule, or the gitlink or both).\n\nFor worktrees there is no \"worktree rm\" as it would probably promise\na bit more than the man pages suggestion of rm -rf $worktree && git\nworktree prune.\n\nThanks,\nStefan\n\n> --\n> Duy\n"},{"id":"292296","messageId":"20160727041058.GA9015@wheezy.local","threadId":"42882","inReplyTo":"20160720172419.25473-4-pclouds@gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Max Kirillov","fromEmail":"max@max630.net","sentAt":"2016-07-27T04:10:58Z","receivedAt":"2016-07-27T04:18:23Z","isPatch":true,"sender":{"key":"max@max630.net","avatar":"https://avatars.githubusercontent.com/u/381560?v=4"},"body":"Hi.\n\nOn Wed, Jul 20, 2016 at 07:24:18PM +0200, Nguyễn Thái Ngọc Duy wrote:\n> + - `remote.*` added by submodules may be per working directory as\n> +   well, unless you are sure remotes from all possible submodules in\n> +   history are consistent.\n...\n> @@ -1114,7 +1114,7 @@ cmd_sync()\n>  \t\t\t\tsanitize_submodule_env\n>  \t\t\t\tcd \"$sm_path\"\n>  \t\t\t\tremote=$(get_default_remote)\n> -\t\t\t\tgit config remote.\"$remote\".url \"$sub_origin_url\"\n> +\t\t\t\tgit config --worktree remote.\"$remote\".url \"$sub_origin_url\"\n>  \n>  \t\t\t\tif test -n \"$recursive\"\n>  \t\t\t\tthen\n\nI don't think remote.* should be per-worktree. \n\n* note that it is sumodule repository, not superproject. It\n  does not even have to have multiple worktrees.\n* it is quite bad to have it different in worktree, because\n  git fetch then results in different ref updates depending\n  on where it was called. So whatever issue it was intended\n  to solve, it hardly made things better.\n* I'm not sure I know all use cases of \"submodule sync\",\n  but as far as I understand, it should be called when the\n  submodule repository stays the \"same\" (however user\n  defines the \"same\"), but older url does not work for some\n  reason. Then I think it is correct to change the remote\n  url for all worktrees.\n\n-- \nMax\n"},{"id":"292318","messageId":"5798C744.5080308@gmail.com","threadId":"42882","inReplyTo":"CAGZ79kYGj7q=SQyHvFdmXasJppTVw56xSBMiSERdx22B+A68gQ@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Jakub Narębski","fromEmail":"jnareb@gmail.com","sentAt":"2016-07-27T14:37:56Z","receivedAt":"2016-07-27T14:38:20Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"W dniu 2016-07-26 o 20:15, Stefan Beller pisze:\n> On Tue, Jul 26, 2016 at 10:20 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n>> On Tue, Jul 26, 2016 at 1:25 AM, Stefan Beller <sbeller@google.com> wrote:\n>>> So what is the design philosophy in worktrees? How much independence does\n>>> one working tree have?\n>>\n>> git-worktree started out as an alternative for git-stash: hmm.. i need\n>> to make some changes in another branch, okay let's leave this worktree\n>> (with all its messy stuff) as-is, create another worktree, make those\n>> changes, then delete the worktree and go back here. There's already\n>> another way of doing that without git-stash: you clone the repo, fix\n>> your stuff, push back and delete the new repo.\n>>\n>> I know I have not really answered your questions. But I think it gives\n>> an idea what are the typical use cases for multiple worktrees. How\n>> much independence would need to be decided case-by-case, I think.\n> \n> Thanks!\n\nHopefully the Git User's Survey 2016 would answer what people really\nuse worktrees for, and what use submodules for.  You are welcome to\nsubmit proposed questions for the survey:\n  http://thread.gmane.org/gmane.comp.version-control.git/299032\n\n> My sudden interest in worktrees came up when I learned the\n> `--reference` flag for submodule operations is broken for\n> our use case, and instead of fixing the `--reference` flag,\n> I think the worktree approach is generally saner (i.e. with the\n> references you may have nasty gc issues IIUC, but in the\n> worktree world gc knows about all the working trees, detached\n> heads and branches.)\n\nI think the problem with `--reference` is that it does not\nsetup backreferences to prevent gc removing borrowed objects;\nwhich is a hard problem to solve, except for limited cases...\nlike git-worktree.\n \n> So I think the current workflow for submodules\n> may need some redesign anyway as the submodule\n> commands were designed with a strict \"one working\n> tree only\" assumption.\n> \n> Submodule URLs  are stored in 3 places:\n>  A) In the .gitmodules file of the superproject\n>  B) In the option submodule.<name>.URL in the superproject\n>  C) In the remote.origin.URL in the submodule\n> \n> A) is a recommendation from the superproject to make life\n> of downstream easier to find and setup the whole thing.\n> You can ignore that if you want, though generally a caring\n> upstream provides good URLs here.\n\nAlso, this URL might have change if the repository moves\nto other server; even when checking out ancient version\nwe usually want to use current URL, not the one in currently\nchecked-out .gitmodules file.\n \n> C) is where we actually fetch from (and hope it has all\n> the sha1s that are recorded as gitlinks in the superproject)\n\nIs it? Or is it only the case if you do `git fetch` or\nequivalent from within inside of submodule? You can fetch\nupdates using `git submodule ...` from supermodule, isn't it?\nBut I might be wrong here.\n\nAlso: if .git file is gitfile link, do submodule even has\nit's own configuration file?\n\n> \n> B) seems like a hack to enable the workflow as below:\n\nIt has overloaded meaning, being used both for current URL\nof submodule as seen in supermodule, AND that submodule\nis checked out / needs to be checked out in the worktree\nof a supermodule.  There might be the case when you check\nout (in given worktree) a version of a supermodule that\ndo not include submodule at all, but you want to know that\nwhen going back, this submodule is to be checked out (or not).\n\nThe second information needs to be per-worktree. How to\nsolve it, be it per-worktree configuration (not shared),\nor a special configuration variable, or worktree having\nunshared copy of configuration -- this what is discussed.\n\n> Current workflow for handling submodule URLs:\n>  1) Clone the superproject\n>  2) Run git submodule init on desired submodules\n\nOr 1-2) clone the superproject recursively, with all its\nsubmodules.\n\n>  3) Inspect .git/config to see if any submodule URL needs adaption\n\nWhich is usually not needed.\n\n>  4) Run git submodule update to obtain the submodules from\n>     the configured place\n\nOr 2+4) run `git submodule update --init`\n\n>  5) In case of superproject adapting the URL\n>     -> git submodule sync, which overwrites the submodule.<name>.URL in the\n>     superprojects .git/config as well as configuring the\n>     remote.\"$remote\".url in the submodule\n\nThis takes information from current .gitmodules, isn't it?\n\n>  6) In case of users desire to change the URL\n>     -> No one command to solve it; possible workaround: edit\n>     .gitmodules and git submodule sync, or configure  the submodule.<name>.URL\n>     in the superprojects .git/config as well as configuring the remote.\"$remote\".url in\n>     the submodule separately. Although just changing the submodules remote works\n>     just as well (until you remove and re-clone the submodule)\n[...]\n\n\n> \"Moving back\" sounds like you use the worktree feature for short lived\n> things only. (e.g. in the man page you refer to the hot fix your boss wants\n> you to make urgently).\n> \n> I thought the worktree feature is more useful for long term \"branches\",\n> e.g. I have one worktree of git now that tracks origin/master so I can\n> use that to \"make install\" to repair my local broken version of git.\n> \n> (I could have a worktree \"continuous integration\", where I only run tests\n> in. I could have one worktree for Documentation changes only.)\n> \n> This long lived stuff probably doesn't make sense for the a single\n> repository, but in combination with submodules (which is another way\n> to approach the \"sparse/narrow\" desire of a large project), I think\n> that makes sense, because the \"continuous integration\" shares a lot\n> of submodules with my \"regular everyday hacking\" or the \"I need to\n> test my colleague work now\" worktree.\n\nOne thing that git-worktree would be very useful, if it could work\nwith submodules: you could use separate worktrees to easily test\nif the supermodule works with and without its submodules present.\n \n[...]\n> If you switch a branch (or to any sha1), the submodule currently stays\n> \"as-is\" and may be updated using \"submodule update\", which goes through\n> the list of existing (checked out) submodules and checks them out to the\n> sha1 pointed to by the superprojects gitlink.\n\nWhich might be simply a problem that submodule UI is not mature enough.\nI would like to see automatic switch of submodule contents, if \nconfigured so.\n\n-- \nJakub Narębski\n\n"},{"id":"292319","messageId":"5798C7D1.4010405@gmail.com","threadId":"42882","inReplyTo":"20160727041058.GA9015@wheezy.local","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Jakub Narębski","fromEmail":"jnareb@gmail.com","sentAt":"2016-07-27T14:40:17Z","receivedAt":"2016-07-27T14:40:36Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"W dniu 2016-07-27 o 06:10, Max Kirillov pisze:\n> Hi.\n> \n> On Wed, Jul 20, 2016 at 07:24:18PM +0200, Nguyễn Thái Ngọc Duy wrote:\n>> + - `remote.*` added by submodules may be per working directory as\n>> +   well, unless you are sure remotes from all possible submodules in\n>> +   history are consistent.\n> ...\n>> @@ -1114,7 +1114,7 @@ cmd_sync()\n>>  \t\t\t\tsanitize_submodule_env\n>>  \t\t\t\tcd \"$sm_path\"\n>>  \t\t\t\tremote=$(get_default_remote)\n>> -\t\t\t\tgit config remote.\"$remote\".url \"$sub_origin_url\"\n>> +\t\t\t\tgit config --worktree remote.\"$remote\".url \"$sub_origin_url\"\n>>  \n>>  \t\t\t\tif test -n \"$recursive\"\n>>  \t\t\t\tthen\n> \n> I don't think remote.* should be per-worktree. \n> \n> * note that it is sumodule repository, not superproject. It\n>   does not even have to have multiple worktrees.\n> * it is quite bad to have it different in worktree, because\n>   git fetch then results in different ref updates depending\n>   on where it was called. So whatever issue it was intended\n>   to solve, it hardly made things better.\n> * I'm not sure I know all use cases of \"submodule sync\",\n>   but as far as I understand, it should be called when the\n>   submodule repository stays the \"same\" (however user\n>   defines the \"same\"), but older url does not work for some\n>   reason. Then I think it is correct to change the remote\n>   url for all worktrees.\n\nBut... I don't know how sane it is, and if anybody uses it,\nbut one might want to use different repositories (different\nforks) for different branches, and thus different worktrees.\nFor example the 'next' branch might want to switch to X.Org,\nbecause XFree86 is moribund, but keep the old repo for 'maint',\nor something like that ;-)\n\n-- \nJakub Narębski\n\n"},{"id":"292320","messageId":"CACsJy8D+pyKXeci8xs+XBmVRat0QjnT94iqPuSz1r8Zbv780jw@mail.gmail.com","threadId":"42882","inReplyTo":"20160727041058.GA9015@wheezy.local","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-27T14:49:05Z","receivedAt":"2016-07-27T14:49:48Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Jul 27, 2016 at 6:10 AM, Max Kirillov <max@max630.net> wrote:\n> Hi.\n>\n> On Wed, Jul 20, 2016 at 07:24:18PM +0200, Nguyễn Thái Ngọc Duy wrote:\n>> + - `remote.*` added by submodules may be per working directory as\n>> +   well, unless you are sure remotes from all possible submodules in\n>> +   history are consistent.\n> ...\n>> @@ -1114,7 +1114,7 @@ cmd_sync()\n>>                               sanitize_submodule_env\n>>                               cd \"$sm_path\"\n>>                               remote=$(get_default_remote)\n>> -                             git config remote.\"$remote\".url \"$sub_origin_url\"\n>> +                             git config --worktree remote.\"$remote\".url \"$sub_origin_url\"\n>>\n>>                               if test -n \"$recursive\"\n>>                               then\n>\n> I don't think remote.* should be per-worktree.\n>\n> * note that it is sumodule repository, not superproject.\n\nAh.. silly me, I thought all these were about supermodule. Yes it\nmakes more sense then to share remote.* (just like it's set up after\nclone).\n\n>   It does not even have to have multiple worktrees.\n\nBut we can turn a submodule into multiple worktrees after \"submodule\ninit\" and I don't think sharing remote.* is a problem even in that\ncase.\n\n> * it is quite bad to have it different in worktree, because\n>   git fetch then results in different ref updates depending\n>   on where it was called. So whatever issue it was intended\n>   to solve, it hardly made things better.\n> * I'm not sure I know all use cases of \"submodule sync\",\n>   but as far as I understand, it should be called when the\n>   submodule repository stays the \"same\" (however user\n>   defines the \"same\"), but older url does not work for some\n>   reason. Then I think it is correct to change the remote\n>   url for all worktrees.\n-- \nDuy\n"},{"id":"292324","messageId":"CACsJy8Dhd2YLmNoRN=j0PQeyG+8=8MALsiw611HMhi2zk_8ouQ@mail.gmail.com","threadId":"42882","inReplyTo":"CAGZ79kYGj7q=SQyHvFdmXasJppTVw56xSBMiSERdx22B+A68gQ@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-27T15:40:48Z","receivedAt":"2016-07-27T15:41:23Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Jul 26, 2016 at 8:15 PM, Stefan Beller <sbeller@google.com> wrote:\n>> How to store stuff in .git is the implementation details that the user\n>> does not care about.\n>\n> They do unfortunately. :(\n\nWell.. i mean the structure of .git. If .git gets big, yeah many\npeople will get pissed.\n\n> My sudden interest in worktrees came up when I learned the\n> `--reference` flag for submodule operations is broken for\n> our use case, and instead of fixing the `--reference` flag,\n> I think the worktree approach is generally saner (i.e. with the\n\nI don't know exactly what that --reference problem is, but keep in\nmind you still have to support single-worktree use case. If it's\nbroken in single-worktree, somebody still has to fix it.\n\n> references you may have nasty gc issues IIUC, but in the\n> worktree world gc knows about all the working trees, detached\n> heads and branches.)\n\nTrue, but not yet. git-gc now does not know about all detached heads\nand reflogs and have cause grief for some people. This should be fixed\nsoon.\n\n>> As long as we keep the behavior the same (they\n>> can still \"git submodule init\" and stuff in the new worktree), sharing\n>> the same object store makes sense (pros: lower disk consumption, cons:\n>> none).\n>\n> So I think the current workflow for submodules\n> may need some redesign anyway as the submodule\n> commands were designed with a strict \"one working\n> tree only\" assumption.\n>\n> Submodule URLs  are stored in 3 places:\n>  A) In the .gitmodules file of the superproject\n>  B) In the option submodule.<name>.URL in the superproject\n>  C) In the remote.origin.URL in the submodule\n>\n> A) is a recommendation from the superproject to make life\n> of downstream easier to find and setup the whole thing.\n> You can ignore that if you want, though generally a caring\n> upstream provides good URLs here.\n>\n> C) is where we actually fetch from (and hope it has all\n> the sha1s that are recorded as gitlinks in the superproejct)\n>\n> B) seems like a hack to enable the workflow as below:\n>\n> Current workflow for handling submodule URLs:\n>  1) Clone the superproject\n>  2) Run git submodule init on desired submodules\n>  3) Inspect .git/config to see if any submodule URL needs adaption\n>  4) Run git submodule update to obtain the submodules from\n>     the configured place\n>  5) In case of superproject adapting the URL\n>     -> git submodule sync, which overwrites the submodule.<name>.URL in the\n>     superprojects .git/config as well as configuring the\n> remote.\"$remote\".url in the submodule\n>  6) In case of users desire to change the URL\n>     -> No one command to solve it; possible workaround: edit\n>     .gitmodules and git submodule sync, or configure  the submodule.<name>.URL\n>     in the superprojects .git/config as well as configuring the\n> remote.\"$remote\".url in\n>     the submodule separately. Although just changing the submodules remote works\n>     just as well (until you remove and re-clone the submodule)\n>\n> One could imagine another workflow:\n>  1) clone the superproject, which creates empty repositories for the\n>     submodules\n>  (2) from the prior workflow is gone\n>  3) instead of inspecting .git/config you can directly manipulate the\n>     remote.$remote.url configuration in the submodule.\n>  4) Run git submodule update to obtain the submodules from\n>     the configured place\n>\n> The current workflow is setup that way because historically you had\n> the submodules .git dir inside the submodule, which would be gone\n> if you deleted a submodule. So if you later checkout an earlier version'\n> that had a submodule, you are missing the objects and more importantly\n> configuration where to get them from.\n>\n> This is now fixed by keeping the actual submodules git dir inside\n> the superprojects git dir.\n\nHmm.. sounds good, but I'm no judge when it comes to submodules :)\n\n>> Hmm.. I didn't realize this. But then I have never given much thought\n>> about submodules, probably because I have an alternative solution for\n>> it (or some of its use cases) anyway :)\n>\n> What is that?\n\nNarrow clone (making progress but not there yet). I know it does not\ncover all cases (e.g. submodule can provide separate access control,\nand even using different dvcs system in theory).\n\n>> OK so it's already a problem. But if we keep sharing submodule stuff\n>> in .git/config, there's a _new_ problem: when you \"submodule init\" a\n>> worktree, .git/config is now tailored for the current worktree, when\n>> you move back to the previous worktree, you need to \"submodule init\"\n>> again.\n>\n> \"Moving back\" sounds like you use the worktree feature for short lived\n> things only. (e.g. in the man page you refer to the hot fix your boss wants\n> you to make urgently).\n>\n> I thought the worktree feature is more useful for long term \"branches\",\n> e.g. I have one worktree of git now that tracks origin/master so I can\n> use that to \"make install\" to repair my local broken version of git.\n\nI use it for both. Sometimes you just want to fix something and not\nmess up your current worktree.\n\n> (I could have a worktree \"continuous integration\", where I only run tests\n> in. I could have one worktree for Documentation changes only.)\n>\n> This long lived stuff probably doesn't make sense for the a single\n> repository,\n\nIt does. You can switch branches in all worktrees. I have a worktree\nspecifically for building mingw32 stuff (separate config.mak and\nstuff). When I'm done with a branch on my normal worktree, I could\nmove over there, check out the same branch then try mingw32 build. If\nit fails I can fix it right right there and update the branch. When\neverything is ok, I just move back to my normal worktree and continue.\n\nYou can achieve the same thing with multiple clones, but it's\ninconvenient (you need to fetch and push...) and multiple clones take\nup more space.\n\n>> So moving to multiple worktrees setup changes how the user uses\n>> submodule, not good in my opinion.\n>\n> Because the submodule user API is built on the strong assumption\n> of \"one working tree only\", we have to at least slightly adapt.\n>\n> So instead of cloning a submodule in a worktree we could just\n> setup a submodule worktree as well there?\n> (i.e. that saves us network as well as disk)\n\nYou still need to clone once then somehow associate the clone with a\nsubmodule, I think.\n\n> For worktrees there is no \"worktree rm\" as it would probably promise\n> a bit more than the man pages suggestion of rm -rf $worktree && git\n> worktree prune.\n\nOh there will be, sooooon :D I prefer not to do that command sequence\nmanually, especially when \"rm -rf\" is involved. \"git worktree remove\"\ncan refuse to delete when your worktree is dirty so you don't\naccidentally rm and cry later.\n-- \nDuy\n"},{"id":"292331","messageId":"CAGZ79kZtfpmZRyOfF6NMhCqjNBgfH1bX+xWUCEdedQGbCbL71Q@mail.gmail.com","threadId":"42882","inReplyTo":"5798C744.5080308@gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-07-27T16:53:13Z","receivedAt":"2016-07-27T16:53:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Jakub wrote:\n> I think the problem with `--reference` is that it does not\n> setup backreferences to prevent gc removing borrowed objects;\n> which is a hard problem to solve, except for limited cases...\n> like git-worktree.\n\nRight. And instead of solving the reference problem, I'd\nrather solve the worktree problem as I think it yields more?\n\n>\n>> So I think the current workflow for submodules\n>> may need some redesign anyway as the submodule\n>> commands were designed with a strict \"one working\n>> tree only\" assumption.\n>>\n>> Submodule URLs  are stored in 3 places:\n>>  A) In the .gitmodules file of the superproject\n>>  B) In the option submodule.<name>.URL in the superproject\n>>  C) In the remote.origin.URL in the submodule\n>>\n>> A) is a recommendation from the superproject to make life\n>> of downstream easier to find and setup the whole thing.\n>> You can ignore that if you want, though generally a caring\n>> upstream provides good URLs here.\n>\n> Also, this URL might have change if the repository moves\n> to other server; even when checking out ancient version\n> we usually want to use current URL, not the one in currently\n> checked-out .gitmodules file.\n\nRight.\n\n>\n>> C) is where we actually fetch from (and hope it has all\n>> the sha1s that are recorded as gitlinks in the superproject)\n>\n> Is it? Or is it only the case if you do `git fetch` or\n> equivalent from within inside of submodule? You can fetch\n> updates using `git submodule ...` from supermodule, isn't it?\n> But I might be wrong here.\n\nIf you call `submodule update` in the  superproject\nit actually just does a `(cd $submodule && git fetch)`.\n\nAnd in the submodule we have a .git file pointing to\nthe superprojects \".git/modules/<name>/\" which is a full\nblown git dir, i.e. it has its own config, HEAD etc.\n\n>\n> Also: if .git file is gitfile link, do submodule even has\n> it's own configuration file?\n\nYes they do.\n\n>\n>>\n>> B) seems like a hack to enable the workflow as below:\n>\n> It has overloaded meaning, being used both for current URL\n> of submodule as seen in supermodule, AND that submodule\n> is checked out / needs to be checked out in the worktree\n> of a supermodule.  There might be the case when you check\n> out (in given worktree) a version of a supermodule that\n> do not include submodule at all, but you want to know that\n> when going back, this submodule is to be checked out (or not).\n\nI am currently working on solving that with a patch series, that\nallows 2 settings. The URL will be used only to overwrite the\nURL from the .gitmodules file and another setting will be used\nto determine if we want to checkout the submodule.\n\n>\n> The second information needs to be per-worktree. How to\n> solve it, be it per-worktree configuration (not shared),\n> or a special configuration variable, or worktree having\n> unshared copy of configuration -- this what is discussed.\n\n>\n>> Current workflow for handling submodule URLs:\n>>  1) Clone the superproject\n>>  2) Run git submodule init on desired submodules\n>\n> Or 1-2) clone the superproject recursively, with all its\n> submodules.\n\nOnly if the URLs are setup properly.\n\n>\n>>  3) Inspect .git/config to see if any submodule URL needs adaption\n>\n> Which is usually not needed.\n\nYeah, I should have added the assertion that the .gitmodules\nmay be out of date or such for this workflow to make sense.\nUsually just go with recursive clone.\n\n>>\n>> This long lived stuff probably doesn't make sense for the a single\n>> repository, but in combination with submodules (which is another way\n>> to approach the \"sparse/narrow\" desire of a large project), I think\n>> that makes sense, because the \"continuous integration\" shares a lot\n>> of submodules with my \"regular everyday hacking\" or the \"I need to\n>> test my colleague work now\" worktree.\n>\n> One thing that git-worktree would be very useful, if it could work\n> with submodules: you could use separate worktrees to easily test\n> if the supermodule works with and without its submodules present.\n\nOh! Yeah that makes sense!\n\n>\n> [...]\n>> If you switch a branch (or to any sha1), the submodule currently stays\n>> \"as-is\" and may be updated using \"submodule update\", which goes through\n>> the list of existing (checked out) submodules and checks them out to the\n>> sha1 pointed to by the superprojects gitlink.\n>\n> Which might be simply a problem that submodule UI is not mature enough.\n> I would like to see automatic switch of submodule contents, if\n> configured so.\n\nMe too. Once upon a time Jens pushed for that with a series found at:\nhttps://github.com/jlehmann/git-submod-enhancements/tree/git-checkout-recurse-submodules\n"},{"id":"293005","messageId":"CAGZ79kZAwV+w0XTgacnAT-iPO3bzYapt2DdR22JJCBz_up6FJw@mail.gmail.com","threadId":"42882","inReplyTo":"CACsJy8Dhd2YLmNoRN=j0PQeyG+8=8MALsiw611HMhi2zk_8ouQ@mail.gmail.com","subject":"Re: [PATCH v4 3/4] submodule: support running in multiple worktree setup","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-08-03T21:47:53Z","receivedAt":"2016-08-03T21:48:24Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jul 27, 2016 at 8:40 AM, Duy Nguyen <pclouds@gmail.com> wrote:\n> On Tue, Jul 26, 2016 at 8:15 PM, Stefan Beller <sbeller@google.com> wrote:\n>>> How to store stuff in .git is the implementation details that the user\n>>> does not care about.\n>>\n>> They do unfortunately. :(\n>\n> Well.. i mean the structure of .git. If .git gets big, yeah many\n> people will get pissed.\n>\n>> My sudden interest in worktrees came up when I learned the\n>> `--reference` flag for submodule operations is broken for\n>> our use case, and instead of fixing the `--reference` flag,\n>> I think the worktree approach is generally saner (i.e. with the\n>\n> I don't know exactly what that --reference problem is, but keep in\n> mind you still have to support single-worktree use case. If it's\n> broken in single-worktree, somebody still has to fix it.\n\nSo --reference let's you point to a directory such that you can clone\nwith less data transmission (borrow objects from the local reference).\n\nFor submodules this is not per submodule, i.e.\n\n  git clone --recursive --reference <path>\n\nwill only look at that <path> to borrow for the superproject and all submodules.\nBut submodules are usually different projects, so you don't find their objects\nin the superprojects reference path.\n\nOne way out would be to extend the path appropriately (assuming the same\nsubmodule structure in the reference repository).\n\nAnother way would be to extend the reference mechanism to look for\nobjects in the given path and any submodule of that path. Then the submodule\nlayout can change and --reference is still super effective.\n\nMy chose way was to look at the submodule support for worktrees, as that\nwill hopefully be less brittle w.r.t. gc eventually.\n\n>\n>> The current workflow is setup that way because historically you had\n>> the submodules .git dir inside the submodule, which would be gone\n>> if you deleted a submodule. So if you later checkout an earlier version'\n>> that had a submodule, you are missing the objects and more importantly\n>> configuration where to get them from.\n>>\n>> This is now fixed by keeping the actual submodules git dir inside\n>> the superprojects git dir.\n>\n> Hmm.. sounds good, but I'm no judge when it comes to submodules :)\n\nyeah I'll try to get feedback from the submodule people. :)\n\n>\n>>> Hmm.. I didn't realize this. But then I have never given much thought\n>>> about submodules, probably because I have an alternative solution for\n>>> it (or some of its use cases) anyway :)\n>>\n>> What is that?\n>\n> Narrow clone (making progress but not there yet). I know it does not\n> cover all cases (e.g. submodule can provide separate access control,\n> and even using different dvcs system in theory).\n\nheh, ok. Yeah ACLs are the big thing here, so we'd rather go with submodules.\n\n>\n>>> OK so it's already a problem. But if we keep sharing submodule stuff\n>>> in .git/config, there's a _new_ problem: when you \"submodule init\" a\n>>> worktree, .git/config is now tailored for the current worktree, when\n>>> you move back to the previous worktree, you need to \"submodule init\"\n>>> again.\n>>\n>> \"Moving back\" sounds like you use the worktree feature for short lived\n>> things only. (e.g. in the man page you refer to the hot fix your boss wants\n>> you to make urgently).\n>>\n>> I thought the worktree feature is more useful for long term \"branches\",\n>> e.g. I have one worktree of git now that tracks origin/master so I can\n>> use that to \"make install\" to repair my local broken version of git.\n>\n> I use it for both. Sometimes you just want to fix something and not\n> mess up your current worktree.\n\nI tried worktrees in my daily workflow and the issue for me is my editor\nthat is worktree agnostic.  As I tried using worktree for different git related\npatch series', the set of files I need to look at are the same in the\ndifferent work trees\n\nWhen switching branches the files are still at the same place, such that\nthe editor, that has a bunch of files open, will just reload the files and you\ndon't need to open/close files in the editor.\nWith worktrees you need to open/close all files that you intend to touch in\nthat worktree, which I dislike as an extra step.\n\n>\n>> (I could have a worktree \"continuous integration\", where I only run tests\n>> in. I could have one worktree for Documentation changes only.)\n>>\n>> This long lived stuff probably doesn't make sense for the a single\n>> repository,\n>\n> It does. You can switch branches in all worktrees. I have a worktree\n> specifically for building mingw32 stuff (separate config.mak and\n> stuff). When I'm done with a branch on my normal worktree, I could\n> move over there, check out the same branch then try mingw32 build. If\n> it fails I can fix it right right there and update the branch. When\n> everything is ok, I just move back to my normal worktree and continue.\n\nSo you use different worktrees for different purposes i.e. editing always\nhappens in the same, but testing or real hot fixes go into a separate\nworktree?\n\n\n>> So instead of cloning a submodule in a worktree we could just\n>> setup a submodule worktree as well there?\n>> (i.e. that saves us network as well as disk)\n>\n> You still need to clone once then somehow associate the clone with a\n> submodule, I think.\n\n(In the single worktree case) if you already have the submodule cloned,\nbut delete it from the working tree, you still keep the repo in .git. Later\nwhen you decide to checkout the submodule again, you don't have to clone\nit again, but just checkout.\n\nI imagine the same is in multiple worktrees. Although you may want to fetch\na different remote first for a different worktree (as it may have configured a\ndifferent remote URL for the worktree)\n\n>\n>> For worktrees there is no \"worktree rm\" as it would probably promise\n>> a bit more than the man pages suggestion of rm -rf $worktree && git\n>> worktree prune.\n>\n> Oh there will be, sooooon :D I prefer not to do that command sequence\n> manually, especially when \"rm -rf\" is involved. \"git worktree remove\"\n> can refuse to delete when your worktree is dirty so you don't\n> accidentally rm and cry later.\n\nyeah that was my line of thinking :)\n\nThanks,\nStefan\n\n> --\n> Duy\n"}]}