{"thread":{"id":"56920","subject":"[PATCH v6 0/5] teach submodules to know they're submodules","startedAt":"2021-11-17T00:57:10Z","lastAt":"2022-03-15T21:19:48Z","messageCount":64,"participants":["Emily Shaffer","Ævar Arnfjörð Bjarmason","Jonathan Tan","Junio C Hamano","Jonathan Nieder","Glen Choo","Eric Sunshine"],"isPatch":true,"patchVersion":6,"patchTotal":5},"messages":[{"id":"441377","messageId":"20211117005701.371808-1-emilyshaffer@google.com","threadId":"56920","inReplyTo":null,"subject":"[PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-17T00:56:56Z","receivedAt":"2021-11-17T00:57:10Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"For the original cover letter, see\nhttps://lore.kernel.org/git/20210611225428.1208973-1-emilyshaffer%40google.com.\n\nCI run: https://github.com/nasamuffin/git/actions/runs/1469463328\n\nSince v5:\n\nA couple things. Firstly, a semantics change *back* to the semantics of\nv3 - we map from gitdir to gitdir, *not* from common dir to common dir,\nso that theoretically a submodule with multiple worktrees in multiple\nsuperproject worktrees will be able to figure out which worktree of the\nsuperproject it's in. (Realistically, that's not really possible right\nnow, but I'd like to change that soon.)\n\nSecondly, a rewording of comments and commit messages to indicate that\nthis isn't a cache of some expensive operation, but rather intended to\nbe the source of truth for all submodules. I also added a fifth commit\nrewriting `git rev-parse --show-superproject-working-tree` to\ndemonstrate what that means in practice - but from a practical\nstandpoint, I'm a little worried about that fifth patch. More details in\nthe patch 5 description.\n\nI did discuss Ævar's idea of relying on in-process filesystem digging to\nfind the superproject's gitdir with the rest of the Google team, but in\nthe end decided that there are some worries about filesystem digging in\nthis way (namely, some ugly interactions with network drives that are\nactually already an issue for Googler Linux machines). Plus, the allure\nof being able to definitively know that we're a submodule is pretty\nstrong. ;) But overall, this is the direction I'd prefer to keep going\nin, rather than trying to guess from the filesystem going forward.\n\nSince v4:\n\nThe only real change here is a slight semantics change to map from\n to . In every case\n*except* for when the superproject has a worktree, this changes nothing.\nFor the case when the superproject has a worktree, this means that now\nsubmodules will refer to the general superproject common dir (e.g. no\nworktree-specific refs or configs or whatnot).\n\nI *think* that because a submodule should exist in the context of the\ncommon dir, not the worktree gitdir, that is ok. However, it does mean\nit would be difficult to do something like sharing a config specific to\nthe worktree (the initial goal of this series).\n\n$ROOT/.git\n$ROOT/.git/config.superproject <- shared by $ROOT/.git/modules/sub\n$ROOT/.git/modules/sub <- points to $ROOT/.git\n$ROOT/.git/worktrees/wt\n$ROOT/.git/worktrees/wt/config.superproject <- contains a certain config-based pre-commit hook\n\nIf the submodule only knows about the common dir, that is tough, because\nthe submodule would basically have to guess which worktree it's in from\nits own path. There would be no way for '$WT/sub' to inherit\n'$ROOT/.git/worktrees/wt/config.superproject'.\n\nThat said... right now, we don't support submodules in worktrees very\nwell at all. A submodule in a worktree will get a brand new gitdir in\n$ROOT/.git/worktrees/modules/ (and that brand new gitdir would point to\nthe super's common dir). So I think we can punt on this entire question\nuntil we teach submodules and worktrees to play more gracefully together\n(it's on my long list...), and at that time we can probably introduce a\npointer from $ROOT/.git/modules/sub/worktrees/wt/ to\n$ROOT/.git/worktrees/wt/....\n\nOr, to summarize the long ramble above: \"this is still kind of weird\nwith worktrees, but let's fix it later when we fix worktrees more\nthoroughly\".\n\n(More rambling about worktree weirdness here:\nhttps://lore.kernel.org/git/YYRaII8YWVxlBqsF%40google.com )\n\n\nSince v3, a pretty major change: the semantics of\nsubmodule.superprojectGitDir has changed, to point from the submodule's\ngitdir to the superproject's gitdir (in v3 and earlier, we kept a path\nfrom the submodule's *worktree* to the superproject's gitdir instead).\nThis cleans up some of the confusions about the behavior when a\nsubmodule worktree moves around in the superproject's tree, or in a\nfuture when we support submodules having multiple worktrees.\n\nI also tried to simplify the tests to use 'test-tool path-utils\nrelative_path' everywhere - I think that makes them much more clear for\na test reader, but if you're reviewing and it isn't obvious what we're\ntesting for, please speak up.\n\nI think this is pretty mature and there was a lot of general agreement\nthat the gitdir->gitdir association was the way to go, so please be\nbrutal and look for nits, leaks, etc. this round ;)\n[/v4 cover letter]\n\nEmily Shaffer (5):\n t7400-submodule-basic: modernize inspect() helper\n introduce submodule.superprojectGitDir record\n submodule: record superproject gitdir during absorbgitdirs\n submodule: record superproject gitdir during 'update'\n submodule: use config to find superproject worktree\n\n Documentation/config/submodule.txt | 12 ++++\n builtin/submodule--helper.c | 11 +++\n git-submodule.sh | 15 ++++\n submodule.c | 108 ++++++++++++++++++++++++++++-\n t/t1500-rev-parse.sh | 9 +++\n t/t7400-submodule-basic.sh | 54 ++++++++-------\n t/t7406-submodule-update.sh | 27 ++++++++\n t/t7412-submodule-absorbgitdirs.sh | 82 +++++++++++++++++++++-\n 8 files changed, 290 insertions(+), 28 deletions(-)\n\nRange-diff against v5:\n1: 6ff10beaf2 = 1: f1b08a7057 t7400-submodule-basic: modernize inspect() helper\n2: d4f4627585 ! 2: d46c8439ab introduce submodule.superprojectGitDir record\n @@ Commit message\n \n By using a relative path instead of an absolute path, we can move the\n superproject directory around on the filesystem without breaking the\n - submodule's cache. And by using the path from gitdir to gitdir, we can\n + submodule's pointer. And by using the path from gitdir to gitdir, we can\n move the submodule within the superproject's tree structure without\n - breaking the submodule's cache, too. Finally, by pointing at\n - \"get_git_common_dir()\" instead of \"get_git_dir()\", we ensure the link\n - will refer to the parent repo, not to a specific worktree.\n + breaking the submodule's pointer, too. Finally, by pointing at the\n + superproject's worktree gitdir (if it exists), we ensure that we can\n + tell which worktree contains our submodule.\n \n Since this hint value is only introduced during new submodule creation\n via `git submodule add`, though, there is more work to do to allow the\n record to be created at other times.\n \n - If the new config is present, we can do some optional value-added\n - behavior, like letting \"git status\" print additional info about the\n - submodule's status in relation to its superproject, or like letting the\n - superproject and submodule share an additional config file separate from\n - either one's local config.\n + Once this new config is reliably in place, we can use it to know\n + definitively that we are working in a submodule, and to know which\n + superproject we are a submodule of. This allows us to do some\n + value-added behavior, like letting \"git status\" print additional info\n + about the submodule's status in relation to its superproject, or like\n + letting the superproject and submodule share an additional config file\n + separate from either one's local config.\n \n Signed-off-by: Emily Shaffer \n Helped-by: Junio C Hamano \n @@ Documentation/config/submodule.txt: submodule.alternateErrorStrategy::\n +\n +submodule.superprojectGitDir::\n +\tThe relative path from the submodule's gitdir to its superproject's\n -+\tcommon dir. When Git is run in a repository, it usually makes no\n ++\tgitdir. When Git is run in a repository, it usually makes no\n +\tdifference whether this repository is standalone or a submodule, but if\n +\tthis configuration variable is present, additional behavior may be\n +\tpossible, such as \"git status\" printing additional information about\n @@ builtin/submodule--helper.c: static int clone_submodule(struct module_clone_data\n \t\tgit_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n \t\t\t\t error_strategy);\n \n -+\tgit_config_set_in_file(p, \"submodule.superprojectGitdir\",\n -+\t\t\t relative_path(absolute_path(get_git_common_dir()),\n ++\t/*\n ++\t * Set the path from submodule's new gitdir to superproject's gitdir.\n ++\t * The latter may be a worktree gitdir. However, it is not possible for\n ++\t * the submodule to have a worktree-specific gitdir or config at clone\n ++\t * time, because \"extensions.worktreeConfig\" is only valid when set in\n ++\t * the local gitconfig, which the brand new submodule does not have yet.\n ++\t */\n ++\tgit_config_set_in_file(p, \"submodule.superprojectGitDir\",\n ++\t\t\t relative_path(absolute_path(get_git_dir()),\n +\t\t\t\t\t sm_gitdir, &sb));\n +\n \tfree(sm_alternate);\n @@ t/t7400-submodule-basic.sh: submodurl=$(pwd -P)\n +\t# Ensure that submodule.superprojectGitDir contains the path from the\n +\t# submodule's gitdir to the superproject's gitdir.\n +\n -+\tsuper_abs_gitdir=$(git -C \"$super_dir\" rev-parse --path-format=absolute --git-common-dir) &&\n -+\tsub_abs_gitdir=$(git -C \"$sub_dir\" rev-parse --path-format=absolute --git-common-dir) &&\n ++\tsuper_abs_gitdir=$(git -C \"$super_dir\" rev-parse --absolute-git-dir) &&\n ++\tsub_abs_gitdir=$(git -C \"$sub_dir\" rev-parse --absolute-git-dir) &&\n +\n +\t[ \"$(git -C \"$sub_dir\" config --get submodule.superprojectGitDir)\" = \\\n +\t \"$(test-tool path-utils relative_path \"$super_abs_gitdir\" \\\n3: 2dae297943 ! 3: 63ddaf5608 submodule: record superproject gitdir during absorbgitdirs\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *p\n \tchar *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL;\n \tstruct strbuf new_gitdir = STRBUF_INIT;\n \tconst struct submodule *sub;\n ++\tstruct config_set sub_cs;\n +\tstruct strbuf config_path = STRBUF_INIT, sb = STRBUF_INIT;\n ++\tint tmp;\n \n \tif (submodule_uses_worktrees(path))\n \t\tdie(_(\"relocate_gitdir for submodule '%s' with \"\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *p\n \n \trelocate_gitdir(path, real_old_git_dir, real_new_git_dir);\n \n -+\t/* cache pointer to superproject's gitdir */\n ++\t/*\n ++\t * Note location of superproject's gitdir. Because the submodule already\n ++\t * has a gitdir and local config, we can store this pointer from\n ++\t * worktree config to worktree config, if the submodule has\n ++\t * extensions.worktreeConfig set.\n ++\t */\n +\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n ++\tgit_configset_init(&sub_cs);\n ++\tgit_configset_add_file(&sub_cs, config_path.buf);\n ++\t/* return 0 indicates config was found - we have a worktree config */\n ++\tif (!git_configset_get_bool(&sub_cs, \"extensions.worktreeConfig\", &tmp))\n ++\t\tstrbuf_addstr(&config_path, \".worktree\");\n ++\n +\tgit_config_set_in_file(config_path.buf, \"submodule.superprojectGitdir\",\n -+\t\t\t relative_path(absolute_path(get_git_common_dir()),\n ++\t\t\t relative_path(absolute_path(get_git_dir()),\n +\t\t\t\t\t real_new_git_dir, &sb));\n +\n ++\tgit_configset_clear(&sub_cs);\n +\tstrbuf_release(&config_path);\n +\tstrbuf_release(&sb);\n \tfree(old_git_dir);\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorbing fails for a s\n +\t# absorb the git dir\n +\tgit submodule absorbgitdirs sub4 &&\n +\n -+\t# make sure the submodule cached the superproject gitdir correctly\n -+\tsubmodule_gitdir=\"$(git -C sub4 rev-parse --path-format=absolute --git-common-dir)\" &&\n -+\tsuperproject_gitdir=\"$(git rev-parse --path-format=absolute --git-common-dir)\" &&\n ++\t# make sure the submodule noted the superproject gitdir correctly\n ++\tsubmodule_gitdir=\"$(git -C sub4 rev-parse --absolute-git-dir)\" &&\n ++\tsuperproject_gitdir=\"$(git rev-parse --absolute-git-dir)\" &&\n +\n +\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n +\t\t\"$submodule_gitdir\" >expect &&\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorbing fails for a s\n +\ttest_cmp expect actual\n +\t)\n +'\n ++\n ++test_expect_success 'absorbgitdirs works with a submodule with worktree config' '\n ++\t# reuse the worktree of the superproject\n ++\t(\n ++\tcd wt &&\n ++\n ++\t# create a new unembedded git dir\n ++\tgit init sub5 &&\n ++\ttest_commit -C sub5 first &&\n ++\tgit submodule add ./sub5 &&\n ++\ttest_tick &&\n ++\n ++\t# turn on worktree configs for submodule\n ++\tgit -C sub5 config extensions.worktreeConfig true &&\n ++\n ++\t# absorb the git dir\n ++\tgit submodule absorbgitdirs sub5 &&\n ++\n ++\t# make sure the submodule noted the superproject gitdir correctly\n ++\tsubmodule_gitdir=\"$(git -C sub5 rev-parse --absolute-git-dir)\" &&\n ++\tsuperproject_gitdir=\"$(git rev-parse --absolute-git-dir)\" &&\n ++\n ++\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n ++\t\t\"$submodule_gitdir\" >expect &&\n ++\tgit -C sub5 config submodule.superprojectGitDir >actual &&\n ++\n ++\ttest_cmp expect actual &&\n ++\n ++\t# make sure the config went into the submodule config.worktree\n ++\ttest_file_not_empty \"$submodule_gitdir/config.worktree\"\n ++\t)\n ++'\n +\n test_done\n4: f0412d6d34 ! 4: 33a582ef13 submodule: record superproject gitdir during 'update'\n @@ Metadata\n ## Commit message ##\n submodule: record superproject gitdir during 'update'\n \n - A recorded hint path to the superproject's gitdir might be added during\n + A recorded path to the superproject's gitdir might be added during\n 'git submodule add', but in some cases - like submodules which were\n created before 'git submodule add' learned to record that info - it might\n be useful to update the hint. Let's do it during 'git submodule\n @@ git-submodule.sh: cmd_update()\n \t\t\t;;\n \t\tesac\n \n -+\t\t# Cache a pointer to the superproject's common dir. This may have\n -+\t\t# changed, unless it's a fresh clone. Writes it to worktree\n -+\t\t# if applicable, otherwise to local.\n ++\t\t# Store a poitner to the superproject's gitdir. This may have\n ++\t\t# changed, unless it's a fresh clone. Write to worktree if\n ++\t\t# applicable, and point to superproject's worktree gitdir if\n ++\t\t# applicable.\n +\t\tif test -z \"$just_cloned\"\n +\t\tthen\n +\t\t\tsm_gitdir=\"$(git -C \"$sm_path\" rev-parse --absolute-git-dir)\"\n +\t\t\trelative_gitdir=\"$(git rev-parse --path-format=relative \\\n +\t\t\t\t\t\t\t --prefix \"${sm_gitdir}\" \\\n -+\t\t\t\t\t\t\t --git-common-dir)\"\n ++\t\t\t\t\t\t\t --git-dir)\"\n +\n +\t\t\tgit -C \"$sm_path\" config --worktree \\\n +\t\t\t\tsubmodule.superprojectgitdir \"$relative_gitdir\"\n @@ t/t7406-submodule-update.sh: test_expect_success 'submodule update --quiet passe\n +\t git -C submodule config --unset submodule.superprojectGitdir &&\n +\t git submodule update &&\n +\t test-tool path-utils relative_path \\\n -+\t\t\"$(git rev-parse --path-format=absolute --git-common-dir)\" \\\n -+\t\t\"$(git -C submodule rev-parse --path-format=absolute --git-common-dir)\" >expect &&\n ++\t\t\"$(git rev-parse --absolute-git-dir)\" \\\n ++\t\t\"$(git -C submodule rev-parse --absolute-git-dir)\" >expect &&\n +\t git -C submodule config submodule.superprojectGitdir >actual &&\n +\t test_cmp expect actual\n +\t)\n +'\n ++\n ++test_expect_success 'submodule update uses config.worktree if applicable' '\n ++\t(cd super &&\n ++\t git -C submodule config --unset submodule.superprojectGitDir &&\n ++\t git -C submodule config extensions.worktreeConfig true &&\n ++\t git submodule update &&\n ++\t test-tool path-utils relative_path \\\n ++\t\t\"$(git rev-parse --absolute-git-dir)\" \\\n ++\t\t\"$(git -C submodule rev-parse --absolute-git-dir)\" >expect &&\n ++\t git -C submodule config submodule.superprojectGitdir >actual &&\n ++\t test_cmp expect actual &&\n ++\n ++\t test_file_not_empty \"$(git -C submodule rev-parse --absolute-git-dir)/config.worktree\"\n ++\t)\n ++'\n +\n test_done\n-: ---------- > 5: a8b5d40a77 submodule: use config to find superproject worktree\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"441378","messageId":"20211117005701.371808-2-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20211117005701.371808-1-emilyshaffer@google.com","subject":"[PATCH v6 1/5] t7400-submodule-basic: modernize inspect() helper","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-17T00:56:57Z","receivedAt":"2021-11-17T00:57:12Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Since the inspect() helper in the submodule-basic test suite was\nwritten, 'git -C ' was added. By using -C, we no longer need a\nreference to the base directory for the test. This simplifies callsites,\nand will make the addition of other arguments in later patches more\nreadable.\n\nSigned-off-by: Emily Shaffer \n---\n t/t7400-submodule-basic.sh | 40 +++++++++++++++-----------------------\n 1 file changed, 16 insertions(+), 24 deletions(-)\n\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex cb1b8e35db..d69a5c0032 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -107,23 +107,15 @@ test_expect_success 'setup - repository to add submodules to' '\n # generates, which will expand symbolic links.\n submodurl=$(pwd -P)\n \n-listbranches() {\n-\tgit for-each-ref --format='%(refname)' 'refs/heads/*'\n-}\n-\n inspect() {\n-\tdir=$1 &&\n-\tdotdot=\"${2:-..}\" &&\n-\n-\t(\n-\t\tcd \"$dir\" &&\n-\t\tlistbranches >\"$dotdot/heads\" &&\n-\t\t{ git symbolic-ref HEAD || :; } >\"$dotdot/head\" &&\n-\t\tgit rev-parse HEAD >\"$dotdot/head-sha1\" &&\n-\t\tgit update-index --refresh &&\n-\t\tgit diff-files --exit-code &&\n-\t\tgit clean -n -d -x >\"$dotdot/untracked\"\n-\t)\n+\tsub_dir=$1 &&\n+\n+\tgit -C \"$sub_dir\" for-each-ref --format='%(refname)' 'refs/heads/*' >heads &&\n+\t{ git -C \"$sub_dir\" symbolic-ref HEAD || :; } >head &&\n+\tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n+\tgit -C \"$sub_dir\" update-index --refresh &&\n+\tgit -C \"$sub_dir\" diff-files --exit-code &&\n+\tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n \n test_expect_success 'submodule add' '\n@@ -146,7 +138,7 @@ test_expect_success 'submodule add' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod ../.. &&\n+\tinspect addtest/submod &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -248,7 +240,7 @@ test_expect_success 'submodule add --branch' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod-branch ../.. &&\n+\tinspect addtest/submod-branch &&\n \ttest_cmp expect-heads heads &&\n \ttest_cmp expect-head head &&\n \ttest_must_be_empty untracked\n@@ -264,7 +256,7 @@ test_expect_success 'submodule add with ./ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotsubmod/frotz ../../.. &&\n+\tinspect addtest/dotsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -280,7 +272,7 @@ test_expect_success 'submodule add with /././ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotslashdotsubmod/frotz ../../.. &&\n+\tinspect addtest/dotslashdotsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -296,7 +288,7 @@ test_expect_success 'submodule add with // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/slashslashsubmod/frotz ../../.. &&\n+\tinspect addtest/slashslashsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -312,7 +304,7 @@ test_expect_success 'submodule add with /.. in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod ../.. &&\n+\tinspect addtest/realsubmod &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -328,7 +320,7 @@ test_expect_success 'submodule add with ./, /.. and // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod2 ../.. &&\n+\tinspect addtest/realsubmod2 &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -359,7 +351,7 @@ test_expect_success 'submodule add in subdirectory' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod3 ../.. &&\n+\tinspect addtest/realsubmod3 &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"441379","messageId":"20211117005701.371808-3-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20211117005701.371808-1-emilyshaffer@google.com","subject":"[PATCH v6 2/5] introduce submodule.superprojectGitDir record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-17T00:56:58Z","receivedAt":"2021-11-17T00:57:13Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Teach submodules a reference to their superproject's gitdir. This allows\nus to A) know that we're running from a submodule, and B) have a\nshortcut to the superproject's vitals, for example, configs.\n\nBy using a relative path instead of an absolute path, we can move the\nsuperproject directory around on the filesystem without breaking the\nsubmodule's pointer. And by using the path from gitdir to gitdir, we can\nmove the submodule within the superproject's tree structure without\nbreaking the submodule's pointer, too. Finally, by pointing at the\nsuperproject's worktree gitdir (if it exists), we ensure that we can\ntell which worktree contains our submodule.\n\nSince this hint value is only introduced during new submodule creation\nvia `git submodule add`, though, there is more work to do to allow the\nrecord to be created at other times.\n\nOnce this new config is reliably in place, we can use it to know\ndefinitively that we are working in a submodule, and to know which\nsuperproject we are a submodule of. This allows us to do some\nvalue-added behavior, like letting \"git status\" print additional info\nabout the submodule's status in relation to its superproject, or like\nletting the superproject and submodule share an additional config file\nseparate from either one's local config.\n\nSigned-off-by: Emily Shaffer \nHelped-by: Junio C Hamano \n---\n Documentation/config/submodule.txt | 12 +++++++++++\n builtin/submodule--helper.c | 11 ++++++++++\n t/t7400-submodule-basic.sh | 32 ++++++++++++++++++++----------\n 3 files changed, 45 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt\nindex ee454f8126..61d975745e 100644\n--- a/Documentation/config/submodule.txt\n+++ b/Documentation/config/submodule.txt\n@@ -91,3 +91,15 @@ submodule.alternateErrorStrategy::\n \t`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`\n \tor `info`, and if there is an error with the computed alternate, the\n \tclone proceeds as if no alternate was specified.\n+\n+submodule.superprojectGitDir::\n+\tThe relative path from the submodule's gitdir to its superproject's\n+\tgitdir. When Git is run in a repository, it usually makes no\n+\tdifference whether this repository is standalone or a submodule, but if\n+\tthis configuration variable is present, additional behavior may be\n+\tpossible, such as \"git status\" printing additional information about\n+\tthis submodule's status with respect to its superproject. This config\n+\tshould only be present in projects which are submodules, but is not\n+\tguaranteed to be present in every submodule, so only optional\n+\tvalue-added behavior should be linked to it. It is set automatically\n+\tduring submodule creation.\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex e630f0c730..24f0ef2a78 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -1838,6 +1838,17 @@ static int clone_submodule(struct module_clone_data *clone_data)\n \t\tgit_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n \t\t\t\t error_strategy);\n \n+\t/*\n+\t * Set the path from submodule's new gitdir to superproject's gitdir.\n+\t * The latter may be a worktree gitdir. However, it is not possible for\n+\t * the submodule to have a worktree-specific gitdir or config at clone\n+\t * time, because \"extensions.worktreeConfig\" is only valid when set in\n+\t * the local gitconfig, which the brand new submodule does not have yet.\n+\t */\n+\tgit_config_set_in_file(p, \"submodule.superprojectGitDir\",\n+\t\t\t relative_path(absolute_path(get_git_dir()),\n+\t\t\t\t\t sm_gitdir, &sb));\n+\n \tfree(sm_alternate);\n \tfree(error_strategy);\n \ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex d69a5c0032..3d146491df 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -109,12 +109,24 @@ submodurl=$(pwd -P)\n \n inspect() {\n \tsub_dir=$1 &&\n+\tsuper_dir=$2 &&\n \n \tgit -C \"$sub_dir\" for-each-ref --format='%(refname)' 'refs/heads/*' >heads &&\n \t{ git -C \"$sub_dir\" symbolic-ref HEAD || :; } >head &&\n \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n \tgit -C \"$sub_dir\" update-index --refresh &&\n \tgit -C \"$sub_dir\" diff-files --exit-code &&\n+\n+\t# Ensure that submodule.superprojectGitDir contains the path from the\n+\t# submodule's gitdir to the superproject's gitdir.\n+\n+\tsuper_abs_gitdir=$(git -C \"$super_dir\" rev-parse --absolute-git-dir) &&\n+\tsub_abs_gitdir=$(git -C \"$sub_dir\" rev-parse --absolute-git-dir) &&\n+\n+\t[ \"$(git -C \"$sub_dir\" config --get submodule.superprojectGitDir)\" = \\\n+\t \"$(test-tool path-utils relative_path \"$super_abs_gitdir\" \\\n+\t\t\t\t\t\t\"$sub_abs_gitdir\")\" ] &&\n+\n \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n \n@@ -138,7 +150,7 @@ test_expect_success 'submodule add' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod &&\n+\tinspect addtest/submod addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -240,7 +252,7 @@ test_expect_success 'submodule add --branch' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod-branch &&\n+\tinspect addtest/submod-branch addtest &&\n \ttest_cmp expect-heads heads &&\n \ttest_cmp expect-head head &&\n \ttest_must_be_empty untracked\n@@ -256,7 +268,7 @@ test_expect_success 'submodule add with ./ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotsubmod/frotz &&\n+\tinspect addtest/dotsubmod/frotz addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -272,7 +284,7 @@ test_expect_success 'submodule add with /././ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotslashdotsubmod/frotz &&\n+\tinspect addtest/dotslashdotsubmod/frotz addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -288,7 +300,7 @@ test_expect_success 'submodule add with // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/slashslashsubmod/frotz &&\n+\tinspect addtest/slashslashsubmod/frotz addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -304,7 +316,7 @@ test_expect_success 'submodule add with /.. in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod &&\n+\tinspect addtest/realsubmod addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -320,7 +332,7 @@ test_expect_success 'submodule add with ./, /.. and // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod2 &&\n+\tinspect addtest/realsubmod2 addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -351,7 +363,7 @@ test_expect_success 'submodule add in subdirectory' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod3 &&\n+\tinspect addtest/realsubmod3 addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -492,7 +504,7 @@ test_expect_success 'update should work when path is an empty dir' '\n \tgit submodule update -q >update.out &&\n \ttest_must_be_empty update.out &&\n \n-\tinspect init &&\n+\tinspect init . &&\n \ttest_cmp expect head-sha1\n '\n \n@@ -551,7 +563,7 @@ test_expect_success 'update should checkout rev1' '\n \techo \"$rev1\" >expect &&\n \n \tgit submodule update init &&\n-\tinspect init &&\n+\tinspect init . &&\n \n \ttest_cmp expect head-sha1\n '\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"441380","messageId":"20211117005701.371808-4-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20211117005701.371808-1-emilyshaffer@google.com","subject":"[PATCH v6 3/5] submodule: record superproject gitdir during absorbgitdirs","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-17T00:56:59Z","receivedAt":"2021-11-17T00:57:15Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Already during 'git submodule add' we record a pointer to the\nsuperproject's gitdir. However, this doesn't help brand-new\nsubmodules created with 'git init' and later absorbed with 'git\nsubmodule absorbgitdirs'. Let's start adding that pointer during 'git\nsubmodule absorbgitdirs' too.\n\nSigned-off-by: Emily Shaffer \n---\n submodule.c | 23 +++++++++\n t/t7412-submodule-absorbgitdirs.sh | 82 +++++++++++++++++++++++++++++-\n 2 files changed, 103 insertions(+), 2 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex c689070524..d7395c7551 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2097,6 +2097,9 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \tchar *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL;\n \tstruct strbuf new_gitdir = STRBUF_INIT;\n \tconst struct submodule *sub;\n+\tstruct config_set sub_cs;\n+\tstruct strbuf config_path = STRBUF_INIT, sb = STRBUF_INIT;\n+\tint tmp;\n \n \tif (submodule_uses_worktrees(path))\n \t\tdie(_(\"relocate_gitdir for submodule '%s' with \"\n@@ -2127,6 +2130,26 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \n \trelocate_gitdir(path, real_old_git_dir, real_new_git_dir);\n \n+\t/*\n+\t * Note location of superproject's gitdir. Because the submodule already\n+\t * has a gitdir and local config, we can store this pointer from\n+\t * worktree config to worktree config, if the submodule has\n+\t * extensions.worktreeConfig set.\n+\t */\n+\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n+\tgit_configset_init(&sub_cs);\n+\tgit_configset_add_file(&sub_cs, config_path.buf);\n+\t/* return 0 indicates config was found - we have a worktree config */\n+\tif (!git_configset_get_bool(&sub_cs, \"extensions.worktreeConfig\", &tmp))\n+\t\tstrbuf_addstr(&config_path, \".worktree\");\n+\n+\tgit_config_set_in_file(config_path.buf, \"submodule.superprojectGitdir\",\n+\t\t\t relative_path(absolute_path(get_git_dir()),\n+\t\t\t\t\t real_new_git_dir, &sb));\n+\n+\tgit_configset_clear(&sub_cs);\n+\tstrbuf_release(&config_path);\n+\tstrbuf_release(&sb);\n \tfree(old_git_dir);\n \tfree(real_old_git_dir);\n \tfree(real_new_git_dir);\ndiff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh\nindex 1cfa150768..5753f90268 100755\n--- a/t/t7412-submodule-absorbgitdirs.sh\n+++ b/t/t7412-submodule-absorbgitdirs.sh\n@@ -30,7 +30,17 @@ test_expect_success 'absorb the git dir' '\n \tgit status >actual.1 &&\n \tgit -C sub1 rev-parse HEAD >actual.2 &&\n \ttest_cmp expect.1 actual.1 &&\n-\ttest_cmp expect.2 actual.2\n+\ttest_cmp expect.2 actual.2 &&\n+\n+\t# make sure the submodule cached the superproject gitdir correctly\n+\tsubmodule_gitdir=\"$(git -C sub1 rev-parse --path-format=absolute --git-common-dir)\" &&\n+\tsuperproject_gitdir=\"$(git rev-parse --path-format=absolute --git-common-dir)\" &&\n+\n+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n+\t\t\"$submodule_gitdir\" >expect &&\n+\tgit -C sub1 config submodule.superprojectGitDir >actual &&\n+\n+\ttest_cmp expect actual\n '\n \n test_expect_success 'absorbing does not fail for deinitialized submodules' '\n@@ -61,7 +71,16 @@ test_expect_success 'absorb the git dir in a nested submodule' '\n \tgit status >actual.1 &&\n \tgit -C sub1/nested rev-parse HEAD >actual.2 &&\n \ttest_cmp expect.1 actual.1 &&\n-\ttest_cmp expect.2 actual.2\n+\ttest_cmp expect.2 actual.2 &&\n+\n+\tsub1_gitdir=\"$(git -C sub1 rev-parse --path-format=absolute --git-common-dir)\" &&\n+\tsub1_nested_gitdir=\"$(git -C sub1/nested rev-parse --path-format=absolute --git-common-dir)\" &&\n+\n+\ttest-tool path-utils relative_path \"$sub1_gitdir\" \"$sub1_nested_gitdir\" \\\n+\t\t>expect &&\n+\tgit -C sub1/nested config submodule.superprojectGitDir >actual &&\n+\n+\ttest_cmp expect actual\n '\n \n test_expect_success 're-setup nested submodule' '\n@@ -130,4 +149,63 @@ test_expect_success 'absorbing fails for a submodule with multiple worktrees' '\n \ttest_i18ngrep \"not supported\" error\n '\n \n+test_expect_success 'absorbgitdirs works when called from a superproject worktree' '\n+\t# set up a worktree of the superproject\n+\tgit worktree add wt &&\n+\t(\n+\tcd wt &&\n+\n+\t# create a new unembedded git dir\n+\tgit init sub4 &&\n+\ttest_commit -C sub4 first &&\n+\tgit submodule add ./sub4 &&\n+\ttest_tick &&\n+\n+\t# absorb the git dir\n+\tgit submodule absorbgitdirs sub4 &&\n+\n+\t# make sure the submodule noted the superproject gitdir correctly\n+\tsubmodule_gitdir=\"$(git -C sub4 rev-parse --absolute-git-dir)\" &&\n+\tsuperproject_gitdir=\"$(git rev-parse --absolute-git-dir)\" &&\n+\n+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n+\t\t\"$submodule_gitdir\" >expect &&\n+\tgit -C sub4 config submodule.superprojectGitDir >actual &&\n+\n+\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success 'absorbgitdirs works with a submodule with worktree config' '\n+\t# reuse the worktree of the superproject\n+\t(\n+\tcd wt &&\n+\n+\t# create a new unembedded git dir\n+\tgit init sub5 &&\n+\ttest_commit -C sub5 first &&\n+\tgit submodule add ./sub5 &&\n+\ttest_tick &&\n+\n+\t# turn on worktree configs for submodule\n+\tgit -C sub5 config extensions.worktreeConfig true &&\n+\n+\t# absorb the git dir\n+\tgit submodule absorbgitdirs sub5 &&\n+\n+\t# make sure the submodule noted the superproject gitdir correctly\n+\tsubmodule_gitdir=\"$(git -C sub5 rev-parse --absolute-git-dir)\" &&\n+\tsuperproject_gitdir=\"$(git rev-parse --absolute-git-dir)\" &&\n+\n+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n+\t\t\"$submodule_gitdir\" >expect &&\n+\tgit -C sub5 config submodule.superprojectGitDir >actual &&\n+\n+\ttest_cmp expect actual &&\n+\n+\t# make sure the config went into the submodule config.worktree\n+\ttest_file_not_empty \"$submodule_gitdir/config.worktree\"\n+\t)\n+'\n+\n test_done\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"441381","messageId":"20211117005701.371808-5-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20211117005701.371808-1-emilyshaffer@google.com","subject":"[PATCH v6 4/5] submodule: record superproject gitdir during 'update'","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-17T00:57:00Z","receivedAt":"2021-11-17T00:57:18Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"A recorded path to the superproject's gitdir might be added during\n'git submodule add', but in some cases - like submodules which were\ncreated before 'git submodule add' learned to record that info - it might\nbe useful to update the hint. Let's do it during 'git submodule\nupdate', when we already have a handle to the superproject while calling\noperations on the submodules.\n\nSigned-off-by: Emily Shaffer \n---\n git-submodule.sh | 15 +++++++++++++++\n t/t7406-submodule-update.sh | 27 +++++++++++++++++++++++++++\n 2 files changed, 42 insertions(+)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 652861aa66..7c247bee7f 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -449,6 +449,21 @@ cmd_update()\n \t\t\t;;\n \t\tesac\n \n+\t\t# Store a poitner to the superproject's gitdir. This may have\n+\t\t# changed, unless it's a fresh clone. Write to worktree if\n+\t\t# applicable, and point to superproject's worktree gitdir if\n+\t\t# applicable.\n+\t\tif test -z \"$just_cloned\"\n+\t\tthen\n+\t\t\tsm_gitdir=\"$(git -C \"$sm_path\" rev-parse --absolute-git-dir)\"\n+\t\t\trelative_gitdir=\"$(git rev-parse --path-format=relative \\\n+\t\t\t\t\t\t\t --prefix \"${sm_gitdir}\" \\\n+\t\t\t\t\t\t\t --git-dir)\"\n+\n+\t\t\tgit -C \"$sm_path\" config --worktree \\\n+\t\t\t\tsubmodule.superprojectgitdir \"$relative_gitdir\"\n+\t\tfi\n+\n \t\tif test -n \"$recursive\"\n \t\tthen\n \t\t\t(\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex 11cccbb333..b42a339982 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -1061,4 +1061,31 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n \t)\n '\n \n+test_expect_success 'submodule update adds superproject gitdir to older repos' '\n+\t(cd super &&\n+\t git -C submodule config --unset submodule.superprojectGitdir &&\n+\t git submodule update &&\n+\t test-tool path-utils relative_path \\\n+\t\t\"$(git rev-parse --absolute-git-dir)\" \\\n+\t\t\"$(git -C submodule rev-parse --absolute-git-dir)\" >expect &&\n+\t git -C submodule config submodule.superprojectGitdir >actual &&\n+\t test_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success 'submodule update uses config.worktree if applicable' '\n+\t(cd super &&\n+\t git -C submodule config --unset submodule.superprojectGitDir &&\n+\t git -C submodule config extensions.worktreeConfig true &&\n+\t git submodule update &&\n+\t test-tool path-utils relative_path \\\n+\t\t\"$(git rev-parse --absolute-git-dir)\" \\\n+\t\t\"$(git -C submodule rev-parse --absolute-git-dir)\" >expect &&\n+\t git -C submodule config submodule.superprojectGitdir >actual &&\n+\t test_cmp expect actual &&\n+\n+\t test_file_not_empty \"$(git -C submodule rev-parse --absolute-git-dir)/config.worktree\"\n+\t)\n+'\n+\n test_done\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"441382","messageId":"20211117005701.371808-6-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20211117005701.371808-1-emilyshaffer@google.com","subject":"[PATCH v6 5/5] submodule: use config to find superproject worktree","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-17T00:57:01Z","receivedAt":"2021-11-17T00:57:20Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Now that submodule.superprojectGitDir is being treated as the point of\ntruth for whether a repo is a submodule or not, let's use it in `git\nrev-parse --show-superproject-working-tree`.\n\nSigned-off-by: Emily Shaffer \n\n---\n\nThis commit may be more of an RFC - to demonstrate what life looks like\nif we use submodule.superprojectGitDir as the source of truth. But since\n'git rev-parse --show-superproject-working-tree' is used in a lot of\nscripts in the wild[1], I'm not so sure it's a great example.\n\nTo be honest, I'd prefer to die(\"Try running 'git submodule update'\")\nhere, but I don't think that's very script-friendly. However, falling\nback on the old implementation kind of undermines the idea of treating\nsubmodule.superprojectGitDir as the point of truth. One thought I did\nhave was to put that error message in builtin/rev-parse.c instead, and\nprint it to stderr (per usual with user-visible messages) so it doesn't\ninterfere with scripts, but gives a hint for debugging.\n\nAnother thought - captured by the NEEDSWORK in the diff - was that we\ncould \"heal\" by adding that config after we know the worktree of the\nsuperproject.\n\nOr, it could be that it won't be a problem for a long time, as anybody\nrunning 'git submodule update' will eventually have that config\nspecified - that's why I included the traces, so we could try and get an\nunderstanding of how long repos remain in this state where they have\nsubmodules but nobody ran 'git submodule update'. But that will only\ngive me visibility into submodule users at Google, who we expect to be\nmaking a lot of workflow changes soon, anyway.\n\n1: https://github.com/search?q=%22--show-superproject-working-tree%22&type=code\n---\n submodule.c | 85 +++++++++++++++++++++++++++++++++++++++++++-\n t/t1500-rev-parse.sh | 9 +++++\n 2 files changed, 93 insertions(+), 1 deletion(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex d7395c7551..ad95cdda07 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2236,7 +2236,7 @@ void absorb_git_dir_into_superproject(const char *path,\n \t}\n }\n \n-int get_superproject_working_tree(struct strbuf *buf)\n+static int get_superproject_working_tree_from_fs(struct strbuf *buf)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -2320,6 +2320,89 @@ int get_superproject_working_tree(struct strbuf *buf)\n \treturn ret;\n }\n \n+int get_superproject_working_tree(struct strbuf *buf)\n+{\n+\tchar *super_gitdir = NULL;\n+\tconst char *cwd = xgetcwd();\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf absolute_super_gitdir = STRBUF_INIT;\n+\tstruct strbuf out = STRBUF_INIT;\n+\tstruct string_list lines = STRING_LIST_INIT_NODUP;\n+\tstruct string_list_item *it = NULL;\n+\tconst char *wt_prefix = \"worktree \";\n+\tint rc = 0;\n+\n+\n+\t/* Do we know we have a superproject? */\n+\tif (git_config_get_string(\"submodule.superprojectgitdir\", &super_gitdir))\n+\t\tgoto fallback;\n+\n+\tstrbuf_addf(&absolute_super_gitdir, \"%s/%s\", get_git_dir(), super_gitdir);\n+\n+\t/*\n+\t * NEEDSWORK: This is a child process call because worktree.c still\n+\t * relies heavily on the_repository. If we can make worktree.c work with\n+\t * a repository object - or, better yet, a gitdir path alone - then we\n+\t * can drop the child process and ask worktree.c directly.\n+\t *\n+\t * Alternatively, if 'git worktree' learns a way to say 'the worktree\n+\t * associated with this gitdir' instead of 'all worktrees', that would\n+\t * be clearer because we could skip the foreach below.\n+\t */\n+\n+\t/* Get the output of `git worktree list` from that superproject */\n+\tprepare_other_repo_env(&cp.env_array, absolute_super_gitdir.buf);\n+\tstrvec_pushl(&cp.args, \"-C\", absolute_super_gitdir.buf, \"worktree\", \"list\",\n+\t\t \"--porcelain\", NULL);\n+\n+\tcp.git_cmd = 1;\n+\tif (capture_command(&cp, &out, 0) ||\n+\t !string_list_split_in_place(&lines, out.buf, '\\n', -1))\n+\t\tdie(\"submodule.superprojectGitDir is stale; run 'git submodule \"\n+\t\t \"update' from the superproject.\");\n+\n+\tfor_each_string_list_item(it, &lines) {\n+\t\tchar *trimmed;\n+\t\t/*\n+\t\t * Lines containing worktree dirs look like\n+\t\t * 'worktree /some/path'\n+\t\t */\n+\t\tif (strncmp(it->string, wt_prefix, strlen(wt_prefix)))\n+\t\t\tcontinue;\n+\t\ttrimmed = it->string + strlen(wt_prefix);\n+\n+\t\t/*\n+\t\t * '/some/path/to/sub' is a prefix of '/some/path' - that's our\n+\t\t * worktree\n+\t\t */\n+\t\tif (!strncmp(cwd, trimmed, strlen(trimmed))) {\n+\t\t\tstrbuf_addstr(buf, trimmed);\n+\t\t\trc = 1;\n+\t\t\tgoto cleanup;\n+\t\t}\n+\t}\n+\n+fallback:\n+\t/*\n+\t * Because a submodule may have been created before\n+\t * submodule.superprojectGitDir was introduced, fall back on checking\n+\t * whether ../ is the superproject.\n+\t */\n+\ttrace2_data_intmax(\"submodule\", the_repository,\n+\t\t\t \"get_superproject_wt/config_missing\", rc);\n+\trc = get_superproject_working_tree_from_fs(buf);\n+\n+\t/*\n+\t * NEEDSWORK: Is it possible to teach a submodule the path to its\n+\t * superproject when this happens?\n+\t */\n+\n+cleanup:\n+\tstring_list_clear(&lines, 0);\n+\tstrbuf_release(&out);\n+\treturn rc;\n+}\n+\n /*\n * Put the gitdir for a submodule (given relative to the main\n * repository worktree) into `buf`, or return -1 on error.\ndiff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\nindex 1c2df08333..e569910289 100755\n--- a/t/t1500-rev-parse.sh\n+++ b/t/t1500-rev-parse.sh\n@@ -247,6 +247,15 @@ test_expect_success 'showing the superproject correctly' '\n \ttest_cmp expect out\n '\n \n+test_expect_success 'show the superproject correctly without superprojectGitDir' '\n+\t# repos created before submodule.superprojectGitDir was introduced which\n+\t# have not been `git submodule update`-ed lately must not break\n+\tgit -C super/dir/sub config --unset submodule.superprojectGitDir &&\n+\techo $(pwd)/super >expect &&\n+\tgit -C super/dir/sub rev-parse --show-superproject-working-tree >out &&\n+\ttest_cmp expect out\n+'\n+\n # at least one external project depends on this behavior:\n test_expect_success 'rev-parse --since= unsqueezed ordering' '\n \tx1=--since=1970-01-01T00:00:01Z &&\n-- \n2.34.0.rc1.387.gb447b232ab-goog\n\n"},{"id":"441474","messageId":"RFC-cover-0.2-00000000000-20211117T113134Z-avarab@gmail.com","threadId":"56920","inReplyTo":"20211117005701.371808-1-emilyshaffer@google.com","subject":"[RFC PATCH 0/2] submodule: test what happens if submodule.superprojectGitDir isn't around","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-17T11:43:38Z","receivedAt":"2021-11-17T11:43:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, Nov 16 2021, Emily Shaffer wrote:\n\n> [...]\n> A couple things. Firstly, a semantics change *back* to the semantics of\n> v3 - we map from gitdir to gitdir, *not* from common dir to common dir,\n> so that theoretically a submodule with multiple worktrees in multiple\n> superproject worktrees will be able to figure out which worktree of the\n> superproject it's in. (Realistically, that's not really possible right\n> now, but I'd like to change that soon.)\n>\n> Secondly, a rewording of comments and commit messages to indicate that\n> this isn't a cache of some expensive operation, but rather intended to\n> be the source of truth for all submodules. I also added a fifth commit\n> rewriting `git rev-parse --show-superproject-working-tree` to\n> demonstrate what that means in practice - but from a practical\n> standpoint, I'm a little worried about that fifth patch. More details in\n> the patch 5 description.\n>\n> I did discuss Ævar's idea of relying on in-process filesystem digging to\n> find the superproject's gitdir with the rest of the Google team, but in\n> the end decided that there are some worries about filesystem digging in\n> this way (namely, some ugly interactions with network drives that are\n> actually already an issue for Googler Linux machines). Plus, the allure\n> of being able to definitively know that we're a submodule is pretty\n> strong. ;) But overall, this is the direction I'd prefer to keep going \n> in, rather than trying to guess from the filesystem going forward.\n\nDid you try running the ad-hoc benchmark I included in [1] on that\nGoogle NFS? I've dealt with some slow-ish network filesystems, but if\nit's slower than AIX's local FS (where I couldn't see a difference) I'd\nput money on it being a cross-Atlantic mount or something :)\n\nRe your:\n\n \"this isn't a cache of some expensive operation, but rather intended to be the source of truth for all submodules.\"\n\nIn your 5/5 it says, in seeming contradiction to this:\n\n This commit may be more of an RFC - to demonstrate what life looks like\n if we use submodule.superprojectGitDir as the source of truth. But since\n 'git rev-parse --show-superproject-working-tree' is used in a lot of\n scripts in the wild[1], I'm not so sure it's a great example.\n\n To be honest, I'd prefer to die(\"Try running 'git submodule update'\")\n here, but I don't think that's very script-friendly. However, falling\n back on the old implementation kind of undermines the idea of treating\n submodule.superprojectGitDir as the point of truth.\n\nMost of what I've been suggesting in my [1] and related is that I'm\nconfused about if & how this is a pure caching mechanism.\n\nRemoving mentions of it being a cache but it seemingly still being a\ncache at the tip of this series has just added to that confusion for\nme :)\n\nAnyway. While I do think this caching mechanism is probably\nunnecessary in the short to medium term, i.e. it seems to the extent\nthat it was ever needed was due to some bridging of *.sh<->*.c that\nwe're *this* close to eliminating anyway.\n\nBut maybe I'm wrong. The benchmark I suggested above on that Google\nNFS might be indicative. I don't really see how something that'll be\ndoing a bunch of FS ops anyway is going to be noticeably slower with\nthat approach, but maybe opening the index/tree of the superproject is\nmore expensive than I'm expecting.\n\nIn any case, all of that's not the hill I'm picking to die on. If\nyou'd like to go ahead with this cache-or-not-a-cache then sure, I\nwon't belabor that point.\n\nI *do* strongly think if we're doing so though that we should have\nsomething like this on top. I.e. let's test wha happens if we do and\ndon't have this \"caching\" variable, which is demonstrably easy to do.\n\nBenchmarking the two gives me:\n\n $ git hyperfine -L rev HEAD~0 -L s true,false -s 'make -j8 all' '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR={s} ./t7412-submodule-absorbgitdirs.sh)'\n Benchmark 1: (cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=true ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0\n Time (mean ± σ): 545.9 ms ± 1.6 ms [User: 490.3 ms, System: 114.0 ms]\n Range (min … max): 543.5 ms … 548.1 ms 10 runs\n \n Benchmark 2: (cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=false ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0\n Time (mean ± σ): 537.9 ms ± 11.4 ms [User: 476.8 ms, System: 117.6 ms]\n Range (min … max): 532.7 ms … 570.1 ms 10 runs\n \n Summary\n '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=false ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0' ran\n 1.01 ± 0.02 times faster than '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=true ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0'\n\nI.e. not using the cache is either indistinguishable or a bit faster\n(the \"a bit faster\" is definitely due to just running less test code\nthough).\n\nI'm sending this before the CI run[2] finishes (which now tests both\nmodes), but both of these work for me locally on a full test suite\nrun.\n\n1. https://lore.kernel.org/git/211109.86v912dtfw.gmgdl@evledraar.gmail.com/\n2. https://github.com/avar/git/runs/4237446991?check_suite_focus=true\n\nÆvar Arnfjörð Bjarmason (2):\n submodule tests: fix potentially broken \"config .. --unset\"\n submodule: add test mode for checking absence of \"superProjectGitDir\"\n\n ci/run-build-and-tests.sh | 1 +\n git-submodule.sh | 2 +-\n submodule.c | 7 +++++++\n t/lib-submodule-superproject.sh | 24 ++++++++++++++++++++++++\n t/t7406-submodule-update.sh | 13 ++++++-------\n t/t7412-submodule-absorbgitdirs.sh | 19 ++++++-------------\n 6 files changed, 45 insertions(+), 21 deletions(-)\n create mode 100644 t/lib-submodule-superproject.sh\n\n-- \n2.34.0.796.g2c87ed6146a\n\n"},{"id":"441475","messageId":"RFC-patch-1.2-a8e31e35392-20211117T113134Z-avarab@gmail.com","threadId":"56920","inReplyTo":"RFC-cover-0.2-00000000000-20211117T113134Z-avarab@gmail.com","subject":"[RFC PATCH 1/2] submodule tests: fix potentially broken \"config .. --unset\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-17T11:43:39Z","receivedAt":"2021-11-17T11:43:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"These \"config ... --unset\" at the start must be guarded by something\nlike a test_might_fail, or we'll fail if a previous test didn't run,\ne.g. due to the --run option.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/t7406-submodule-update.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex b42a339982b..01e1acaf300 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -1063,7 +1063,7 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n \n test_expect_success 'submodule update adds superproject gitdir to older repos' '\n \t(cd super &&\n-\t git -C submodule config --unset submodule.superprojectGitdir &&\n+\t test_might_fail git -C submodule config --unset submodule.superprojectGitdir &&\n \t git submodule update &&\n \t test-tool path-utils relative_path \\\n \t\t\"$(git rev-parse --absolute-git-dir)\" \\\n@@ -1075,7 +1075,7 @@ test_expect_success 'submodule update adds superproject gitdir to older repos' '\n \n test_expect_success 'submodule update uses config.worktree if applicable' '\n \t(cd super &&\n-\t git -C submodule config --unset submodule.superprojectGitDir &&\n+\t test_might_fail git -C submodule config --unset submodule.superprojectGitDir &&\n \t git -C submodule config extensions.worktreeConfig true &&\n \t git submodule update &&\n \t test-tool path-utils relative_path \\\n-- \n2.34.0.796.g2c87ed6146a\n\n"},{"id":"441476","messageId":"RFC-patch-2.2-b49d4c8db7d-20211117T113134Z-avarab@gmail.com","threadId":"56920","inReplyTo":"RFC-cover-0.2-00000000000-20211117T113134Z-avarab@gmail.com","subject":"[RFC PATCH 2/2] submodule: add test mode for checking absence of \"superProjectGitDir\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-17T11:43:40Z","receivedAt":"2021-11-17T11:43:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=false test mode to\nassert what happens if \"submodule.superProjectGitDir\" is absent or\nmissing, this checks if the \"fallback\" codepath in\nget_superproject_working_tree() is equivalent to the config lookup.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n ci/run-build-and-tests.sh | 1 +\n git-submodule.sh | 2 +-\n submodule.c | 7 +++++++\n t/lib-submodule-superproject.sh | 24 ++++++++++++++++++++++++\n t/t7406-submodule-update.sh | 9 ++++-----\n t/t7412-submodule-absorbgitdirs.sh | 19 ++++++-------------\n 6 files changed, 43 insertions(+), 19 deletions(-)\n create mode 100644 t/lib-submodule-superproject.sh\n\ndiff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh\nindex cc62616d806..5132a210057 100755\n--- a/ci/run-build-and-tests.sh\n+++ b/ci/run-build-and-tests.sh\n@@ -33,6 +33,7 @@ linux-gcc)\n \texport GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\n \texport GIT_TEST_WRITE_REV_INDEX=1\n \texport GIT_TEST_CHECKOUT_WORKERS=2\n+\texport GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=false\n \tmake test\n \t;;\n linux-clang)\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 7c247bee7f6..2b423ee05bc 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -453,7 +453,7 @@ cmd_update()\n \t\t# changed, unless it's a fresh clone. Write to worktree if\n \t\t# applicable, and point to superproject's worktree gitdir if\n \t\t# applicable.\n-\t\tif test -z \"$just_cloned\"\n+\t\tif test -z \"$just_cloned\" && test \"$GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR\" != \"false\"\n \t\tthen\n \t\t\tsm_gitdir=\"$(git -C \"$sm_path\" rev-parse --absolute-git-dir)\"\n \t\t\trelative_gitdir=\"$(git rev-parse --path-format=relative \\\ndiff --git a/submodule.c b/submodule.c\nindex ad95cdda07d..f0411a320a8 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2143,6 +2143,9 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \tif (!git_configset_get_bool(&sub_cs, \"extensions.worktreeConfig\", &tmp))\n \t\tstrbuf_addstr(&config_path, \".worktree\");\n \n+\tif (!git_env_bool(\"GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR\", 1))\n+\t\tgoto fallback;\n+\n \tgit_config_set_in_file(config_path.buf, \"submodule.superprojectGitdir\",\n \t\t\t relative_path(absolute_path(get_git_dir()),\n \t\t\t\t\t real_new_git_dir, &sb));\n@@ -2150,6 +2153,8 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \tgit_configset_clear(&sub_cs);\n \tstrbuf_release(&config_path);\n \tstrbuf_release(&sb);\n+\n+fallback:\n \tfree(old_git_dir);\n \tfree(real_old_git_dir);\n \tfree(real_new_git_dir);\n@@ -2332,6 +2337,8 @@ int get_superproject_working_tree(struct strbuf *buf)\n \tconst char *wt_prefix = \"worktree \";\n \tint rc = 0;\n \n+\tif (!git_env_bool(\"GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR\", 1))\n+\t\tgoto fallback;\n \n \t/* Do we know we have a superproject? */\n \tif (git_config_get_string(\"submodule.superprojectgitdir\", &super_gitdir))\ndiff --git a/t/lib-submodule-superproject.sh b/t/lib-submodule-superproject.sh\nnew file mode 100644\nindex 00000000000..4d49dd3782e\n--- /dev/null\n+++ b/t/lib-submodule-superproject.sh\n@@ -0,0 +1,24 @@\n+#!/bin/sh\n+\n+test_lazy_prereq SUBMODULE_CACHE_SUPERPROJECT_DIR '\n+\ttest_bool_env GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR true\n+'\n+\n+test_cmp_submodule_superprojectgitdir () {\n+\tif ! test_have_prereq SUBMODULE_CACHE_SUPERPROJECT_DIR\n+\tthen\n+\t\treturn 0\n+\tfi\n+\n+\tgit -C \"$1\" config submodule.superprojectGitDir >actual &&\n+\ttest_cmp expect actual\n+}\n+\n+test_file_not_empty_superprojectgitdir () {\n+\tif ! test_have_prereq SUBMODULE_CACHE_SUPERPROJECT_DIR\n+\tthen\n+\t\treturn 0\n+\tfi\n+\n+\ttest_file_not_empty \"$(git -C $1 rev-parse --absolute-git-dir)/$2\"\n+}\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex 01e1acaf300..f362f8d0ef0 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -13,6 +13,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n \n . ./test-lib.sh\n+. \"$TEST_DIRECTORY\"/lib-submodule-superproject.sh\n \n \n compare_head()\n@@ -1068,8 +1069,7 @@ test_expect_success 'submodule update adds superproject gitdir to older repos' '\n \t test-tool path-utils relative_path \\\n \t\t\"$(git rev-parse --absolute-git-dir)\" \\\n \t\t\"$(git -C submodule rev-parse --absolute-git-dir)\" >expect &&\n-\t git -C submodule config submodule.superprojectGitdir >actual &&\n-\t test_cmp expect actual\n+\t test_cmp_submodule_superprojectgitdir submodule\n \t)\n '\n \n@@ -1081,10 +1081,9 @@ test_expect_success 'submodule update uses config.worktree if applicable' '\n \t test-tool path-utils relative_path \\\n \t\t\"$(git rev-parse --absolute-git-dir)\" \\\n \t\t\"$(git -C submodule rev-parse --absolute-git-dir)\" >expect &&\n-\t git -C submodule config submodule.superprojectGitdir >actual &&\n-\t test_cmp expect actual &&\n+\t test_cmp_submodule_superprojectgitdir submodule &&\n \n-\t test_file_not_empty \"$(git -C submodule rev-parse --absolute-git-dir)/config.worktree\"\n+\t test_file_not_empty_superprojectgitdir submodule config.worktree\n \t)\n '\n \ndiff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh\nindex 5753f902687..6faab7e56e9 100755\n--- a/t/t7412-submodule-absorbgitdirs.sh\n+++ b/t/t7412-submodule-absorbgitdirs.sh\n@@ -7,6 +7,7 @@ directory into the superproject.\n '\n \n . ./test-lib.sh\n+. \"$TEST_DIRECTORY\"/lib-submodule-superproject.sh\n \n test_expect_success 'setup a real submodule' '\n \tgit init sub1 &&\n@@ -38,9 +39,7 @@ test_expect_success 'absorb the git dir' '\n \n \ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n \t\t\"$submodule_gitdir\" >expect &&\n-\tgit -C sub1 config submodule.superprojectGitDir >actual &&\n-\n-\ttest_cmp expect actual\n+\ttest_cmp_submodule_superprojectgitdir sub1\n '\n \n test_expect_success 'absorbing does not fail for deinitialized submodules' '\n@@ -78,9 +77,7 @@ test_expect_success 'absorb the git dir in a nested submodule' '\n \n \ttest-tool path-utils relative_path \"$sub1_gitdir\" \"$sub1_nested_gitdir\" \\\n \t\t>expect &&\n-\tgit -C sub1/nested config submodule.superprojectGitDir >actual &&\n-\n-\ttest_cmp expect actual\n+\ttest_cmp_submodule_superprojectgitdir sub1\n '\n \n test_expect_success 're-setup nested submodule' '\n@@ -170,9 +167,7 @@ test_expect_success 'absorbgitdirs works when called from a superproject worktre\n \n \ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n \t\t\"$submodule_gitdir\" >expect &&\n-\tgit -C sub4 config submodule.superprojectGitDir >actual &&\n-\n-\ttest_cmp expect actual\n+\ttest_cmp_submodule_superprojectgitdir sub4\n \t)\n '\n \n@@ -199,12 +194,10 @@ test_expect_success 'absorbgitdirs works with a submodule with worktree config'\n \n \ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n \t\t\"$submodule_gitdir\" >expect &&\n-\tgit -C sub5 config submodule.superprojectGitDir >actual &&\n-\n-\ttest_cmp expect actual &&\n+\ttest_cmp_submodule_superprojectgitdir sub5 &&\n \n \t# make sure the config went into the submodule config.worktree\n-\ttest_file_not_empty \"$submodule_gitdir/config.worktree\"\n+\ttest_file_not_empty_superprojectgitdir sub5 config.worktree\n \t)\n '\n \n-- \n2.34.0.796.g2c87ed6146a\n\n"},{"id":"441533","messageId":"20211117232846.2596110-1-jonathantanmy@google.com","threadId":"56920","inReplyTo":"20211117005701.371808-1-emilyshaffer@google.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-17T23:28:46Z","receivedAt":"2021-11-17T23:28:55Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Emily Shaffer writes:\n> For the original cover letter, see\n> https://lore.kernel.org/git/20210611225428.1208973-1-emilyshaffer%40google.com.\n\nAlso for reference, v4 and v5 (as a reply to v4) can be found here:\nhttps://lore.kernel.org/git/20211014203416.2802639-1-emilyshaffer@google.com/\n\n> Since v5:\n> \n> A couple things. Firstly, a semantics change *back* to the semantics of\n> v3 - we map from gitdir to gitdir, *not* from common dir to common dir,\n> so that theoretically a submodule with multiple worktrees in multiple\n> superproject worktrees will be able to figure out which worktree of the\n> superproject it's in. (Realistically, that's not really possible right\n> now, but I'd like to change that soon.)\n\nMakes sense. Also, thanks for the tests covering what happens when this\nis run from worktrees.\n\n> Secondly, a rewording of comments and commit messages to indicate that\n> this isn't a cache of some expensive operation, but rather intended to\n> be the source of truth for all submodules. I also added a fifth commit\n> rewriting `git rev-parse --show-superproject-working-tree` to\n> demonstrate what that means in practice - but from a practical\n> standpoint, I'm a little worried about that fifth patch. More details in\n> the patch 5 description.\n\nOK - this is not the \"this variable being missing is OK\" idea that I had\n[1], but we want to be able to depend on it to some extent. (And it is\nnot a cache either - we are not planning to perform an operation to\nobtain the superproject gitdir if the cache is missing, but we are just\ngoing to assume that there is no superproject.)\n\nTo that end, the 5th patch is misleading - it is behaving exactly like a\ncache. I think it's better to drop it.\n\nWhat would make sense to me (and seems to be in the spirit of this patch\nset) is to describe this as something that Git commands can rely on to\ndetermine if the current repo is a submodule, for performance reasons.\nSo maybe Git commands/parameters that directly reference the submodule\nconcept like \"--show-superproject-working-tree\" will work hard to find\nthe superproject (by searching the filesystem), but those that do not\n(e.g. \"git status\") can make assumptions.\n\nMaking this variable a source of truth wouldn't work, I think, because\nthe source of truth is whether this repo appears in a .gitmodules file\n(and that hasn't changed).\n\nTo this end, I'll comment on the changes I'd like to see on the\nindividual patches too.\n\n[1] https://lore.kernel.org/git/20210727174650.2462099-1-jonathantanmy@google.com/\n"},{"id":"441537","messageId":"20211117234300.2598132-1-jonathantanmy@google.com","threadId":"56920","inReplyTo":"20211117005701.371808-3-emilyshaffer@google.com","subject":"Re: [PATCH v6 2/5] introduce submodule.superprojectGitDir record","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2021-11-17T23:43:00Z","receivedAt":"2021-11-17T23:43:11Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"Emily Shaffer writes:\n> Teach submodules a reference to their superproject's gitdir. This allows\n> us to A) know that we're running from a submodule, and B) have a\n> shortcut to the superproject's vitals, for example, configs.\n\nIf we're going with my proposal [1], I think it's worth further\nexplaining the concept right at the beginning of the commit message:\n\n Teach submodules a config variable referencing their superproject's\n gitdir. Git commands may rely on this reference to determine if the\n current repo is a submodule to another repo: if this reference is\n absent, Git may assume that the current repo is not a submodule. In\n practice, commands and arguments that specifially reference the\n submodule relationship (like \"rev-parse\n --show-superproject-working-tree\") will still search the ancestor\n directory, but others (say, a \"git status\" that prints the submodule's\n status in relation to its superproject or a config option that lets\n the superproject and submodule share config) would not.\n\n[1] https://lore.kernel.org/git/20211117232846.2596110-1-jonathantanmy@google.com/\n\n> By using a relative path instead of an absolute path, we can move the\n> superproject directory around on the filesystem without breaking the\n> submodule's pointer. And by using the path from gitdir to gitdir, we can\n> move the submodule within the superproject's tree structure without\n> breaking the submodule's pointer, too. Finally, by pointing at the\n> superproject's worktree gitdir (if it exists), we ensure that we can\n> tell which worktree contains our submodule.\n\nOK.\n\n> Since this hint value is only introduced during new submodule creation\n> via `git submodule add`, though, there is more work to do to allow the\n> record to be created at other times.\n\nThis is not a hint anymore. I would reword as:\n\n This commit teaches \"git submodule add\" to add the aforementioned\n config variable. Subsequent commits will teach other commands to do\n so.\n\n> Once this new config is reliably in place, we can use it to know\n> definitively that we are working in a submodule, and to know which\n> superproject we are a submodule of. This allows us to do some\n> value-added behavior, like letting \"git status\" print additional info\n> about the submodule's status in relation to its superproject, or like\n> letting the superproject and submodule share an additional config file\n> separate from either one's local config.\n\nI folded this into the first paragraph above, so this would no longer\nneeded if you used my suggestion above.\n\n> +submodule.superprojectGitDir::\n> +\tThe relative path from the submodule's gitdir to its superproject's\n> +\tgitdir. When Git is run in a repository, it usually makes no\n> +\tdifference whether this repository is standalone or a submodule, but if\n> +\tthis configuration variable is present, additional behavior may be\n> +\tpossible, such as \"git status\" printing additional information about\n> +\tthis submodule's status with respect to its superproject. This config\n> +\tshould only be present in projects which are submodules, but is not\n> +\tguaranteed to be present in every submodule, so only optional\n> +\tvalue-added behavior should be linked to it. It is set automatically\n> +\tduring submodule creation.\n\nIf we're going to rely on the variable more, this needs to be updated.\nMaybe:\n\n If this repository is a submodule, the relative path from this repo's\n gitdir to its superproject's gitdir. Git commands may rely on this\n reference to determine if the current repo is a submodule to another\n repo: if this reference is absent, Git may assume that the current\n repo is not a submodule (this does not make a difference to most Git\n commands). It is set automatically during ??\n\n(and probably each subsequent patch will need to update the ??)\n"},{"id":"442196","messageId":"YZ1KLNwsxx7IR1+5@google.com","threadId":"56920","inReplyTo":"RFC-cover-0.2-00000000000-20211117T113134Z-avarab@gmail.com","subject":"Re: [RFC PATCH 0/2] submodule: test what happens if submodule.superprojectGitDir isn't around","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-23T20:08:12Z","receivedAt":"2021-11-23T20:08:19Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Wed, Nov 17, 2021 at 12:43:38PM +0100, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Tue, Nov 16 2021, Emily Shaffer wrote:\n> \n> > [...]\n> > A couple things. Firstly, a semantics change *back* to the semantics of\n> > v3 - we map from gitdir to gitdir, *not* from common dir to common dir,\n> > so that theoretically a submodule with multiple worktrees in multiple\n> > superproject worktrees will be able to figure out which worktree of the\n> > superproject it's in. (Realistically, that's not really possible right\n> > now, but I'd like to change that soon.)\n> >\n> > Secondly, a rewording of comments and commit messages to indicate that\n> > this isn't a cache of some expensive operation, but rather intended to\n> > be the source of truth for all submodules. I also added a fifth commit\n> > rewriting `git rev-parse --show-superproject-working-tree` to\n> > demonstrate what that means in practice - but from a practical\n> > standpoint, I'm a little worried about that fifth patch. More details in\n> > the patch 5 description.\n> >\n> > I did discuss Ævar's idea of relying on in-process filesystem digging to\n> > find the superproject's gitdir with the rest of the Google team, but in\n> > the end decided that there are some worries about filesystem digging in\n> > this way (namely, some ugly interactions with network drives that are\n> > actually already an issue for Googler Linux machines). Plus, the allure\n> > of being able to definitively know that we're a submodule is pretty\n> > strong. ;) But overall, this is the direction I'd prefer to keep going \n> > in, rather than trying to guess from the filesystem going forward.\n> \n> Did you try running the ad-hoc benchmark I included in [1] on that\n> Google NFS? I've dealt with some slow-ish network filesystems, but if\n> it's slower than AIX's local FS (where I couldn't see a difference) I'd\n> put money on it being a cross-Atlantic mount or something :)\n> \n> Re your:\n> \n> \"this isn't a cache of some expensive operation, but rather intended to be the source of truth for all submodules.\"\n> \n> In your 5/5 it says, in seeming contradiction to this:\n> \n> This commit may be more of an RFC - to demonstrate what life looks like\n> if we use submodule.superprojectGitDir as the source of truth. But since\n> 'git rev-parse --show-superproject-working-tree' is used in a lot of\n> scripts in the wild[1], I'm not so sure it's a great example.\n> \n> To be honest, I'd prefer to die(\"Try running 'git submodule update'\")\n> here, but I don't think that's very script-friendly. However, falling\n> back on the old implementation kind of undermines the idea of treating\n> submodule.superprojectGitDir as the point of truth.\n> \n> Most of what I've been suggesting in my [1] and related is that I'm\n> confused about if & how this is a pure caching mechanism.\n> \n> Removing mentions of it being a cache but it seemingly still being a\n> cache at the tip of this series has just added to that confusion for\n> me :)\n\nYeah, I think this was a bad choice for me to include that patch. I was\nreally hopeful that I could show off \"look, we don't need to ever hunt\nin the FS above us\", but for established repos, that's a bad idea\n(because lots of people are already using this 'git rev-parse\n--show-superproject-work-tree' thing in scripts, like I mentioned). So I\nthink it was a mistake to include it at all. Rather, I think it's\nprobably a better idea to treat that particular entry point as \"legacy\"\nand implement other things using 'submodule.superprojectGitDir'\ndirectly.\n\nBecause the patch 5 illustrates: \"I'm saying that this new config isn't\na cache, but look, here's how I can treat it like a cache that might be\ninvalid and here's how I can fall back on a potentially expensive\noperation anyways.\" I think I could have illustrated it a little better\nwith something like \"here's a brand new 'git rev-parse\n--show-superproject-gitdir'\" which directly calls on the new config.\n\nSo, sorry about that.\n\n> \n> Anyway. While I do think this caching mechanism is probably\n> unnecessary in the short to medium term, i.e. it seems to the extent\n> that it was ever needed was due to some bridging of *.sh<->*.c that\n> we're *this* close to eliminating anyway.\n> \n> But maybe I'm wrong. The benchmark I suggested above on that Google\n> NFS might be indicative. I don't really see how something that'll be\n> doing a bunch of FS ops anyway is going to be noticeably slower with\n> that approach, but maybe opening the index/tree of the superproject is\n> more expensive than I'm expecting.\n> \n> In any case, all of that's not the hill I'm picking to die on. If\n> you'd like to go ahead with this cache-or-not-a-cache then sure, I\n> won't belabor that point.\n\nYeah, I think I would. I've heard some serious reservations from others\non my team about trying to use filesystem traversal here at all, so I\nthink that would be an uphill battle.\n\n> \n> I *do* strongly think if we're doing so though that we should have\n> something like this on top. I.e. let's test wha happens if we do and\n> don't have this \"caching\" variable, which is demonstrably easy to do.\n> \n> Benchmarking the two gives me:\n> \n> $ git hyperfine -L rev HEAD~0 -L s true,false -s 'make -j8 all' '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR={s} ./t7412-submodule-absorbgitdirs.sh)'\n> Benchmark 1: (cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=true ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0\n> Time (mean ± σ): 545.9 ms ± 1.6 ms [User: 490.3 ms, System: 114.0 ms]\n> Range (min … max): 543.5 ms … 548.1 ms 10 runs\n> \n> Benchmark 2: (cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=false ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0\n> Time (mean ± σ): 537.9 ms ± 11.4 ms [User: 476.8 ms, System: 117.6 ms]\n> Range (min … max): 532.7 ms … 570.1 ms 10 runs\n> \n> Summary\n> '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=false ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0' ran\n> 1.01 ± 0.02 times faster than '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=true ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0'\n> \n> I.e. not using the cache is either indistinguishable or a bit faster\n> (the \"a bit faster\" is definitely due to just running less test code\n> though).\n\nYeah, once again, I think it is better to treat \"git rev-parse\n--show-superproject-work-tree\" as \"legacy\" and to rely solely on the\nconfig for new options, meaning that \"what happens without this\nvariable\" is as simple as \"we treat it like it's a standalone repository\nwith no superproject\", rather than a performance difference at all.\n\n - Emily\n\n> \n> I'm sending this before the CI run[2] finishes (which now tests both\n> modes), but both of these work for me locally on a full test suite\n> run.\n> \n> 1. https://lore.kernel.org/git/211109.86v912dtfw.gmgdl@evledraar.gmail.com/\n> 2. https://github.com/avar/git/runs/4237446991?check_suite_focus=true\n> \n> Ævar Arnfjörð Bjarmason (2):\n> submodule tests: fix potentially broken \"config .. --unset\"\n> submodule: add test mode for checking absence of \"superProjectGitDir\"\n> \n> ci/run-build-and-tests.sh | 1 +\n> git-submodule.sh | 2 +-\n> submodule.c | 7 +++++++\n> t/lib-submodule-superproject.sh | 24 ++++++++++++++++++++++++\n> t/t7406-submodule-update.sh | 13 ++++++-------\n> t/t7412-submodule-absorbgitdirs.sh | 19 ++++++-------------\n> 6 files changed, 45 insertions(+), 21 deletions(-)\n> create mode 100644 t/lib-submodule-superproject.sh\n> \n> -- \n> 2.34.0.796.g2c87ed6146a\n> \n"},{"id":"442198","messageId":"YZ1PA3YBdOfg0/cf@google.com","threadId":"56920","inReplyTo":"20211117232846.2596110-1-jonathantanmy@google.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2021-11-23T20:28:51Z","receivedAt":"2021-11-23T20:28:58Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Wed, Nov 17, 2021 at 03:28:46PM -0800, Jonathan Tan wrote:\n> \n> Emily Shaffer writes:\n> > For the original cover letter, see\n> > https://lore.kernel.org/git/20210611225428.1208973-1-emilyshaffer%40google.com.\n> \n> Also for reference, v4 and v5 (as a reply to v4) can be found here:\n> https://lore.kernel.org/git/20211014203416.2802639-1-emilyshaffer@google.com/\n> \n> > Since v5:\n> > \n> > A couple things. Firstly, a semantics change *back* to the semantics of\n> > v3 - we map from gitdir to gitdir, *not* from common dir to common dir,\n> > so that theoretically a submodule with multiple worktrees in multiple\n> > superproject worktrees will be able to figure out which worktree of the\n> > superproject it's in. (Realistically, that's not really possible right\n> > now, but I'd like to change that soon.)\n> \n> Makes sense. Also, thanks for the tests covering what happens when this\n> is run from worktrees.\n> \n> > Secondly, a rewording of comments and commit messages to indicate that\n> > this isn't a cache of some expensive operation, but rather intended to\n> > be the source of truth for all submodules. I also added a fifth commit\n> > rewriting `git rev-parse --show-superproject-working-tree` to\n> > demonstrate what that means in practice - but from a practical\n> > standpoint, I'm a little worried about that fifth patch. More details in\n> > the patch 5 description.\n> \n> OK - this is not the \"this variable being missing is OK\" idea that I had\n> [1], but we want to be able to depend on it to some extent. (And it is\n> not a cache either - we are not planning to perform an operation to\n> obtain the superproject gitdir if the cache is missing, but we are just\n> going to assume that there is no superproject.)\n> \n> To that end, the 5th patch is misleading - it is behaving exactly like a\n> cache. I think it's better to drop it.\n\nYeah, I think you are right.\n\n> \n> What would make sense to me (and seems to be in the spirit of this patch\n> set) is to describe this as something that Git commands can rely on to\n> determine if the current repo is a submodule, for performance reasons.\n> So maybe Git commands/parameters that directly reference the submodule\n> concept like \"--show-superproject-working-tree\" will work hard to find\n> the superproject (by searching the filesystem), but those that do not\n> (e.g. \"git status\") can make assumptions.\n\nOh interesting, I like the idea of that distinction. Thanks for the\nsuggestion.\n\nI wonder, though, how best to delineate. I'd almost rather say like I\nsuggested to Ævar\n(https://lore.kernel.org/git/YZ1KLNwsxx7IR1%2B5%40google.com), that we\ncan treat \"--show-superproject-working-tree\" as a \"legacy\" option people\nare already using, and treat anything added from now on as \"if it\ndoesn't think it is a submodule, you should 'git submodule update' in\nthe superproject\". That relies on us being able to reliably keep the\nvalue of submodule.superprojectGitDir correct, but I think the\ngitdir->gitdir linking helps with that (as opposed to earlier\niterations).\n> \n> Making this variable a source of truth wouldn't work, I think, because\n> the source of truth is whether this repo appears in a .gitmodules file\n> (and that hasn't changed).\n\nInterestingly, '--show-superproject-working-tree' doesn't check the\n.gitmodules file at all. It checks whether the project found in the\nfilesystem above it knows about an object named path/to/superproject/\nwhich is a gitlink - that is, it asks the index, not .gitmodules, as far\nas I understand it. So if the only existing alternative to\nsubmodule.superprojectGitDir isn't treating .gitmodules as source of\ntruth, then what does that mean?\n\n> \n> To this end, I'll comment on the changes I'd like to see on the\n> individual patches too.\n> \n> [1] https://lore.kernel.org/git/20210727174650.2462099-1-jonathantanmy@google.com/\n"},{"id":"442223","messageId":"211124.86a6hue2wk.gmgdl@evledraar.gmail.com","threadId":"56920","inReplyTo":"YZ1KLNwsxx7IR1+5@google.com","subject":"Re: [RFC PATCH 0/2] submodule: test what happens if submodule.superprojectGitDir isn't around","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-11-24T01:38:20Z","receivedAt":"2021-11-24T01:52:16Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Nov 23 2021, Emily Shaffer wrote:\n\n> On Wed, Nov 17, 2021 at 12:43:38PM +0100, Ævar Arnfjörð Bjarmason wrote:\n>> \n>> On Tue, Nov 16 2021, Emily Shaffer wrote:\n>> \n>> > [...]\n>> > A couple things. Firstly, a semantics change *back* to the semantics of\n>> > v3 - we map from gitdir to gitdir, *not* from common dir to common dir,\n>> > so that theoretically a submodule with multiple worktrees in multiple\n>> > superproject worktrees will be able to figure out which worktree of the\n>> > superproject it's in. (Realistically, that's not really possible right\n>> > now, but I'd like to change that soon.)\n>> >\n>> > Secondly, a rewording of comments and commit messages to indicate that\n>> > this isn't a cache of some expensive operation, but rather intended to\n>> > be the source of truth for all submodules. I also added a fifth commit\n>> > rewriting `git rev-parse --show-superproject-working-tree` to\n>> > demonstrate what that means in practice - but from a practical\n>> > standpoint, I'm a little worried about that fifth patch. More details in\n>> > the patch 5 description.\n>> >\n>> > I did discuss Ævar's idea of relying on in-process filesystem digging to\n>> > find the superproject's gitdir with the rest of the Google team, but in\n>> > the end decided that there are some worries about filesystem digging in\n>> > this way (namely, some ugly interactions with network drives that are\n>> > actually already an issue for Googler Linux machines). Plus, the allure\n>> > of being able to definitively know that we're a submodule is pretty\n>> > strong. ;) But overall, this is the direction I'd prefer to keep\n>> > going\n>> > in, rather than trying to guess from the filesystem going forward.\n>> \n>> Did you try running the ad-hoc benchmark I included in [1] on that\n>> Google NFS? I've dealt with some slow-ish network filesystems, but if\n>> it's slower than AIX's local FS (where I couldn't see a difference) I'd\n>> put money on it being a cross-Atlantic mount or something :)\n>> \n>> Re your:\n>> \n>> \"this isn't a cache of some expensive operation, but rather\n>> intended to be the source of truth for all submodules.\"\n>> \n>> In your 5/5 it says, in seeming contradiction to this:\n>> \n>> This commit may be more of an RFC - to demonstrate what life looks like\n>> if we use submodule.superprojectGitDir as the source of truth. But since\n>> 'git rev-parse --show-superproject-working-tree' is used in a lot of\n>> scripts in the wild[1], I'm not so sure it's a great example.\n>> \n>> To be honest, I'd prefer to die(\"Try running 'git submodule update'\")\n>> here, but I don't think that's very script-friendly. However, falling\n>> back on the old implementation kind of undermines the idea of treating\n>> submodule.superprojectGitDir as the point of truth.\n>> \n>> Most of what I've been suggesting in my [1] and related is that I'm\n>> confused about if & how this is a pure caching mechanism.\n>> \n>> Removing mentions of it being a cache but it seemingly still being a\n>> cache at the tip of this series has just added to that confusion for\n>> me :)\n>\n> Yeah, I think this was a bad choice for me to include that patch. I was\n> really hopeful that I could show off \"look, we don't need to ever hunt\n> in the FS above us\", but for established repos, that's a bad idea\n> (because lots of people are already using this 'git rev-parse\n> --show-superproject-work-tree' thing in scripts, like I mentioned). So I\n> think it was a mistake to include it at all. Rather, I think it's\n> probably a better idea to treat that particular entry point as \"legacy\"\n> and implement other things using 'submodule.superprojectGitDir'\n> directly.\n>\n> Because the patch 5 illustrates: \"I'm saying that this new config isn't\n> a cache, but look, here's how I can treat it like a cache that might be\n> invalid and here's how I can fall back on a potentially expensive\n> operation anyways.\" I think I could have illustrated it a little better\n> with something like \"here's a brand new 'git rev-parse\n> --show-superproject-gitdir'\" which directly calls on the new config.\n>\n> So, sorry about that.\n\nThe RFC series here shows that the config & the on-the-fly discovery 1=1\nmap to each other (at least in terms of the tests your series adds).\n\nWhy wouldn't the same be the case with a --show-superproject-gitdir?\nUnless the fallback implementation was intentionally removed.\n\n>> \n>> Anyway. While I do think this caching mechanism is probably\n>> unnecessary in the short to medium term, i.e. it seems to the extent\n>> that it was ever needed was due to some bridging of *.sh<->*.c that\n>> we're *this* close to eliminating anyway.\n>> \n>> But maybe I'm wrong. The benchmark I suggested above on that Google\n>> NFS might be indicative. I don't really see how something that'll be\n>> doing a bunch of FS ops anyway is going to be noticeably slower with\n>> that approach, but maybe opening the index/tree of the superproject is\n>> more expensive than I'm expecting.\n>> \n>> In any case, all of that's not the hill I'm picking to die on. If\n>> you'd like to go ahead with this cache-or-not-a-cache then sure, I\n>> won't belabor that point.\n>\n> Yeah, I think I would. I've heard some serious reservations from others\n> on my team about trying to use filesystem traversal here at all, so I\n> think that would be an uphill battle.\n\nDo they have any benchmarks to share? :)\n\n>> \n>> I *do* strongly think if we're doing so though that we should have\n>> something like this on top. I.e. let's test wha happens if we do and\n>> don't have this \"caching\" variable, which is demonstrably easy to do.\n>> \n>> Benchmarking the two gives me:\n>> \n>> $ git hyperfine -L rev HEAD~0 -L s true,false -s 'make -j8 all' '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR={s} ./t7412-submodule-absorbgitdirs.sh)'\n>> Benchmark 1: (cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=true ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0\n>> Time (mean ± σ): 545.9 ms ± 1.6 ms [User: 490.3 ms, System: 114.0 ms]\n>> Range (min … max): 543.5 ms … 548.1 ms 10 runs\n>> \n>> Benchmark 2: (cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=false ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0\n>> Time (mean ± σ): 537.9 ms ± 11.4 ms [User: 476.8 ms, System: 117.6 ms]\n>> Range (min … max): 532.7 ms … 570.1 ms 10 runs\n>> \n>> Summary\n>> '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=false ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0' ran\n>> 1.01 ± 0.02 times faster than '(cd t && GIT_TEST_SUBMODULE_CACHE_SUPERPROJECT_DIR=true ./t7412-submodule-absorbgitdirs.sh)' in 'HEAD~0'\n>> \n>> I.e. not using the cache is either indistinguishable or a bit faster\n>> (the \"a bit faster\" is definitely due to just running less test code\n>> though).\n>\n> Yeah, once again, I think it is better to treat \"git rev-parse\n> --show-superproject-work-tree\" as \"legacy\" and to rely solely on the\n> config for new options, meaning that \"what happens without this\n> variable\" is as simple as \"we treat it like it's a standalone repository\n> with no superproject\", rather than a performance difference at all.\n\nWhy would it be better to have a hard reliance on the config for new\nfeatures or options? Your original CL says[1]:\n\n It's expensive and non-definitive to try and guess whether or not the\n current repo is a submodule.\n\nMaybe that's the case somewhere, I haven't been able to dig up the\n\"expensive\" part (aside from the fixable case of exec-ing rev-parse in a\nloop).\n\nWhich leaves \"non-definitive\", that may be the case, but as the RFC\npatches here show if there is such a case, it's not covered by any of\nyour existing tests in this series.\n\nI'd think if those two things hold true (not expensive, and we can\nunambiguously discover it on the fly) our bias should be towards not\nintroducing an indirection in the config, as such a thing needs to be\nkept up-to-date, and which may become inaccurate for whatever\nreason. Whereas the FS relationship between the won't ever be\nout-of-date.\n\n1. https://lore.kernel.org/git/20210611225428.1208973-1-emilyshaffer@google.com\n"},{"id":"447671","messageId":"20220203215914.683922-1-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20211117005701.371808-1-emilyshaffer@google.com","subject":"[PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-02-03T21:59:10Z","receivedAt":"2022-02-03T21:59:23Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"For the original cover letter, see\nhttps://lore.kernel.org/git/20210611225428.1208973-1-emilyshaffer%40google.com.\n\nCI run: https://github.com/nasamuffin/git/actions/runs/1780282431\n\nSince v6:\n\nI've dropped the fifth commit to use this new config for `git rev-parse\n--show-superproject-working-tree`. I think it did more harm than good -\nthat tool uses an odd way to determine whether the superproject is\nactually the superproject, anyways.\n\nI poked a little bit at trying to find some benchmark to demonstrate\nthat \"submodule.superprojectGitDir\" is actually faster - but I didn't\nend up able to write one without writing a ton of new code to traverse\nthe filesystem. To be honest, I'm not all that interested in performance\n- I want the config added for correctness, instead.\n\nSo, the only real changes between v6 and v7 are some documentation\nchanges suggested by Jonathan Tan\n(https://lore.kernel.org/git/20211117234300.2598132-1-jonathantanmy%40google.com).\n\nSince v5:\n\nA couple things. Firstly, a semantics change *back* to the semantics of\nv3 - we map from gitdir to gitdir, *not* from common dir to common dir,\nso that theoretically a submodule with multiple worktrees in multiple\nsuperproject worktrees will be able to figure out which worktree of the\nsuperproject it's in. (Realistically, that's not really possible right\nnow, but I'd like to change that soon.)\n\nSecondly, a rewording of comments and commit messages to indicate that\nthis isn't a cache of some expensive operation, but rather intended to\nbe the source of truth for all submodules. I also added a fifth commit\nrewriting `git rev-parse --show-superproject-working-tree` to\ndemonstrate what that means in practice - but from a practical\nstandpoint, I'm a little worried about that fifth patch. More details in\nthe patch 5 description.\n\nI did discuss Ævar's idea of relying on in-process filesystem digging to\nfind the superproject's gitdir with the rest of the Google team, but in\nthe end decided that there are some worries about filesystem digging in\nthis way (namely, some ugly interactions with network drives that are\nactually already an issue for Googler Linux machines). Plus, the allure\nof being able to definitively know that we're a submodule is pretty\nstrong. ;) But overall, this is the direction I'd prefer to keep going\nin, rather than trying to guess from the filesystem going forward.\n\nSince v4:\n\nThe only real change here is a slight semantics change to map from\n to . In every case\n*except* for when the superproject has a worktree, this changes nothing.\nFor the case when the superproject has a worktree, this means that now\nsubmodules will refer to the general superproject common dir (e.g. no\nworktree-specific refs or configs or whatnot).\n\nI *think* that because a submodule should exist in the context of the\ncommon dir, not the worktree gitdir, that is ok. However, it does mean\nit would be difficult to do something like sharing a config specific to\nthe worktree (the initial goal of this series).\n\n$ROOT/.git\n$ROOT/.git/config.superproject <- shared by $ROOT/.git/modules/sub\n$ROOT/.git/modules/sub <- points to $ROOT/.git\n$ROOT/.git/worktrees/wt\n$ROOT/.git/worktrees/wt/config.superproject <- contains a certain config-based pre-commit hook\n\nIf the submodule only knows about the common dir, that is tough, because\nthe submodule would basically have to guess which worktree it's in from\nits own path. There would be no way for '$WT/sub' to inherit\n'$ROOT/.git/worktrees/wt/config.superproject'.\n\nThat said... right now, we don't support submodules in worktrees very\nwell at all. A submodule in a worktree will get a brand new gitdir in\n$ROOT/.git/worktrees/modules/ (and that brand new gitdir would point to\nthe super's common dir). So I think we can punt on this entire question\nuntil we teach submodules and worktrees to play more gracefully together\n(it's on my long list...), and at that time we can probably introduce a\npointer from $ROOT/.git/modules/sub/worktrees/wt/ to\n$ROOT/.git/worktrees/wt/....\n\nOr, to summarize the long ramble above: \"this is still kind of weird\nwith worktrees, but let's fix it later when we fix worktrees more\nthoroughly\".\n\n(More rambling about worktree weirdness here:\nhttps://lore.kernel.org/git/YYRaII8YWVxlBqsF%40google.com )\n\n\nSince v3, a pretty major change: the semantics of\nsubmodule.superprojectGitDir has changed, to point from the submodule's\ngitdir to the superproject's gitdir (in v3 and earlier, we kept a path\nfrom the submodule's *worktree* to the superproject's gitdir instead).\nThis cleans up some of the confusions about the behavior when a\nsubmodule worktree moves around in the superproject's tree, or in a\nfuture when we support submodules having multiple worktrees.\n\nI also tried to simplify the tests to use 'test-tool path-utils\nrelative_path' everywhere - I think that makes them much more clear for\na test reader, but if you're reviewing and it isn't obvious what we're\ntesting for, please speak up.\n\nI think this is pretty mature and there was a lot of general agreement\nthat the gitdir->gitdir association was the way to go, so please be\nbrutal and look for nits, leaks, etc. this round ;)\n[/v4 cover letter]\n\n\nEmily Shaffer (4):\n t7400-submodule-basic: modernize inspect() helper\n introduce submodule.superprojectGitDir record\n submodule: record superproject gitdir during absorbgitdirs\n submodule: record superproject gitdir during 'update'\n\n Documentation/config/submodule.txt | 9 ++++\n builtin/submodule--helper.c | 11 ++++\n git-submodule.sh | 15 ++++++\n submodule.c | 23 +++++++++\n t/t7400-submodule-basic.sh | 54 +++++++++++---------\n t/t7406-submodule-update.sh | 27 ++++++++++\n t/t7412-submodule-absorbgitdirs.sh | 82 +++++++++++++++++++++++++++++-\n 7 files changed, 194 insertions(+), 27 deletions(-)\n\nRange-diff against v6:\n1: f1b08a7057 = 1: 251510c687 t7400-submodule-basic: modernize inspect() helper\n2: d46c8439ab ! 2: 1a85deb1c5 introduce submodule.superprojectGitDir record\n @@ Metadata\n ## Commit message ##\n introduce submodule.superprojectGitDir record\n \n - Teach submodules a reference to their superproject's gitdir. This allows\n - us to A) know that we're running from a submodule, and B) have a\n - shortcut to the superproject's vitals, for example, configs.\n + Teach submodules a config variable referencing to their superproject's\n + gitdir. Git commands can rely on this reference to determine whether the\n + current repo is a submodule to another repo. If this reference is\n + absent, Git may assume the current repo is not a submodule.\n +\n + In practice, some commands which specifically reference the submodule\n + relationship, like 'git rev-parse --show-superproject-working-tree',\n + will still search the parent directory. Other newly added or implicit\n + behavior, such as \"git status\" showing the submodule's status in\n + relation to the superproject, or a config which is shared between the\n + superproject and the submodule, should not search the parent directory\n + and instead rely on this \"submodule.superprojectGitDir\" config.\n \n By using a relative path instead of an absolute path, we can move the\n superproject directory around on the filesystem without breaking the\n @@ Commit message\n superproject's worktree gitdir (if it exists), we ensure that we can\n tell which worktree contains our submodule.\n \n - Since this hint value is only introduced during new submodule creation\n - via `git submodule add`, though, there is more work to do to allow the\n - record to be created at other times.\n -\n - Once this new config is reliably in place, we can use it to know\n - definitively that we are working in a submodule, and to know which\n - superproject we are a submodule of. This allows us to do some\n - value-added behavior, like letting \"git status\" print additional info\n - about the submodule's status in relation to its superproject, or like\n - letting the superproject and submodule share an additional config file\n - separate from either one's local config.\n + This commit teaches \"git submodule add\" to add the aformentioned config\n + variable. Subsequent commits will teach other commands to do so.\n \n Signed-off-by: Emily Shaffer \n Helped-by: Junio C Hamano \n @@ Documentation/config/submodule.txt: submodule.alternateErrorStrategy::\n \tclone proceeds as if no alternate was specified.\n +\n +submodule.superprojectGitDir::\n -+\tThe relative path from the submodule's gitdir to its superproject's\n -+\tgitdir. When Git is run in a repository, it usually makes no\n -+\tdifference whether this repository is standalone or a submodule, but if\n -+\tthis configuration variable is present, additional behavior may be\n -+\tpossible, such as \"git status\" printing additional information about\n -+\tthis submodule's status with respect to its superproject. This config\n -+\tshould only be present in projects which are submodules, but is not\n -+\tguaranteed to be present in every submodule, so only optional\n -+\tvalue-added behavior should be linked to it. It is set automatically\n -+\tduring submodule creation.\n ++\tIf this repository is a submodule, the relative path from this repo's\n ++\tgitdir to its superproject's gitdir. Git commands may rely on this\n ++\treference to determine whether the current repo is a submodule to\n ++\tanother repo; if this reference is absent, Git will treat the current\n ++\trepo as though it is not a submodule (this does not make a difference to\n ++\tmost Git commands). It is set automatically during submodule creation.\n \n ## builtin/submodule--helper.c ##\n @@ builtin/submodule--helper.c: static int clone_submodule(struct module_clone_data *clone_data)\n3: 63ddaf5608 ! 3: 7a44b0edf9 submodule: record superproject gitdir during absorbgitdirs\n @@ Commit message\n \n Signed-off-by: Emily Shaffer \n \n + ## Documentation/config/submodule.txt ##\n +@@ Documentation/config/submodule.txt: submodule.superprojectGitDir::\n + \treference to determine whether the current repo is a submodule to\n + \tanother repo; if this reference is absent, Git will treat the current\n + \trepo as though it is not a submodule (this does not make a difference to\n +-\tmost Git commands). It is set automatically during submodule creation.\n ++\tmost Git commands). It is set automatically during submodule creation\n ++\tand 'git submodule absorbgitdir'.\n +\n ## submodule.c ##\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *path)\n \tchar *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL;\n4: 33a582ef13 ! 4: 63e736e69d submodule: record superproject gitdir during 'update'\n @@ Commit message\n \n Signed-off-by: Emily Shaffer \n \n + ## Documentation/config/submodule.txt ##\n +@@ Documentation/config/submodule.txt: submodule.superprojectGitDir::\n + \treference to determine whether the current repo is a submodule to\n + \tanother repo; if this reference is absent, Git will treat the current\n + \trepo as though it is not a submodule (this does not make a difference to\n +-\tmost Git commands). It is set automatically during submodule creation\n +-\tand 'git submodule absorbgitdir'.\n ++\tmost Git commands). It is set automatically during submodule creation,\n ++\tupdate, and 'git submodule absorbgitdir'.\n +\n ## git-submodule.sh ##\n @@ git-submodule.sh: cmd_update()\n \t\t\t;;\n5: a8b5d40a77 < -: ---------- submodule: use config to find superproject worktree\n-- \n2.35.0.263.gb82422642f-goog\n\n"},{"id":"447672","messageId":"20220203215914.683922-2-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220203215914.683922-1-emilyshaffer@google.com","subject":"[PATCH v7 1/4] t7400-submodule-basic: modernize inspect() helper","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-02-03T21:59:11Z","receivedAt":"2022-02-03T21:59:28Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Since the inspect() helper in the submodule-basic test suite was\nwritten, 'git -C ' was added. By using -C, we no longer need a\nreference to the base directory for the test. This simplifies callsites,\nand will make the addition of other arguments in later patches more\nreadable.\n\nSigned-off-by: Emily Shaffer \n---\n t/t7400-submodule-basic.sh | 40 +++++++++++++++-----------------------\n 1 file changed, 16 insertions(+), 24 deletions(-)\n\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex e7cec2e457..40cf8d89aa 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -107,23 +107,15 @@ test_expect_success 'setup - repository to add submodules to' '\n # generates, which will expand symbolic links.\n submodurl=$(pwd -P)\n \n-listbranches() {\n-\tgit for-each-ref --format='%(refname)' 'refs/heads/*'\n-}\n-\n inspect() {\n-\tdir=$1 &&\n-\tdotdot=\"${2:-..}\" &&\n-\n-\t(\n-\t\tcd \"$dir\" &&\n-\t\tlistbranches >\"$dotdot/heads\" &&\n-\t\t{ git symbolic-ref HEAD || :; } >\"$dotdot/head\" &&\n-\t\tgit rev-parse HEAD >\"$dotdot/head-sha1\" &&\n-\t\tgit update-index --refresh &&\n-\t\tgit diff-files --exit-code &&\n-\t\tgit clean -n -d -x >\"$dotdot/untracked\"\n-\t)\n+\tsub_dir=$1 &&\n+\n+\tgit -C \"$sub_dir\" for-each-ref --format='%(refname)' 'refs/heads/*' >heads &&\n+\t{ git -C \"$sub_dir\" symbolic-ref HEAD || :; } >head &&\n+\tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n+\tgit -C \"$sub_dir\" update-index --refresh &&\n+\tgit -C \"$sub_dir\" diff-files --exit-code &&\n+\tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n \n test_expect_success 'submodule add' '\n@@ -146,7 +138,7 @@ test_expect_success 'submodule add' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod ../.. &&\n+\tinspect addtest/submod &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -248,7 +240,7 @@ test_expect_success 'submodule add --branch' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod-branch ../.. &&\n+\tinspect addtest/submod-branch &&\n \ttest_cmp expect-heads heads &&\n \ttest_cmp expect-head head &&\n \ttest_must_be_empty untracked\n@@ -264,7 +256,7 @@ test_expect_success 'submodule add with ./ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotsubmod/frotz ../../.. &&\n+\tinspect addtest/dotsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -280,7 +272,7 @@ test_expect_success 'submodule add with /././ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotslashdotsubmod/frotz ../../.. &&\n+\tinspect addtest/dotslashdotsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -296,7 +288,7 @@ test_expect_success 'submodule add with // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/slashslashsubmod/frotz ../../.. &&\n+\tinspect addtest/slashslashsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -312,7 +304,7 @@ test_expect_success 'submodule add with /.. in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod ../.. &&\n+\tinspect addtest/realsubmod &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -328,7 +320,7 @@ test_expect_success 'submodule add with ./, /.. and // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod2 ../.. &&\n+\tinspect addtest/realsubmod2 &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -359,7 +351,7 @@ test_expect_success 'submodule add in subdirectory' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod3 ../.. &&\n+\tinspect addtest/realsubmod3 &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n-- \n2.35.0.263.gb82422642f-goog\n\n"},{"id":"447673","messageId":"20220203215914.683922-3-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220203215914.683922-1-emilyshaffer@google.com","subject":"[PATCH v7 2/4] introduce submodule.superprojectGitDir record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-02-03T21:59:12Z","receivedAt":"2022-02-03T21:59:30Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Teach submodules a config variable referencing to their superproject's\ngitdir. Git commands can rely on this reference to determine whether the\ncurrent repo is a submodule to another repo. If this reference is\nabsent, Git may assume the current repo is not a submodule.\n\nIn practice, some commands which specifically reference the submodule\nrelationship, like 'git rev-parse --show-superproject-working-tree',\nwill still search the parent directory. Other newly added or implicit\nbehavior, such as \"git status\" showing the submodule's status in\nrelation to the superproject, or a config which is shared between the\nsuperproject and the submodule, should not search the parent directory\nand instead rely on this \"submodule.superprojectGitDir\" config.\n\nBy using a relative path instead of an absolute path, we can move the\nsuperproject directory around on the filesystem without breaking the\nsubmodule's pointer. And by using the path from gitdir to gitdir, we can\nmove the submodule within the superproject's tree structure without\nbreaking the submodule's pointer, too. Finally, by pointing at the\nsuperproject's worktree gitdir (if it exists), we ensure that we can\ntell which worktree contains our submodule.\n\nThis commit teaches \"git submodule add\" to add the aformentioned config\nvariable. Subsequent commits will teach other commands to do so.\n\nSigned-off-by: Emily Shaffer \nHelped-by: Junio C Hamano \n---\n Documentation/config/submodule.txt | 8 ++++++++\n builtin/submodule--helper.c | 11 ++++++++++\n t/t7400-submodule-basic.sh | 32 ++++++++++++++++++++----------\n 3 files changed, 41 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt\nindex ee454f8126..bebd25684e 100644\n--- a/Documentation/config/submodule.txt\n+++ b/Documentation/config/submodule.txt\n@@ -91,3 +91,11 @@ submodule.alternateErrorStrategy::\n \t`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`\n \tor `info`, and if there is an error with the computed alternate, the\n \tclone proceeds as if no alternate was specified.\n+\n+submodule.superprojectGitDir::\n+\tIf this repository is a submodule, the relative path from this repo's\n+\tgitdir to its superproject's gitdir. Git commands may rely on this\n+\treference to determine whether the current repo is a submodule to\n+\tanother repo; if this reference is absent, Git will treat the current\n+\trepo as though it is not a submodule (this does not make a difference to\n+\tmost Git commands). It is set automatically during submodule creation.\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex c5d3fc3817..0485b384a1 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -1839,6 +1839,17 @@ static int clone_submodule(struct module_clone_data *clone_data)\n \t\tgit_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n \t\t\t\t error_strategy);\n \n+\t/*\n+\t * Set the path from submodule's new gitdir to superproject's gitdir.\n+\t * The latter may be a worktree gitdir. However, it is not possible for\n+\t * the submodule to have a worktree-specific gitdir or config at clone\n+\t * time, because \"extensions.worktreeConfig\" is only valid when set in\n+\t * the local gitconfig, which the brand new submodule does not have yet.\n+\t */\n+\tgit_config_set_in_file(p, \"submodule.superprojectGitDir\",\n+\t\t\t relative_path(absolute_path(get_git_dir()),\n+\t\t\t\t\t sm_gitdir, &sb));\n+\n \tfree(sm_alternate);\n \tfree(error_strategy);\n \ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex 40cf8d89aa..7e4cc89bf5 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -109,12 +109,24 @@ submodurl=$(pwd -P)\n \n inspect() {\n \tsub_dir=$1 &&\n+\tsuper_dir=$2 &&\n \n \tgit -C \"$sub_dir\" for-each-ref --format='%(refname)' 'refs/heads/*' >heads &&\n \t{ git -C \"$sub_dir\" symbolic-ref HEAD || :; } >head &&\n \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n \tgit -C \"$sub_dir\" update-index --refresh &&\n \tgit -C \"$sub_dir\" diff-files --exit-code &&\n+\n+\t# Ensure that submodule.superprojectGitDir contains the path from the\n+\t# submodule's gitdir to the superproject's gitdir.\n+\n+\tsuper_abs_gitdir=$(git -C \"$super_dir\" rev-parse --absolute-git-dir) &&\n+\tsub_abs_gitdir=$(git -C \"$sub_dir\" rev-parse --absolute-git-dir) &&\n+\n+\t[ \"$(git -C \"$sub_dir\" config --get submodule.superprojectGitDir)\" = \\\n+\t \"$(test-tool path-utils relative_path \"$super_abs_gitdir\" \\\n+\t\t\t\t\t\t\"$sub_abs_gitdir\")\" ] &&\n+\n \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n \n@@ -138,7 +150,7 @@ test_expect_success 'submodule add' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod &&\n+\tinspect addtest/submod addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -240,7 +252,7 @@ test_expect_success 'submodule add --branch' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod-branch &&\n+\tinspect addtest/submod-branch addtest &&\n \ttest_cmp expect-heads heads &&\n \ttest_cmp expect-head head &&\n \ttest_must_be_empty untracked\n@@ -256,7 +268,7 @@ test_expect_success 'submodule add with ./ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotsubmod/frotz &&\n+\tinspect addtest/dotsubmod/frotz addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -272,7 +284,7 @@ test_expect_success 'submodule add with /././ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotslashdotsubmod/frotz &&\n+\tinspect addtest/dotslashdotsubmod/frotz addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -288,7 +300,7 @@ test_expect_success 'submodule add with // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/slashslashsubmod/frotz &&\n+\tinspect addtest/slashslashsubmod/frotz addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -304,7 +316,7 @@ test_expect_success 'submodule add with /.. in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod &&\n+\tinspect addtest/realsubmod addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -320,7 +332,7 @@ test_expect_success 'submodule add with ./, /.. and // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod2 &&\n+\tinspect addtest/realsubmod2 addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -351,7 +363,7 @@ test_expect_success 'submodule add in subdirectory' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod3 &&\n+\tinspect addtest/realsubmod3 addtest &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -492,7 +504,7 @@ test_expect_success 'update should work when path is an empty dir' '\n \tgit submodule update -q >update.out &&\n \ttest_must_be_empty update.out &&\n \n-\tinspect init &&\n+\tinspect init . &&\n \ttest_cmp expect head-sha1\n '\n \n@@ -551,7 +563,7 @@ test_expect_success 'update should checkout rev1' '\n \techo \"$rev1\" >expect &&\n \n \tgit submodule update init &&\n-\tinspect init &&\n+\tinspect init . &&\n \n \ttest_cmp expect head-sha1\n '\n-- \n2.35.0.263.gb82422642f-goog\n\n"},{"id":"447674","messageId":"20220203215914.683922-4-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220203215914.683922-1-emilyshaffer@google.com","subject":"[PATCH v7 3/4] submodule: record superproject gitdir during absorbgitdirs","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-02-03T21:59:13Z","receivedAt":"2022-02-03T21:59:31Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Already during 'git submodule add' we record a pointer to the\nsuperproject's gitdir. However, this doesn't help brand-new\nsubmodules created with 'git init' and later absorbed with 'git\nsubmodule absorbgitdirs'. Let's start adding that pointer during 'git\nsubmodule absorbgitdirs' too.\n\nSigned-off-by: Emily Shaffer \n---\n Documentation/config/submodule.txt | 3 +-\n submodule.c | 23 +++++++++\n t/t7412-submodule-absorbgitdirs.sh | 82 +++++++++++++++++++++++++++++-\n 3 files changed, 105 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt\nindex bebd25684e..f801f49ea1 100644\n--- a/Documentation/config/submodule.txt\n+++ b/Documentation/config/submodule.txt\n@@ -98,4 +98,5 @@ submodule.superprojectGitDir::\n \treference to determine whether the current repo is a submodule to\n \tanother repo; if this reference is absent, Git will treat the current\n \trepo as though it is not a submodule (this does not make a difference to\n-\tmost Git commands). It is set automatically during submodule creation.\n+\tmost Git commands). It is set automatically during submodule creation\n+\tand 'git submodule absorbgitdir'.\ndiff --git a/submodule.c b/submodule.c\nindex c689070524..d7395c7551 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2097,6 +2097,9 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \tchar *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL;\n \tstruct strbuf new_gitdir = STRBUF_INIT;\n \tconst struct submodule *sub;\n+\tstruct config_set sub_cs;\n+\tstruct strbuf config_path = STRBUF_INIT, sb = STRBUF_INIT;\n+\tint tmp;\n \n \tif (submodule_uses_worktrees(path))\n \t\tdie(_(\"relocate_gitdir for submodule '%s' with \"\n@@ -2127,6 +2130,26 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \n \trelocate_gitdir(path, real_old_git_dir, real_new_git_dir);\n \n+\t/*\n+\t * Note location of superproject's gitdir. Because the submodule already\n+\t * has a gitdir and local config, we can store this pointer from\n+\t * worktree config to worktree config, if the submodule has\n+\t * extensions.worktreeConfig set.\n+\t */\n+\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n+\tgit_configset_init(&sub_cs);\n+\tgit_configset_add_file(&sub_cs, config_path.buf);\n+\t/* return 0 indicates config was found - we have a worktree config */\n+\tif (!git_configset_get_bool(&sub_cs, \"extensions.worktreeConfig\", &tmp))\n+\t\tstrbuf_addstr(&config_path, \".worktree\");\n+\n+\tgit_config_set_in_file(config_path.buf, \"submodule.superprojectGitdir\",\n+\t\t\t relative_path(absolute_path(get_git_dir()),\n+\t\t\t\t\t real_new_git_dir, &sb));\n+\n+\tgit_configset_clear(&sub_cs);\n+\tstrbuf_release(&config_path);\n+\tstrbuf_release(&sb);\n \tfree(old_git_dir);\n \tfree(real_old_git_dir);\n \tfree(real_new_git_dir);\ndiff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh\nindex 1cfa150768..5753f90268 100755\n--- a/t/t7412-submodule-absorbgitdirs.sh\n+++ b/t/t7412-submodule-absorbgitdirs.sh\n@@ -30,7 +30,17 @@ test_expect_success 'absorb the git dir' '\n \tgit status >actual.1 &&\n \tgit -C sub1 rev-parse HEAD >actual.2 &&\n \ttest_cmp expect.1 actual.1 &&\n-\ttest_cmp expect.2 actual.2\n+\ttest_cmp expect.2 actual.2 &&\n+\n+\t# make sure the submodule cached the superproject gitdir correctly\n+\tsubmodule_gitdir=\"$(git -C sub1 rev-parse --path-format=absolute --git-common-dir)\" &&\n+\tsuperproject_gitdir=\"$(git rev-parse --path-format=absolute --git-common-dir)\" &&\n+\n+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n+\t\t\"$submodule_gitdir\" >expect &&\n+\tgit -C sub1 config submodule.superprojectGitDir >actual &&\n+\n+\ttest_cmp expect actual\n '\n \n test_expect_success 'absorbing does not fail for deinitialized submodules' '\n@@ -61,7 +71,16 @@ test_expect_success 'absorb the git dir in a nested submodule' '\n \tgit status >actual.1 &&\n \tgit -C sub1/nested rev-parse HEAD >actual.2 &&\n \ttest_cmp expect.1 actual.1 &&\n-\ttest_cmp expect.2 actual.2\n+\ttest_cmp expect.2 actual.2 &&\n+\n+\tsub1_gitdir=\"$(git -C sub1 rev-parse --path-format=absolute --git-common-dir)\" &&\n+\tsub1_nested_gitdir=\"$(git -C sub1/nested rev-parse --path-format=absolute --git-common-dir)\" &&\n+\n+\ttest-tool path-utils relative_path \"$sub1_gitdir\" \"$sub1_nested_gitdir\" \\\n+\t\t>expect &&\n+\tgit -C sub1/nested config submodule.superprojectGitDir >actual &&\n+\n+\ttest_cmp expect actual\n '\n \n test_expect_success 're-setup nested submodule' '\n@@ -130,4 +149,63 @@ test_expect_success 'absorbing fails for a submodule with multiple worktrees' '\n \ttest_i18ngrep \"not supported\" error\n '\n \n+test_expect_success 'absorbgitdirs works when called from a superproject worktree' '\n+\t# set up a worktree of the superproject\n+\tgit worktree add wt &&\n+\t(\n+\tcd wt &&\n+\n+\t# create a new unembedded git dir\n+\tgit init sub4 &&\n+\ttest_commit -C sub4 first &&\n+\tgit submodule add ./sub4 &&\n+\ttest_tick &&\n+\n+\t# absorb the git dir\n+\tgit submodule absorbgitdirs sub4 &&\n+\n+\t# make sure the submodule noted the superproject gitdir correctly\n+\tsubmodule_gitdir=\"$(git -C sub4 rev-parse --absolute-git-dir)\" &&\n+\tsuperproject_gitdir=\"$(git rev-parse --absolute-git-dir)\" &&\n+\n+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n+\t\t\"$submodule_gitdir\" >expect &&\n+\tgit -C sub4 config submodule.superprojectGitDir >actual &&\n+\n+\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success 'absorbgitdirs works with a submodule with worktree config' '\n+\t# reuse the worktree of the superproject\n+\t(\n+\tcd wt &&\n+\n+\t# create a new unembedded git dir\n+\tgit init sub5 &&\n+\ttest_commit -C sub5 first &&\n+\tgit submodule add ./sub5 &&\n+\ttest_tick &&\n+\n+\t# turn on worktree configs for submodule\n+\tgit -C sub5 config extensions.worktreeConfig true &&\n+\n+\t# absorb the git dir\n+\tgit submodule absorbgitdirs sub5 &&\n+\n+\t# make sure the submodule noted the superproject gitdir correctly\n+\tsubmodule_gitdir=\"$(git -C sub5 rev-parse --absolute-git-dir)\" &&\n+\tsuperproject_gitdir=\"$(git rev-parse --absolute-git-dir)\" &&\n+\n+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n+\t\t\"$submodule_gitdir\" >expect &&\n+\tgit -C sub5 config submodule.superprojectGitDir >actual &&\n+\n+\ttest_cmp expect actual &&\n+\n+\t# make sure the config went into the submodule config.worktree\n+\ttest_file_not_empty \"$submodule_gitdir/config.worktree\"\n+\t)\n+'\n+\n test_done\n-- \n2.35.0.263.gb82422642f-goog\n\n"},{"id":"447675","messageId":"20220203215914.683922-5-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220203215914.683922-1-emilyshaffer@google.com","subject":"[PATCH v7 4/4] submodule: record superproject gitdir during 'update'","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-02-03T21:59:14Z","receivedAt":"2022-02-03T21:59:39Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"A recorded path to the superproject's gitdir might be added during\n'git submodule add', but in some cases - like submodules which were\ncreated before 'git submodule add' learned to record that info - it might\nbe useful to update the pointer. Let's do it during 'git submodule\nupdate', when we already have a handle to the superproject while calling\noperations on the submodules.\n\nSigned-off-by: Emily Shaffer \n---\n Documentation/config/submodule.txt | 4 ++--\n git-submodule.sh | 15 +++++++++++++++\n t/t7406-submodule-update.sh | 27 +++++++++++++++++++++++++++\n 3 files changed, 44 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt\nindex f801f49ea1..ab37800954 100644\n--- a/Documentation/config/submodule.txt\n+++ b/Documentation/config/submodule.txt\n@@ -98,5 +98,5 @@ submodule.superprojectGitDir::\n \treference to determine whether the current repo is a submodule to\n \tanother repo; if this reference is absent, Git will treat the current\n \trepo as though it is not a submodule (this does not make a difference to\n-\tmost Git commands). It is set automatically during submodule creation\n-\tand 'git submodule absorbgitdir'.\n+\tmost Git commands). It is set automatically during submodule creation,\n+\tupdate, and 'git submodule absorbgitdir'.\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 652861aa66..7c247bee7f 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -449,6 +449,21 @@ cmd_update()\n \t\t\t;;\n \t\tesac\n \n+\t\t# Store a poitner to the superproject's gitdir. This may have\n+\t\t# changed, unless it's a fresh clone. Write to worktree if\n+\t\t# applicable, and point to superproject's worktree gitdir if\n+\t\t# applicable.\n+\t\tif test -z \"$just_cloned\"\n+\t\tthen\n+\t\t\tsm_gitdir=\"$(git -C \"$sm_path\" rev-parse --absolute-git-dir)\"\n+\t\t\trelative_gitdir=\"$(git rev-parse --path-format=relative \\\n+\t\t\t\t\t\t\t --prefix \"${sm_gitdir}\" \\\n+\t\t\t\t\t\t\t --git-dir)\"\n+\n+\t\t\tgit -C \"$sm_path\" config --worktree \\\n+\t\t\t\tsubmodule.superprojectgitdir \"$relative_gitdir\"\n+\t\tfi\n+\n \t\tif test -n \"$recursive\"\n \t\tthen\n \t\t\t(\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex 11cccbb333..b42a339982 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -1061,4 +1061,31 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n \t)\n '\n \n+test_expect_success 'submodule update adds superproject gitdir to older repos' '\n+\t(cd super &&\n+\t git -C submodule config --unset submodule.superprojectGitdir &&\n+\t git submodule update &&\n+\t test-tool path-utils relative_path \\\n+\t\t\"$(git rev-parse --absolute-git-dir)\" \\\n+\t\t\"$(git -C submodule rev-parse --absolute-git-dir)\" >expect &&\n+\t git -C submodule config submodule.superprojectGitdir >actual &&\n+\t test_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success 'submodule update uses config.worktree if applicable' '\n+\t(cd super &&\n+\t git -C submodule config --unset submodule.superprojectGitDir &&\n+\t git -C submodule config extensions.worktreeConfig true &&\n+\t git submodule update &&\n+\t test-tool path-utils relative_path \\\n+\t\t\"$(git rev-parse --absolute-git-dir)\" \\\n+\t\t\"$(git -C submodule rev-parse --absolute-git-dir)\" >expect &&\n+\t git -C submodule config submodule.superprojectGitdir >actual &&\n+\t test_cmp expect actual &&\n+\n+\t test_file_not_empty \"$(git -C submodule rev-parse --absolute-git-dir)/config.worktree\"\n+\t)\n+'\n+\n test_done\n-- \n2.35.0.263.gb82422642f-goog\n\n"},{"id":"447679","messageId":"xmqqczk3r2wa.fsf@gitster.g","threadId":"56920","inReplyTo":"20220203215914.683922-1-emilyshaffer@google.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-02-03T22:39:17Z","receivedAt":"2022-02-03T22:39:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> A couple things. Firstly, a semantics change *back* to the semantics of\n> v3 - we map from gitdir to gitdir, *not* from common dir to common dir,\n> so that theoretically a submodule with multiple worktrees in multiple\n> superproject worktrees will be able to figure out which worktree of the\n> superproject it's in. (Realistically, that's not really possible right\n> now, but I'd like to change that soon.)\n\nSounds sensible.\n\n> Secondly, a rewording of comments and commit messages to indicate that\n> this isn't a cache of some expensive operation, but rather intended to\n> be the source of truth for all submodules.\n\nI'd expect that there is a way (e.g. \"git fsck\") that helps the\nusers notice when the actual filesystem layout contradicts with what\nthe gitdir-to-gitdir link says, and repair the repositories when\nthey go out of sync if possible.\n\nIt would be similar to \"git worktree\", where a link between the\n\".git\" file that records \"gitdir:\" in a secondary worktree and the\nrepository's $GIT_DIR/worktrees/*/gitdir, and the \"repair\" command\ncan be used to bring them back in sync after moving the real\nrepository without telling the secondary worktree about the move.\n\n> I did discuss Ævar's idea of relying on in-process filesystem digging to\n> find the superproject's gitdir with the rest of the Google team, but in\n> the end decided that there are some worries about filesystem digging in\n> this way (namely, some ugly interactions with network drives that are\n> actually already an issue for Googler Linux machines). Plus, the allure\n> of being able to definitively know that we're a submodule is pretty\n> strong.\n\nThe other side of the coin is that, even when a configuration\nvariable says that you are a submodule of the superproject at\nlocation X, if such a submodule gets moved out of the superproject\n(perhaps because the end-user wanted to concentrate on that\nsubmodule project alone as an independent project) and the\nsuperproject that used to be at location X got archived away,\ntrusting and relying on what the configuration variable says would\nnot help us access the now-gone superproject. And that would not\nchange no matter how strongly we declare that it is the source of\ntruth.\n\nUnless we have a very good way to detect inconsistency and stop\nspreading the damage (e.g. the setting thought our superproject sits\nat directory X, but that location is now occupied by a different\nrepository that is not related), I am still skeptical about the\n\"setting is the sole truth\" design.\n"},{"id":"447695","messageId":"220204.86pmo34d2m.gmgdl@evledraar.gmail.com","threadId":"56920","inReplyTo":"20220203215914.683922-1-emilyshaffer@google.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-02-04T01:15:27Z","receivedAt":"2022-02-04T01:48:07Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Feb 03 2022, Emily Shaffer wrote:\n\n> Since v6:\n\nThanks for the re-roll!\n\n> I've dropped the fifth commit to use this new config for `git rev-parse\n> --show-superproject-working-tree`. I think it did more harm than good -\n> that tool uses an odd way to determine whether the superproject is\n> actually the superproject, anyways.\n>\n> I poked a little bit at trying to find some benchmark to demonstrate\n> that \"submodule.superprojectGitDir\" is actually faster - but I didn't\n> end up able to write one without writing a ton of new code to traverse\n> the filesystem.\n\nI'm assuming that's tested against some variant of the submodule-in-C[1]\nconversion. I.e. at least when I tested it [2][3] it seemed easy to come\nup with (probably overly artificial) benchmarks where it would matter\nfor the shell-out in this series.\n\nThe question performance-wise was rather whether we'd just be\nintroducing the config mechanism as a transitory performance workaround,\nand the need for it would evaporate once that C migration happened (re\noriginal CL quoted in [3]).\n\n> To be honest, I'm not all that interested in performance\n> - I want the config added for correctness, instead.\n\nAnd I'm honestly still at the point of not even being against this whole\nthing, although it probably sounds like that. I'm really not.\n\nI just genuinely don't get where this is headed. I.e. for the last\niteration I did a demo patch on top that showed that there was no case\nadded by the series where the on-the-fly discovery wasn't equivalent to\nthe set-in-config value[4].\n\nThat change showed that after this series in a state where the config\n*is* redundant to on the fly discovery (or maybe not, and we're just\nmissing test coverage).\n\nBut since you're citing correctness do you have some repo->sub\nrelationship in mind that would be ambiguous in a way where the\nconfiguration would resolve the ambiguity?\n\nI can imagine how such a thing might work, e.g. if we gave submodules\nsome git-worktree-like method of being completely detached from the\nparent. I.e. being able to place a /usr/me/repo.git whose submodule\nentry for a \"test\" dir lives in /opt/tests or something. So when you \"cd\n/opt/tests\" you wouldn't be able to detect you're within a submodule.\n\n(I'm assuming the case where the submodule has its own \"in-tree\" .git,\nis that even supported anymore...?)\n\nBut I can't think of one where such an ambiguity would arise within our\ncurrent featureset.\n\nWhat I really can't see is how if the need for such \"config [...] for\ncorrectness\" would arise how that doesn't also invalidate the\nassumptions you're making in 3/4 and 4/4.\n\nI.e. surely if we need the config for correctness it's also true that we\ncan't after-the-fact add the config on the fly to (such) existing\nsubmodules without user intervention. Or maybe the ambiguity would only\narise from the POV of the submodule, but not for commands executed\nwithin the parent?\n\n1. https://lore.kernel.org/git/cover-v5-0.9-00000000000-20220128T125206Z-avarab@gmail.com/\n2. https://lore.kernel.org/git/RFC-cover-0.2-00000000000-20211117T113134Z-avarab@gmail.com/\n3. https://lore.kernel.org/git/211124.86a6hue2wk.gmgdl@evledraar.gmail.com/\n4. https://lore.kernel.org/git/RFC-patch-2.2-b49d4c8db7d-20211117T113134Z-avarab@gmail.com/\n"},{"id":"447747","messageId":"xmqqiltuob6r.fsf@gitster.g","threadId":"56920","inReplyTo":"220204.86pmo34d2m.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-02-04T16:20:44Z","receivedAt":"2022-02-04T16:20:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> I just genuinely don't get where this is headed. I.e. for the last\n> iteration I did a demo patch on top that showed that there was no case\n> added by the series where the on-the-fly discovery wasn't equivalent to\n> the set-in-config value[4].\n>\n> That change showed that after this series in a state where the config\n> *is* redundant to on the fly discovery (or maybe not, and we're just\n> missing test coverage).\n>\n> But since you're citing correctness do you have some repo->sub\n> relationship in mind that would be ambiguous in a way where the\n> configuration would resolve the ambiguity?\n\nThis is an excellent question, which I wish I could have raised in\nmy earlier response. A clear explanation why this setting is not\na redundant copy but adds real information on top of what we should\nbe able to learn from the filesystem structure would really help in\njustifying the new thing.\n\nThanks.\n"},{"id":"447902","messageId":"YgF5V2Y0Btr8B4cd@google.com","threadId":"56920","inReplyTo":"220204.86pmo34d2m.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2022-02-07T19:56:07Z","receivedAt":"2022-02-07T19:59:46Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nÆvar Arnfjörð Bjarmason wrote:\n> On Thu, Feb 03 2022, Emily Shaffer wrote:\n\n>> To be honest, I'm not all that interested in performance\n>> - I want the config added for correctness, instead.\n>\n> And I'm honestly still at the point of not even being against this whole\n> thing, although it probably sounds like that. I'm really not.\n>\n> I just genuinely don't get where this is headed. I.e. for the last\n> iteration I did a demo patch on top that showed that there was no case\n> added by the series where the on-the-fly discovery wasn't equivalent to\n> the set-in-config value[4].\n\nHere's a few examples:\n\n1. Suppose I track my $HOME directory as a git repository. Within my\n home directory, I have a src/git/ subdirectory with a clone of\n git.git, but I never intended to treat this as a submodule.\n\n If I run \"git rev-parse --show-superproject-working-tree\", then it\n will discover my home directory repository, run ls-files in there\n to see if it has GITLINK entries, and either see one for src/git if\n I had \"git add\"ed it by mistake or not see one. In either case,\n it would it would view my src/git/ directory as being a submodule\n of my home directory even though I hadn't intended it to be so.\n\n2. Suppose I have a copy of a repository such as\n https://gerrit.googlesource.com/gerrit/, with all its submodules.\n I am in the plugins/replication/ directory.\n\n If I run \"git rev-parse --show-superproject-working-tree\", then it\n will discover my gerrit repository, run ls-files in there to see if\n it has GITLINK entries, and use the result to decide whether the\n cwd is a submodule. So for example, if I had run \"git rm --cached\n plugins/replication\" to _prepare to_ remove the plugins/replication\n submodule, then \"git rev-parse --show-superproject-working-tree\"\n will produce the wrong result.\n\n3. Suppose I am not using submodules at all. I have a clone of\n mawk.git and I am working there.\n\n If I run \"git rev-parse --show-superproject-working-tree\", then I'm\n presumably interested in doing something submodule-specific;\n nothing wrong with that. But the series we're responding to is\n meant to support a wider variety of operations --- for example,\n suppose I am running a plain \"git status\" operation.\n\n If \"git status\" runs \"git rev-parse\n --show-superproject-working-tree\", then git would walk up the\n filesystem above my mawk/ directory, looking for another .git dir.\n We can reach an NFS automounter directory and just hang. Even\n without an NFS automounter, we'd expect this to take a while\n because, unlike normal repository discovery, we have no reason to\n believe that the walk is going to quickly discover a .git directory\n and terminate. So this would violate user expectations.\n\nThanks and hope that helps,\nJonathan\n\n> 4. https://lore.kernel.org/git/RFC-patch-2.2-b49d4c8db7d-20211117T113134Z-avarab@gmail.com/\n"},{"id":"447923","messageId":"xmqqk0e6gt5j.fsf@gitster.g","threadId":"56920","inReplyTo":"YgF5V2Y0Btr8B4cd@google.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-02-07T23:21:12Z","receivedAt":"2022-02-08T01:06:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n> Here's a few examples:\n>\n> 1. Suppose I track my $HOME directory as a git repository. Within my\n> home directory, I have a src/git/ subdirectory with a clone of\n> git.git, but I never intended to treat this as a submodule.\n>\n> If I run \"git rev-parse --show-superproject-working-tree\", then it\n> will discover my home directory repository, run ls-files in there\n> to see if it has GITLINK entries, and either see one for src/git if\n> I had \"git add\"ed it by mistake or not see one. In either case,\n> it would it would view my src/git/ directory as being a submodule\n> of my home directory even though I hadn't intended it to be so.\n\nI am not sure about this one. If you added an unrelated one with\n\"git add\" by mistake, you'd want to know about the mistake sooner\nrather than later, no?\n\n> 2. Suppose I have a copy of a repository such as\n> https://gerrit.googlesource.com/gerrit/, with all its submodules.\n> I am in the plugins/replication/ directory.\n>\n> If I run \"git rev-parse --show-superproject-working-tree\", then it\n> will discover my gerrit repository, run ls-files in there to see if\n> it has GITLINK entries, and use the result to decide whether the\n> cwd is a submodule. So for example, if I had run \"git rm --cached\n> plugins/replication\" to _prepare to_ remove the plugins/replication\n> submodule, then \"git rev-parse --show-superproject-working-tree\"\n> will produce the wrong result.\n\nYes, looking only at the index of the superproject will have that\nproblem, but don't other things in the superproject point at the\nsubmodule, too, e.g. submodule..* configuration variables?\n\nAnd then, after removing them to truly dissociate the submodule from\nthe superproject, \"git rev-parse --show-superproject-working-tree\"\nmay stop saying that it is a submodule, but this series wants to\nmake it irrelevant what the command says. Until you unset the\nconfiguration variable in the submodule, it will stay to be a\nsubmodule of the superproject, but the superproject no longer thinks\nit is responsible for the submodule. You'll have to deal with an\ninconsistent state during the transition either way, so I am not\nsure it is the best solution to introduce an extra setting that can\neasily go out of sync.\n\n> 3. Suppose I am not using submodules at all. I have a clone of\n> mawk.git and I am working there.\n>\n> If I run \"git rev-parse --show-superproject-working-tree\", then I'm\n> presumably interested in doing something submodule-specific;\n> nothing wrong with that. But the series we're responding to is\n> meant to support a wider variety of operations --- for example,\n> suppose I am running a plain \"git status\" operation.\n>\n> If \"git status\" runs \"git rev-parse\n> --show-superproject-working-tree\", then git would walk up the\n> filesystem above my mawk/ directory, looking for another .git dir.\n> We can reach an NFS automounter directory and just hang. Even\n> without an NFS automounter, we'd expect this to take a while\n> because, unlike normal repository discovery, we have no reason to\n> believe that the walk is going to quickly discover a .git directory\n> and terminate. So this would violate user expectations.\n\nIt would be a problem, but I do not know if \"this is a submodule of\nthat superproject\" link is the only solution, let alone the most\neffective one. It seems to me that you are looking more for\nsomething like GIT_CEILING_DIRECTORIES.\n"},{"id":"447933","messageId":"YgHE4iaV8QHRw64U@google.com","threadId":"56920","inReplyTo":"xmqqk0e6gt5j.fsf@gitster.g","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2022-02-08T01:18:26Z","receivedAt":"2022-02-08T01:23:11Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n> Jonathan Nieder writes:\n\n>> Here's a few examples:\n>>\n>> 1. Suppose I track my $HOME directory as a git repository. Within my\n>> home directory, I have a src/git/ subdirectory with a clone of\n>> git.git, but I never intended to treat this as a submodule.\n>>\n>> If I run \"git rev-parse --show-superproject-working-tree\", then it\n>> will discover my home directory repository, run ls-files in there\n>> to see if it has GITLINK entries, and either see one for src/git if\n>> I had \"git add\"ed it by mistake or not see one. In either case,\n>> it would it would view my src/git/ directory as being a submodule\n>> of my home directory even though I hadn't intended it to be so.\n>\n> I am not sure about this one. If you added an unrelated one with\n> \"git add\" by mistake, you'd want to know about the mistake sooner\n> rather than later, no?\n\nMy point with this example is that it's useful to have a definition of\nwhat is a submodule repository, to make it unambiguous whether this\nrepository is a submodule or whether it's just a repository that\nhappens to have been cloned inside of a git-managed worktree.\n\nFor the specific example of having run \"git add\", I don't have any\nvery strong opinions.\n\n[...]\n>> 2. Suppose I have a copy of a repository such as\n>> https://gerrit.googlesource.com/gerrit/, with all its submodules.\n>> I am in the plugins/replication/ directory.\n[...]\n>> So for example, if I had run \"git rm --cached\n>> plugins/replication\" to _prepare to_ remove the plugins/replication\n>> submodule, then \"git rev-parse --show-superproject-working-tree\"\n>> will produce the wrong result.\n>\n> Yes, looking only at the index of the superproject will have that\n> problem, but don't other things in the superproject point at the\n> submodule, too, e.g. submodule..* configuration variables?\n\nWhat all of those suggested alternatives have in common is that they\nare pointers from another repository to the submodule.\n\nThis would be the first time in git history that we are saying a\nproperty of a repository depends on having to examine files outside of\nit. I guess the main question I'd have is, why _wouldn't_ I want a\nsubmodule to be able to point to the superproject containing it? I\ncan think of many advantages to having that linkage, and the main\ndisadvantage I can think of is that it is a change.\n\nI don't think that submodule..* is an adequate substitute for\nhaving this setting, because it requires\n- finding the superproject\n- mapping the to a path, using .gitmodules\n- comparing the path to the submodule location\n\nwhich would be complex, slow, and error-prone.\n\nThe one thing that I think could approach being an adequate substitute\nis examining the path to the current repository and stripping off path\ncomponents until we find modules/; then the parent is the containing\nsuperproject. That would only work for absorbed submodules, though,\nand it would be less explicit than having a config item.\n\n> And then, after removing them to truly dissociate the submodule from\n> the superproject, \"git rev-parse --show-superproject-working-tree\"\n> may stop saying that it is a submodule, but this series wants to\n> make it irrelevant what the command says. Until you unset the\n> configuration variable in the submodule, it will stay to be a\n> submodule of the superproject, but the superproject no longer thinks\n> it is responsible for the submodule. You'll have to deal with an\n> inconsistent state during the transition either way, so I am not\n> sure it is the best solution to introduce an extra setting that can\n> easily go out of sync.\n\nThis hints at a reason why one wouldn't want the linkage back ---\ndealing with the ambiguity of inconsistencies (what if a submodule\ndeclares a superproject but the superproject does not declare the\nsubmodule?).\n\nI would not expect that ambiguity to be much of a problem,\nbecause the typical way to use superproject linkage would be to\nprint output from commands like \"git status\": for example,\n\n\tThis is a submodule of ../../gerrit; you can run\n\n\t\tgit -C ../../gerrit status\n\n\tto get the status of the superproject.\n\nAn inconsistency could occur due to the user using \"mv\" (instead of\n\"git mv\") to move a submodule to a path a different number of path\ncomponents from its superproject. One way to handle that would be to\nmake submodules record a boolean setting reflecting whether they are a\nsubmodule, instead of the path to the superproject. (This would be\nsimilar to settings like core.bare.) Alternatively, if the path to\nthe superproject is recorded and if \"git fsck\" is able to notice such\nan inconsistency, then the user should be able to have an okay\nexperience repairing it.\n\n[...]\n>> If \"git status\" runs \"git rev-parse\n>> --show-superproject-working-tree\", then git would walk up the\n>> filesystem above my mawk/ directory, looking for another .git dir.\n>> We can reach an NFS automounter directory and just hang. Even\n>> without an NFS automounter, we'd expect this to take a while\n>> because, unlike normal repository discovery, we have no reason to\n>> believe that the walk is going to quickly discover a .git directory\n>> and terminate. So this would violate user expectations.\n>\n> It would be a problem, but I do not know if \"this is a submodule of\n> that superproject\" link is the only solution, let alone the most\n> effective one. It seems to me that you are looking more for\n> something like GIT_CEILING_DIRECTORIES.\n\nWho is the \"you\" addressed here? The end user can use\nGIT_CEILING_DIRECTORIES if they are expecting to run git commands\nwithin an NFS automounter directory and outside of any git repository,\nbut they'd be right to be surprised if that suddenly became required\nwhen inside git repositories. I don't think we should assume that\nrunning an extra .git discovery walk is cost-free to users who are not\nusing submodules and an acceptable burden to impose on them for the\nsake of submodule users.\n\nThanks and hope that helps,\nJonathan\n"},{"id":"447995","messageId":"xmqqy22lcj2m.fsf@gitster.g","threadId":"56920","inReplyTo":"YgHE4iaV8QHRw64U@google.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-02-08T18:24:49Z","receivedAt":"2022-02-08T18:24:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n> My point with this example is that it's useful to have a definition of\n> what is a submodule repository, to make it unambiguous whether this\n> repository is a submodule or whether it's just a repository that\n> happens to have been cloned inside of a git-managed worktree.\n\nOK, together with the other \"no need to let NFS automounter worry\nabout parent directories\", it makes a very successful argument for a\nsingle bit (i.e. this is a free-standing repository and is not a\nsubmodule, so no need to auto-discover if it is one). I think the\n\"Alternatively\" you later mention to solve ambiguity with just a\nsingle bit, without \"this is a submodule of that superproject\"\nlinkage, is essentially the same?\n\nBut I do not think it argues for the need to say \"a config, not\nfilesystem layout, must be the single source of truth to say which\nsuperproject this repository belongs as its submodule\".\n\n> This would be the first time in git history that we are saying a\n> property of a repository depends on having to examine files outside of\n> it.\n\nWell, path-based configuration inclusion, with configuration driven\nhooks, I do not think the distinction matters much anymore in these\ndays.\n\n> I guess the main question I'd have is, why _wouldn't_ I want a\n> submodule to be able to point to the superproject containing it?\n\nBecause with (the absense of) a single \"this is freestanding\" bit, \nby default the filesystem layout can already \"point\" at it?\n"},{"id":"448192","messageId":"YgWNwIE4ZLSWAr6n@google.com","threadId":"56920","inReplyTo":"xmqqy22lcj2m.fsf@gitster.g","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-02-10T22:12:16Z","receivedAt":"2022-02-10T22:12:25Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Tue, Feb 08, 2022 at 10:24:49AM -0800, Junio C Hamano wrote:\n> \n> Jonathan Nieder writes:\n> \n> > My point with this example is that it's useful to have a definition of\n> > what is a submodule repository, to make it unambiguous whether this\n> > repository is a submodule or whether it's just a repository that\n> > happens to have been cloned inside of a git-managed worktree.\n> \n> OK, together with the other \"no need to let NFS automounter worry\n> about parent directories\", it makes a very successful argument for a\n> single bit (i.e. this is a free-standing repository and is not a\n> submodule, so no need to auto-discover if it is one). I think the\n> \"Alternatively\" you later mention to solve ambiguity with just a\n> single bit, without \"this is a submodule of that superproject\"\n> linkage, is essentially the same?\n\nThat resolution - \"teach submodules to know they're submodules, but not\nwhose submodule they are\" - would still count as a success to me. The\nreason I proposed a path instead of a boolean here was simply because\nstoring a path is a boolean (by whether it's present or not) *and*\nadditional information (the path to the superproject), and it seemed\nsilly to me to opt for less information. Or, to put it another way - \"am\nI a submodule?\" seems pretty vital, and \"yes, and I belong to xyz\" is an\noptimization on top of that. So I don't terribly mind sending this as\njust a boolean, if we feel that the effort to keep it up outweighs the\nbenefit of saving us a filesystem walk.\n\nI'm not completely convinced that it does, though - would the addition\nof a 'git fsck' check for this config be satisfactory? In other words,\nis the problem that the execution of this series wasn't thorough enough\nand it should be refined, or that the concept itself is beyond saving?\n\n - Emily\n\n> \n> But I do not think it argues for the need to say \"a config, not\n> filesystem layout, must be the single source of truth to say which\n> superproject this repository belongs as its submodule\".\n> \n> > This would be the first time in git history that we are saying a\n> > property of a repository depends on having to examine files outside of\n> > it.\n> \n> Well, path-based configuration inclusion, with configuration driven\n> hooks, I do not think the distinction matters much anymore in these\n> days.\n> \n> > I guess the main question I'd have is, why _wouldn't_ I want a\n> > submodule to be able to point to the superproject containing it?\n> \n> Because with (the absense of) a single \"this is freestanding\" bit, \n> by default the filesystem layout can already \"point\" at it?\n"},{"id":"448198","messageId":"YgWXVybMBIyKSvN9@google.com","threadId":"56920","inReplyTo":"YgWNwIE4ZLSWAr6n@google.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2022-02-10T22:53:11Z","receivedAt":"2022-02-10T22:53:17Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Emily Shaffer wrote:\n> On Tue, Feb 08, 2022 at 10:24:49AM -0800, Junio C Hamano wrote:\n>> Jonathan Nieder writes:\n\n>>> My point with this example is that it's useful to have a definition of\n>>> what is a submodule repository, to make it unambiguous whether this\n>>> repository is a submodule or whether it's just a repository that\n>>> happens to have been cloned inside of a git-managed worktree.\n>>\n>> OK, together with the other \"no need to let NFS automounter worry\n>> about parent directories\", it makes a very successful argument for a\n>> single bit (i.e. this is a free-standing repository and is not a\n>> submodule, so no need to auto-discover if it is one). I think the\n>> \"Alternatively\" you later mention to solve ambiguity with just a\n>> single bit, without \"this is a submodule of that superproject\"\n>> linkage, is essentially the same?\n>\n> That resolution - \"teach submodules to know they're submodules, but not\n> whose submodule they are\" - would still count as a success to me.\n\nThanks, both. Sounds like a good path forward.\n\n[...]\n> So I don't terribly mind sending this as\n> just a boolean, if we feel that the effort to keep it up outweighs the\n> benefit of saving us a filesystem walk.\n>\n> I'm not completely convinced that it does, though\n\nPersonally, I'm convinced --- e.g., life gets painful enough when\ncore.worktree ends up pointing to the wrong path, so being able to\navoid that complexity seems like a nice outcome.\n\n> - would the addition\n> of a 'git fsck' check for this config be satisfactory? In other words,\n> is the problem that the execution of this series wasn't thorough enough\n> and it should be refined, or that the concept itself is beyond saving?\n\nFor the absorbed case, the path to the superproject should be pretty\nstable, and in that case it's probably possible to make this robust\nenough. (That said, the path to the superproject gitdir would\ntypically just be \"../..\", at least as long as we have the other patch\nto escape slashes in the submodule name.) In the non-absorbed case,\nit seems likely to get messy fast because a user can \"mv\" the\nsubmodule around.\n\nAnother kind of case that gets interesting is when there are multiple\nsuperproject worktrees and multiple submodule worktrees. Does the\nrelative path become a per-worktree variable? Using a boolean saves\nus from having to think through it.\n\nThanks for the clear explanations,\nJonathan\n"},{"id":"448320","messageId":"220212.864k53yfws.gmgdl@evledraar.gmail.com","threadId":"56920","inReplyTo":"YgF5V2Y0Btr8B4cd@google.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-02-12T20:35:48Z","receivedAt":"2022-02-12T20:43:37Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Feb 07 2022, Jonathan Nieder wrote:\n\n> Hi,\n>\n> Ævar Arnfjörð Bjarmason wrote:\n>> On Thu, Feb 03 2022, Emily Shaffer wrote:\n>\n>>> To be honest, I'm not all that interested in performance\n>>> - I want the config added for correctness, instead.\n>>\n>> And I'm honestly still at the point of not even being against this whole\n>> thing, although it probably sounds like that. I'm really not.\n>>\n>> I just genuinely don't get where this is headed. I.e. for the last\n>> iteration I did a demo patch on top that showed that there was no case\n>> added by the series where the on-the-fly discovery wasn't equivalent to\n>> the set-in-config value[4].\n>\n> Here's a few examples:\n\nI've read the downthread, but it's probably best to reply to this...\n\n> 1. Suppose I track my $HOME directory as a git repository. Within my\n> home directory, I have a src/git/ subdirectory with a clone of\n> git.git, but I never intended to treat this as a submodule.\n>\n> If I run \"git rev-parse --show-superproject-working-tree\", then it\n> will discover my home directory repository, run ls-files in there\n> to see if it has GITLINK entries, and either see one for src/git if\n> I had \"git add\"ed it by mistake or not see one. In either case,\n> it would it would view my src/git/ directory as being a submodule\n> of my home directory even though I hadn't intended it to be so.\n>\n> 2. Suppose I have a copy of a repository such as\n> https://gerrit.googlesource.com/gerrit/, with all its submodules.\n> I am in the plugins/replication/ directory.\n>\n> If I run \"git rev-parse --show-superproject-working-tree\", then it\n> will discover my gerrit repository, run ls-files in there to see if\n> it has GITLINK entries, and use the result to decide whether the\n> cwd is a submodule. So for example, if I had run \"git rm --cached\n> plugins/replication\" to _prepare to_ remove the plugins/replication\n> submodule, then \"git rev-parse --show-superproject-working-tree\"\n> will produce the wrong result.\n\nThese both seem like valid edge cases, but they're still going to be the\nsame edge case on the \"parent\" side even with a proposed cache (whether\nit's a boolean or a path).\n\nI.e. the question here is really not one of caching, but of what it\nmeans for Y to be a submodule of X.\n\nI assumed that we'd prefer a 1=1 relationship between the parent\nreporting that Y is a submodule of it, and Y reporting that it is a\nsubmodule (of the parent at some ).\n\nIf that's the case we can walk up and ask parent .git's whether they\nthink the is their submodule.\n\nIf it's not the case perhaps a config is needed, but then that surely\nhas wider implications. I.e. won't it be the case that we can't add the\nconfig after-the-fact as this series proposes in those some ambiguous\ncases?xo\n\n> 3. Suppose I am not using submodules at all. I have a clone of\n> mawk.git and I am working there.\n>\n> If I run \"git rev-parse --show-superproject-working-tree\", then I'm\n> presumably interested in doing something submodule-specific;\n> nothing wrong with that. But the series we're responding to is\n> meant to support a wider variety of operations --- for example,\n> suppose I am running a plain \"git status\" operation.\n>\n> If \"git status\" runs \"git rev-parse\n> --show-superproject-working-tree\", then git would walk up the\n> filesystem above my mawk/ directory, looking for another .git dir.\n> We can reach an NFS automounter directory and just hang. Even\n> without an NFS automounter, we'd expect this to take a while\n> because, unlike normal repository discovery, we have no reason to\n> believe that the walk is going to quickly discover a .git directory\n> and terminate. So this would violate user expectations.\n\nWe have a /a/b/c/d.git mounted, but not a parent /a/b/, and walking\nupwards causes it to be mounted?\n"},{"id":"448337","messageId":"xmqqbkzb705s.fsf@gitster.g","threadId":"56920","inReplyTo":"220212.864k53yfws.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v6 0/5] teach submodules to know they're submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-02-13T06:25:51Z","receivedAt":"2022-02-13T06:25:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> We have a /a/b/c/d.git mounted, but not a parent /a/b/, and walking\n> upwards causes it to be mounted?\n\n;-)\n"},{"id":"449836","messageId":"20220301002613.1459916-1-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220203215914.683922-1-emilyshaffer@google.com","subject":"[PATCH v8 0/3] teach submodules to know they're submodules","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-01T00:26:10Z","receivedAt":"2022-03-01T00:26:23Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"For the original cover letter, see\nhttps://lore.kernel.org/git/20210611225428.1208973-1-emilyshaffer%40google.com.\n\nCI run: https://github.com/nasamuffin/git/actions/runs/1866957146\n\nSince v7:\n\nActually a fairly large rework. Rather than keeping the path from gitdir\nto gitdir, just keep a boolean under 'submodule.hasSuperproject'. The\nidea is that from this boolean, we can decide whether to traverse the\nfilesystem looking for a superproject.\n\nBecause this simplifies the implementation, I compressed the three\nmiddle commits into one. As proof-of-concept, I added a patch at the end\nto check for this boolean when running `git rev-parse\n--show-superproject-working-tree`.\n\nOne thing I'm not sure about: in the tests, I check whether the config\nis set, but not what the boolean value of it is. Is there a better way\nto do that? For example, I could imagine someone deciding to set\n`submodule.hasSuperproject = false` and the tests would not function\ncorrectly in that case. I think we don't really normalize the value on a\nboolean config like that, so I didn't want to write a lot of comparison\nto check if the value is 1 or true or True or TRUE or Yes or .... Am I\noverthinking it?\n\nThe other thing I'm not sure about: since it's just a bool, we're not\nrestricted to setting this config only when we have both gitdir paths\navailable. That makes me want to set the config any time we are doing\nsomething with submodules anyway, like any time 'git-submodule--helper'\nis used. But that helper seems to be called in the context of the\nsuperproject, not of the submodules, so adding this config for each\nsubmodule we touch would be a second child process. Is there some other\ncommon entry point for submodules that we can use?\n\n - Emily\n\nSince v6:\n\nI've dropped the fifth commit to use this new config for `git rev-parse\n--show-superproject-working-tree`. I think it did more harm than good -\nthat tool uses an odd way to determine whether the superproject is\nactually the superproject, anyways.\n\nI poked a little bit at trying to find some benchmark to demonstrate\nthat \"submodule.superprojectGitDir\" is actually faster - but I didn't\nend up able to write one without writing a ton of new code to traverse\nthe filesystem. To be honest, I'm not all that interested in performance\n- I want the config added for correctness, instead.\n\nSo, the only real changes between v6 and v7 are some documentation\nchanges suggested by Jonathan Tan\n(https://lore.kernel.org/git/20211117234300.2598132-1-jonathantanmy%40google.com).\n\nSince v5:\n\nA couple things. Firstly, a semantics change *back* to the semantics of\nv3 - we map from gitdir to gitdir, *not* from common dir to common dir,\nso that theoretically a submodule with multiple worktrees in multiple\nsuperproject worktrees will be able to figure out which worktree of the\nsuperproject it's in. (Realistically, that's not really possible right\nnow, but I'd like to change that soon.)\n\nSecondly, a rewording of comments and commit messages to indicate that\nthis isn't a cache of some expensive operation, but rather intended to\nbe the source of truth for all submodules. I also added a fifth commit\nrewriting `git rev-parse --show-superproject-working-tree` to\ndemonstrate what that means in practice - but from a practical\nstandpoint, I'm a little worried about that fifth patch. More details in\nthe patch 5 description.\n\nI did discuss Ævar's idea of relying on in-process filesystem digging to\nfind the superproject's gitdir with the rest of the Google team, but in\nthe end decided that there are some worries about filesystem digging in\nthis way (namely, some ugly interactions with network drives that are\nactually already an issue for Googler Linux machines). Plus, the allure\nof being able to definitively know that we're a submodule is pretty\nstrong. ;) But overall, this is the direction I'd prefer to keep going\nin, rather than trying to guess from the filesystem going forward.\n\nSince v4:\n\nThe only real change here is a slight semantics change to map from\n to . In every case\n*except* for when the superproject has a worktree, this changes nothing.\nFor the case when the superproject has a worktree, this means that now\nsubmodules will refer to the general superproject common dir (e.g. no\nworktree-specific refs or configs or whatnot).\n\nI *think* that because a submodule should exist in the context of the\ncommon dir, not the worktree gitdir, that is ok. However, it does mean\nit would be difficult to do something like sharing a config specific to\nthe worktree (the initial goal of this series).\n\n$ROOT/.git\n$ROOT/.git/config.superproject <- shared by $ROOT/.git/modules/sub\n$ROOT/.git/modules/sub <- points to $ROOT/.git\n$ROOT/.git/worktrees/wt\n$ROOT/.git/worktrees/wt/config.superproject <- contains a certain config-based pre-commit hook\n\nIf the submodule only knows about the common dir, that is tough, because\nthe submodule would basically have to guess which worktree it's in from\nits own path. There would be no way for '$WT/sub' to inherit\n'$ROOT/.git/worktrees/wt/config.superproject'.\n\nThat said... right now, we don't support submodules in worktrees very\nwell at all. A submodule in a worktree will get a brand new gitdir in\n$ROOT/.git/worktrees/modules/ (and that brand new gitdir would point to\nthe super's common dir). So I think we can punt on this entire question\nuntil we teach submodules and worktrees to play more gracefully together\n(it's on my long list...), and at that time we can probably introduce a\npointer from $ROOT/.git/modules/sub/worktrees/wt/ to\n$ROOT/.git/worktrees/wt/....\n\nOr, to summarize the long ramble above: \"this is still kind of weird\nwith worktrees, but let's fix it later when we fix worktrees more\nthoroughly\".\n\n(More rambling about worktree weirdness here:\nhttps://lore.kernel.org/git/YYRaII8YWVxlBqsF%40google.com )\n\n\nSince v3, a pretty major change: the semantics of\nsubmodule.superprojectGitDir has changed, to point from the submodule's\ngitdir to the superproject's gitdir (in v3 and earlier, we kept a path\nfrom the submodule's *worktree* to the superproject's gitdir instead).\nThis cleans up some of the confusions about the behavior when a\nsubmodule worktree moves around in the superproject's tree, or in a\nfuture when we support submodules having multiple worktrees.\n\nI also tried to simplify the tests to use 'test-tool path-utils\nrelative_path' everywhere - I think that makes them much more clear for\na test reader, but if you're reviewing and it isn't obvious what we're\ntesting for, please speak up.\n\nI think this is pretty mature and there was a lot of general agreement\nthat the gitdir->gitdir association was the way to go, so please be\nbrutal and look for nits, leaks, etc. this round ;)\n[/v4 cover letter]\n\nEmily Shaffer (3):\n t7400-submodule-basic: modernize inspect() helper\n introduce submodule.hasSuperproject record\n rev-parse: short-circuit superproject worktree when config unset\n\n Documentation/config/submodule.txt | 6 ++++\n builtin/submodule--helper.c | 5 +++\n git-submodule.sh | 3 ++\n submodule.c | 30 ++++++++++++++++++\n t/t7400-submodule-basic.sh | 42 ++++++++++++-------------\n t/t7406-submodule-update.sh | 8 +++++\n t/t7412-submodule-absorbgitdirs.sh | 50 ++++++++++++++++++++++++++++--\n 7 files changed, 119 insertions(+), 25 deletions(-)\n\nRange-diff against v7:\n1: 1a85deb1c5 < -: ---------- introduce submodule.superprojectGitDir record\n-: ---------- > 1: 251510c687 t7400-submodule-basic: modernize inspect() helper\n2: 7a44b0edf9 ! 2: 34cbfd81ee submodule: record superproject gitdir during absorbgitdirs\n @@ Metadata\n Author: Emily Shaffer \n \n ## Commit message ##\n - submodule: record superproject gitdir during absorbgitdirs\n + introduce submodule.hasSuperproject record\n \n - Already during 'git submodule add' we record a pointer to the\n - superproject's gitdir. However, this doesn't help brand-new\n - submodules created with 'git init' and later absorbed with 'git\n - submodule absorbgitdirs'. Let's start adding that pointer during 'git\n - submodule absorbgitdirs' too.\n + Teach submodules a config variable indicating the fact that they are a\n + submodule. If this config is set to false or unset, Git may assume the\n + current repo is not a submodule.\n +\n + Git commands can use this variable to decide whether to traverse the\n + filesystem and look for a superproject at all. 'git rev-parse\n + --show-superproject-working-tree' can learn to exit early if this config\n + is unset or false. Other newly added or implicit behavior - like \"git\n + status\" showing the submodule's status in relation to the superproject,\n + or a config shared between the superproject and submodule - can use this\n + config to decide whether to search the parent directory to find a\n + superproject.\n +\n + Introduce this config everywhere we add a new submodule, or touch one\n + that already exists, so that we can proliferate it in repos which are\n + already out in the world using submodules.\n \n Signed-off-by: Emily Shaffer \n + Helped-by: Junio C Hamano \n \n ## Documentation/config/submodule.txt ##\n -@@ Documentation/config/submodule.txt: submodule.superprojectGitDir::\n - \treference to determine whether the current repo is a submodule to\n - \tanother repo; if this reference is absent, Git will treat the current\n - \trepo as though it is not a submodule (this does not make a difference to\n --\tmost Git commands). It is set automatically during submodule creation.\n -+\tmost Git commands). It is set automatically during submodule creation\n -+\tand 'git submodule absorbgitdir'.\n +@@ Documentation/config/submodule.txt: submodule.alternateErrorStrategy::\n + \t`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`\n + \tor `info`, and if there is an error with the computed alternate, the\n + \tclone proceeds as if no alternate was specified.\n ++\n ++submodule.hasSuperproject::\n ++\tIndicates whether this repository is a submodule. If this config is set\n ++\tto 'true', Git may traverse the filesystem above this submodule in order\n ++\tto identify the superproject. It is set automatically during submodule\n ++\tcreation, update, and 'git submodule absorbgitdir'.\n +\n + ## builtin/submodule--helper.c ##\n +@@ builtin/submodule--helper.c: static int clone_submodule(struct module_clone_data *clone_data)\n + \t\tgit_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n + \t\t\t\t error_strategy);\n + \n ++\t/*\n ++\t * Teach the submodule that it's a submodule.\n ++\t */\n ++\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n ++\n + \tfree(sm_alternate);\n + \tfree(error_strategy);\n + \n +\n + ## git-submodule.sh ##\n +@@ git-submodule.sh: cmd_update()\n + \t\t\t;;\n + \t\tesac\n + \n ++\t\t# Note that the submodule is a submodule.\n ++\t\tgit -C \"$sm_path\" config submodule.hasSuperproject \"true\"\n ++\n + \t\tif test -n \"$recursive\"\n + \t\tthen\n + \t\t\t(\n \n ## submodule.c ##\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *path)\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *p\n \tconst struct submodule *sub;\n +\tstruct config_set sub_cs;\n +\tstruct strbuf config_path = STRBUF_INIT, sb = STRBUF_INIT;\n -+\tint tmp;\n \n \tif (submodule_uses_worktrees(path))\n \t\tdie(_(\"relocate_gitdir for submodule '%s' with \"\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *p\n +\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n +\tgit_configset_init(&sub_cs);\n +\tgit_configset_add_file(&sub_cs, config_path.buf);\n -+\t/* return 0 indicates config was found - we have a worktree config */\n -+\tif (!git_configset_get_bool(&sub_cs, \"extensions.worktreeConfig\", &tmp))\n -+\t\tstrbuf_addstr(&config_path, \".worktree\");\n +\n -+\tgit_config_set_in_file(config_path.buf, \"submodule.superprojectGitdir\",\n -+\t\t\t relative_path(absolute_path(get_git_dir()),\n -+\t\t\t\t\t real_new_git_dir, &sb));\n ++\tgit_config_set_in_file(config_path.buf, \"submodule.hasSuperproject\",\n ++\t\t\t \"true\");\n +\n +\tgit_configset_clear(&sub_cs);\n +\tstrbuf_release(&config_path);\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *p\n \tfree(real_old_git_dir);\n \tfree(real_new_git_dir);\n \n + ## t/t7400-submodule-basic.sh ##\n +@@ t/t7400-submodule-basic.sh: inspect() {\n + \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n + \tgit -C \"$sub_dir\" update-index --refresh &&\n + \tgit -C \"$sub_dir\" diff-files --exit-code &&\n ++\n ++\t# Ensure that submodule.hasSuperproject is set.\n ++\tgit -C \"$sub_dir\" config \"submodule.hasSuperproject\"\n ++\n + \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n + }\n + \n +\n + ## t/t7406-submodule-update.sh ##\n +@@ t/t7406-submodule-update.sh: test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n + \t)\n + '\n + \n ++test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n ++\t(cd super &&\n ++\t git -C submodule config --unset submodule.hasSuperproject &&\n ++\t git submodule update &&\n ++\t git -C submodule config submodule.hasSuperproject\n ++\t)\n ++'\n ++\n + test_done\n +\n ## t/t7412-submodule-absorbgitdirs.sh ##\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorb the git dir' '\n \tgit status >actual.1 &&\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorb the git dir' '\n -\ttest_cmp expect.2 actual.2\n +\ttest_cmp expect.2 actual.2 &&\n +\n -+\t# make sure the submodule cached the superproject gitdir correctly\n -+\tsubmodule_gitdir=\"$(git -C sub1 rev-parse --path-format=absolute --git-common-dir)\" &&\n -+\tsuperproject_gitdir=\"$(git rev-parse --path-format=absolute --git-common-dir)\" &&\n -+\n -+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n -+\t\t\"$submodule_gitdir\" >expect &&\n -+\tgit -C sub1 config submodule.superprojectGitDir >actual &&\n -+\n -+\ttest_cmp expect actual\n ++\tgit -C sub1 config submodule.hasSuperproject\n '\n \n test_expect_success 'absorbing does not fail for deinitialized submodules' '\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorb the git dir in a\n -\ttest_cmp expect.2 actual.2\n +\ttest_cmp expect.2 actual.2 &&\n +\n -+\tsub1_gitdir=\"$(git -C sub1 rev-parse --path-format=absolute --git-common-dir)\" &&\n -+\tsub1_nested_gitdir=\"$(git -C sub1/nested rev-parse --path-format=absolute --git-common-dir)\" &&\n -+\n -+\ttest-tool path-utils relative_path \"$sub1_gitdir\" \"$sub1_nested_gitdir\" \\\n -+\t\t>expect &&\n -+\tgit -C sub1/nested config submodule.superprojectGitDir >actual &&\n -+\n -+\ttest_cmp expect actual\n ++\tgit -C sub1/nested config submodule.hasSuperproject\n '\n \n test_expect_success 're-setup nested submodule' '\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorbing fails for a s\n +\t# absorb the git dir\n +\tgit submodule absorbgitdirs sub4 &&\n +\n -+\t# make sure the submodule noted the superproject gitdir correctly\n -+\tsubmodule_gitdir=\"$(git -C sub4 rev-parse --absolute-git-dir)\" &&\n -+\tsuperproject_gitdir=\"$(git rev-parse --absolute-git-dir)\" &&\n -+\n -+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n -+\t\t\"$submodule_gitdir\" >expect &&\n -+\tgit -C sub4 config submodule.superprojectGitDir >actual &&\n -+\n -+\ttest_cmp expect actual\n ++\t# make sure the submodule noted the superproject\n ++\tgit -C sub4 config submodule.hasSuperproject\n +\t)\n +'\n +\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorbing fails for a s\n +\t# absorb the git dir\n +\tgit submodule absorbgitdirs sub5 &&\n +\n -+\t# make sure the submodule noted the superproject gitdir correctly\n -+\tsubmodule_gitdir=\"$(git -C sub5 rev-parse --absolute-git-dir)\" &&\n -+\tsuperproject_gitdir=\"$(git rev-parse --absolute-git-dir)\" &&\n -+\n -+\ttest-tool path-utils relative_path \"$superproject_gitdir\" \\\n -+\t\t\"$submodule_gitdir\" >expect &&\n -+\tgit -C sub5 config submodule.superprojectGitDir >actual &&\n -+\n -+\ttest_cmp expect actual &&\n -+\n -+\t# make sure the config went into the submodule config.worktree\n -+\ttest_file_not_empty \"$submodule_gitdir/config.worktree\"\n ++\t# make sure the submodule noted the superproject\n ++\tgit -C sub5 config submodule.hasSuperproject\n +\t)\n +'\n +\n3: 63e736e69d < -: ---------- submodule: record superproject gitdir during 'update'\n-: ---------- > 3: c14ee8760f rev-parse: short-circuit superproject worktree when config unset\n-- \n2.35.1.574.g5d30c73bfb-goog\n\n"},{"id":"449837","messageId":"20220301002613.1459916-2-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220301002613.1459916-1-emilyshaffer@google.com","subject":"[PATCH v8 1/3] t7400-submodule-basic: modernize inspect() helper","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-01T00:26:11Z","receivedAt":"2022-03-01T00:26:36Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Since the inspect() helper in the submodule-basic test suite was\nwritten, 'git -C ' was added. By using -C, we no longer need a\nreference to the base directory for the test. This simplifies callsites,\nand will make the addition of other arguments in later patches more\nreadable.\n\nSigned-off-by: Emily Shaffer \n---\n t/t7400-submodule-basic.sh | 40 +++++++++++++++-----------------------\n 1 file changed, 16 insertions(+), 24 deletions(-)\n\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex e7cec2e457..40cf8d89aa 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -107,23 +107,15 @@ test_expect_success 'setup - repository to add submodules to' '\n # generates, which will expand symbolic links.\n submodurl=$(pwd -P)\n \n-listbranches() {\n-\tgit for-each-ref --format='%(refname)' 'refs/heads/*'\n-}\n-\n inspect() {\n-\tdir=$1 &&\n-\tdotdot=\"${2:-..}\" &&\n-\n-\t(\n-\t\tcd \"$dir\" &&\n-\t\tlistbranches >\"$dotdot/heads\" &&\n-\t\t{ git symbolic-ref HEAD || :; } >\"$dotdot/head\" &&\n-\t\tgit rev-parse HEAD >\"$dotdot/head-sha1\" &&\n-\t\tgit update-index --refresh &&\n-\t\tgit diff-files --exit-code &&\n-\t\tgit clean -n -d -x >\"$dotdot/untracked\"\n-\t)\n+\tsub_dir=$1 &&\n+\n+\tgit -C \"$sub_dir\" for-each-ref --format='%(refname)' 'refs/heads/*' >heads &&\n+\t{ git -C \"$sub_dir\" symbolic-ref HEAD || :; } >head &&\n+\tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n+\tgit -C \"$sub_dir\" update-index --refresh &&\n+\tgit -C \"$sub_dir\" diff-files --exit-code &&\n+\tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n \n test_expect_success 'submodule add' '\n@@ -146,7 +138,7 @@ test_expect_success 'submodule add' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod ../.. &&\n+\tinspect addtest/submod &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -248,7 +240,7 @@ test_expect_success 'submodule add --branch' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod-branch ../.. &&\n+\tinspect addtest/submod-branch &&\n \ttest_cmp expect-heads heads &&\n \ttest_cmp expect-head head &&\n \ttest_must_be_empty untracked\n@@ -264,7 +256,7 @@ test_expect_success 'submodule add with ./ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotsubmod/frotz ../../.. &&\n+\tinspect addtest/dotsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -280,7 +272,7 @@ test_expect_success 'submodule add with /././ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotslashdotsubmod/frotz ../../.. &&\n+\tinspect addtest/dotslashdotsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -296,7 +288,7 @@ test_expect_success 'submodule add with // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/slashslashsubmod/frotz ../../.. &&\n+\tinspect addtest/slashslashsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -312,7 +304,7 @@ test_expect_success 'submodule add with /.. in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod ../.. &&\n+\tinspect addtest/realsubmod &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -328,7 +320,7 @@ test_expect_success 'submodule add with ./, /.. and // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod2 ../.. &&\n+\tinspect addtest/realsubmod2 &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -359,7 +351,7 @@ test_expect_success 'submodule add in subdirectory' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod3 ../.. &&\n+\tinspect addtest/realsubmod3 &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n-- \n2.35.1.574.g5d30c73bfb-goog\n\n"},{"id":"449838","messageId":"20220301002613.1459916-3-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220301002613.1459916-1-emilyshaffer@google.com","subject":"[PATCH v8 2/3] introduce submodule.hasSuperproject record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-01T00:26:12Z","receivedAt":"2022-03-01T00:26:39Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Teach submodules a config variable indicating the fact that they are a\nsubmodule. If this config is set to false or unset, Git may assume the\ncurrent repo is not a submodule.\n\nGit commands can use this variable to decide whether to traverse the\nfilesystem and look for a superproject at all. 'git rev-parse\n--show-superproject-working-tree' can learn to exit early if this config\nis unset or false. Other newly added or implicit behavior - like \"git\nstatus\" showing the submodule's status in relation to the superproject,\nor a config shared between the superproject and submodule - can use this\nconfig to decide whether to search the parent directory to find a\nsuperproject.\n\nIntroduce this config everywhere we add a new submodule, or touch one\nthat already exists, so that we can proliferate it in repos which are\nalready out in the world using submodules.\n\nSigned-off-by: Emily Shaffer \nHelped-by: Junio C Hamano \n---\n Documentation/config/submodule.txt | 6 ++++\n builtin/submodule--helper.c | 5 +++\n git-submodule.sh | 3 ++\n submodule.c | 18 +++++++++++\n t/t7400-submodule-basic.sh | 4 +++\n t/t7406-submodule-update.sh | 8 +++++\n t/t7412-submodule-absorbgitdirs.sh | 50 ++++++++++++++++++++++++++++--\n 7 files changed, 92 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt\nindex ee454f8126..99d5260b8e 100644\n--- a/Documentation/config/submodule.txt\n+++ b/Documentation/config/submodule.txt\n@@ -91,3 +91,9 @@ submodule.alternateErrorStrategy::\n \t`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`\n \tor `info`, and if there is an error with the computed alternate, the\n \tclone proceeds as if no alternate was specified.\n+\n+submodule.hasSuperproject::\n+\tIndicates whether this repository is a submodule. If this config is set\n+\tto 'true', Git may traverse the filesystem above this submodule in order\n+\tto identify the superproject. It is set automatically during submodule\n+\tcreation, update, and 'git submodule absorbgitdir'.\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex c5d3fc3817..92986646bc 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -1839,6 +1839,11 @@ static int clone_submodule(struct module_clone_data *clone_data)\n \t\tgit_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n \t\t\t\t error_strategy);\n \n+\t/*\n+\t * Teach the submodule that it's a submodule.\n+\t */\n+\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n+\n \tfree(sm_alternate);\n \tfree(error_strategy);\n \ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 652861aa66..59dffda775 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -449,6 +449,9 @@ cmd_update()\n \t\t\t;;\n \t\tesac\n \n+\t\t# Note that the submodule is a submodule.\n+\t\tgit -C \"$sm_path\" config submodule.hasSuperproject \"true\"\n+\n \t\tif test -n \"$recursive\"\n \t\tthen\n \t\t\t(\ndiff --git a/submodule.c b/submodule.c\nindex c689070524..741104af8a 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2097,6 +2097,8 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \tchar *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL;\n \tstruct strbuf new_gitdir = STRBUF_INIT;\n \tconst struct submodule *sub;\n+\tstruct config_set sub_cs;\n+\tstruct strbuf config_path = STRBUF_INIT, sb = STRBUF_INIT;\n \n \tif (submodule_uses_worktrees(path))\n \t\tdie(_(\"relocate_gitdir for submodule '%s' with \"\n@@ -2127,6 +2129,22 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \n \trelocate_gitdir(path, real_old_git_dir, real_new_git_dir);\n \n+\t/*\n+\t * Note location of superproject's gitdir. Because the submodule already\n+\t * has a gitdir and local config, we can store this pointer from\n+\t * worktree config to worktree config, if the submodule has\n+\t * extensions.worktreeConfig set.\n+\t */\n+\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n+\tgit_configset_init(&sub_cs);\n+\tgit_configset_add_file(&sub_cs, config_path.buf);\n+\n+\tgit_config_set_in_file(config_path.buf, \"submodule.hasSuperproject\",\n+\t\t\t \"true\");\n+\n+\tgit_configset_clear(&sub_cs);\n+\tstrbuf_release(&config_path);\n+\tstrbuf_release(&sb);\n \tfree(old_git_dir);\n \tfree(real_old_git_dir);\n \tfree(real_new_git_dir);\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex 40cf8d89aa..833fa01961 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -115,6 +115,10 @@ inspect() {\n \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n \tgit -C \"$sub_dir\" update-index --refresh &&\n \tgit -C \"$sub_dir\" diff-files --exit-code &&\n+\n+\t# Ensure that submodule.hasSuperproject is set.\n+\tgit -C \"$sub_dir\" config \"submodule.hasSuperproject\"\n+\n \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n \ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex 11cccbb333..422c3cc343 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n \t)\n '\n \n+test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n+\t(cd super &&\n+\t git -C submodule config --unset submodule.hasSuperproject &&\n+\t git submodule update &&\n+\t git -C submodule config submodule.hasSuperproject\n+\t)\n+'\n+\n test_done\ndiff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh\nindex 1cfa150768..187fb6bbbc 100755\n--- a/t/t7412-submodule-absorbgitdirs.sh\n+++ b/t/t7412-submodule-absorbgitdirs.sh\n@@ -30,7 +30,9 @@ test_expect_success 'absorb the git dir' '\n \tgit status >actual.1 &&\n \tgit -C sub1 rev-parse HEAD >actual.2 &&\n \ttest_cmp expect.1 actual.1 &&\n-\ttest_cmp expect.2 actual.2\n+\ttest_cmp expect.2 actual.2 &&\n+\n+\tgit -C sub1 config submodule.hasSuperproject\n '\n \n test_expect_success 'absorbing does not fail for deinitialized submodules' '\n@@ -61,7 +63,9 @@ test_expect_success 'absorb the git dir in a nested submodule' '\n \tgit status >actual.1 &&\n \tgit -C sub1/nested rev-parse HEAD >actual.2 &&\n \ttest_cmp expect.1 actual.1 &&\n-\ttest_cmp expect.2 actual.2\n+\ttest_cmp expect.2 actual.2 &&\n+\n+\tgit -C sub1/nested config submodule.hasSuperproject\n '\n \n test_expect_success 're-setup nested submodule' '\n@@ -130,4 +134,46 @@ test_expect_success 'absorbing fails for a submodule with multiple worktrees' '\n \ttest_i18ngrep \"not supported\" error\n '\n \n+test_expect_success 'absorbgitdirs works when called from a superproject worktree' '\n+\t# set up a worktree of the superproject\n+\tgit worktree add wt &&\n+\t(\n+\tcd wt &&\n+\n+\t# create a new unembedded git dir\n+\tgit init sub4 &&\n+\ttest_commit -C sub4 first &&\n+\tgit submodule add ./sub4 &&\n+\ttest_tick &&\n+\n+\t# absorb the git dir\n+\tgit submodule absorbgitdirs sub4 &&\n+\n+\t# make sure the submodule noted the superproject\n+\tgit -C sub4 config submodule.hasSuperproject\n+\t)\n+'\n+\n+test_expect_success 'absorbgitdirs works with a submodule with worktree config' '\n+\t# reuse the worktree of the superproject\n+\t(\n+\tcd wt &&\n+\n+\t# create a new unembedded git dir\n+\tgit init sub5 &&\n+\ttest_commit -C sub5 first &&\n+\tgit submodule add ./sub5 &&\n+\ttest_tick &&\n+\n+\t# turn on worktree configs for submodule\n+\tgit -C sub5 config extensions.worktreeConfig true &&\n+\n+\t# absorb the git dir\n+\tgit submodule absorbgitdirs sub5 &&\n+\n+\t# make sure the submodule noted the superproject\n+\tgit -C sub5 config submodule.hasSuperproject\n+\t)\n+'\n+\n test_done\n-- \n2.35.1.574.g5d30c73bfb-goog\n\n"},{"id":"449839","messageId":"20220301002613.1459916-4-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220301002613.1459916-1-emilyshaffer@google.com","subject":"[PATCH v8 3/3] rev-parse: short-circuit superproject worktree when config unset","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-01T00:26:13Z","receivedAt":"2022-03-01T00:26:46Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"In the previous commit, submodules learned a config\n'submodule.hasSuperproject' to indicate whether or not we should attempt\nto traverse the filesystem to find their superproject. To help test that\nthis config was added everywhere it should have been, begin using it to\ndecide whether to exit early from 'git rev-parse\n--show-superproject-working-dir'.\n\nSigned-off-by: Emily Shaffer \n\n---\n\nMaybe it's actually better to warn instead of error here? Or maybe it's\nbest not to say anything, but to set 'submodule.hasSuperproject' after\nwe successfully find the superproject?\n\nEither way - I ran the test suite with this early exit added and\neverything still passed. I made this change hoping to get a little\nsignal on whether the series achieved its goal, and in that regard I'm\nsatisfied.\n---\n submodule.c | 12 ++++++++++++\n 1 file changed, 12 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 741104af8a..463e7f0c48 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2237,6 +2237,7 @@ int get_superproject_working_tree(struct strbuf *buf)\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct strbuf one_up = STRBUF_INIT;\n \tconst char *cwd = xgetcwd();\n+\tint has_superproject_cfg = 0;\n \tint ret = 0;\n \tconst char *subpath;\n \tint code;\n@@ -2250,6 +2251,17 @@ int get_superproject_working_tree(struct strbuf *buf)\n \t\t */\n \t\treturn 0;\n \n+\tif (git_config_get_bool(\"submodule.hassuperproject\", &has_superproject_cfg)\n+\t || !has_superproject_cfg) {\n+\t\t/*\n+\t\t * If we don't have a superproject, then we're probably not a\n+\t\t * submodule. If this is failing and shouldn't be, investigate\n+\t\t * why the config was never set.\n+\t\t */\n+\t\terror(_(\"Asked to find a superproject, but submodule.hasSuperproject != true\"));\n+\t\treturn 0;\n+\t}\n+\n \tif (!strbuf_realpath(&one_up, \"../\", 0))\n \t\treturn 0;\n \n-- \n2.35.1.574.g5d30c73bfb-goog\n\n"},{"id":"449850","messageId":"xmqq7d9ewerw.fsf@gitster.g","threadId":"56920","inReplyTo":"20220301002613.1459916-1-emilyshaffer@google.com","subject":"Re: [PATCH v8 0/3] teach submodules to know they're submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-01T03:08:35Z","receivedAt":"2022-03-01T03:08:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> One thing I'm not sure about: in the tests, I check whether the config\n> is set, but not what the boolean value of it is. Is there a better way\n> to do that?\n\nAre you looking for value normalization during both setting and\nretrieving, i.e.\n\n\t$ git config vari.able 0 ;# or \"no\" or \"off\"\n\t$ git config --type=bool vari.abble\n\tfalse\n\t$ git config vari.able 1 ;# or \"yes\" or \"on\"\n\t$ git config --type=bool vari.abble\n\ttrue\n\n\t$ git config --type=bool vari.able yes ;# or \"1\" or \"on\"\n\t$ git config vari.able\n\ttrue\n\n"},{"id":"449871","messageId":"xmqqbkyqupg6.fsf@gitster.g","threadId":"56920","inReplyTo":"20220301002613.1459916-3-emilyshaffer@google.com","subject":"Re: [PATCH v8 2/3] introduce submodule.hasSuperproject record","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-01T07:00:57Z","receivedAt":"2022-03-01T07:01:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> +\t/*\n> +\t * Note location of superproject's gitdir. Because the submodule already\n> +\t * has a gitdir and local config, we can store this pointer from\n> +\t * worktree config to worktree config, if the submodule has\n> +\t * extensions.worktreeConfig set.\n> +\t */\n\nProbably the comment is a bit stale. There is no longer a pointer\nor location of superproject's gitdir recorded anywhere.\n\n> +\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n> +\tgit_configset_init(&sub_cs);\n> +\tgit_configset_add_file(&sub_cs, config_path.buf);\n> +\n> +\tgit_config_set_in_file(config_path.buf, \"submodule.hasSuperproject\",\n> +\t\t\t \"true\");\n> +\n> +\tgit_configset_clear(&sub_cs);\n> +\tstrbuf_release(&config_path);\n> +\tstrbuf_release(&sb);\n> \tfree(old_git_dir);\n> \tfree(real_old_git_dir);\n> \tfree(real_new_git_dir);\n> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\n> index 40cf8d89aa..833fa01961 100755\n> --- a/t/t7400-submodule-basic.sh\n> +++ b/t/t7400-submodule-basic.sh\n> @@ -115,6 +115,10 @@ inspect() {\n> \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n> \tgit -C \"$sub_dir\" update-index --refresh &&\n> \tgit -C \"$sub_dir\" diff-files --exit-code &&\n> +\n> +\t# Ensure that submodule.hasSuperproject is set.\n> +\tgit -C \"$sub_dir\" config \"submodule.hasSuperproject\"\n\nAre we sufficiently happy to see the variable is set to anything, or\ndo we require it to be set to boolean true?\n\nIf the former, the above is fine, with trailing && added.\n\nIf the latter, then something like\n\n\tval=$(git config --type=bool \"submodule.hasSuperproject\") &&\n\ttest \"$val\" = true &&\n\nwould be more appropriate, but I wonder something like\n\ntest_config_is () {\n\tlocal var expect val\n\tvar=\"$1\" expect=\"$2\"\n\tshift 2\n val=$(git \"$@\" config --type=bool \"$var\") &&\n\ttest \"$val\" = \"$expect\"\n}\n\nwould be in order. That would allow us to write\n\n\ttest_config_is submodule.hasSuperproject true -C \"$sub_dir\" &&\n\n> \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n> }\n> \n> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\n> index 11cccbb333..422c3cc343 100755\n> --- a/t/t7406-submodule-update.sh\n> +++ b/t/t7406-submodule-update.sh\n> @@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n> \t)\n> '\n> \n> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n> +\t(cd super &&\n> +\t git -C submodule config --unset submodule.hasSuperproject &&\n\nAre we testing that submodule.hasSuperproject is set, and that\nit can successfully be unset? \"config --unset no.such.var\" will\nexit with non-zero status.\n\n> +\t git submodule update &&\n> +\t git -C submodule config submodule.hasSuperproject\n\nDitto.\n"},{"id":"449874","messageId":"xmqq7d9eup7k.fsf@gitster.g","threadId":"56920","inReplyTo":"20220301002613.1459916-4-emilyshaffer@google.com","subject":"Re: [PATCH v8 3/3] rev-parse: short-circuit superproject worktree when config unset","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-01T07:06:07Z","receivedAt":"2022-03-01T07:06:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> diff --git a/submodule.c b/submodule.c\n> index 741104af8a..463e7f0c48 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -2237,6 +2237,7 @@ int get_superproject_working_tree(struct strbuf *buf)\n> \tstruct strbuf sb = STRBUF_INIT;\n> \tstruct strbuf one_up = STRBUF_INIT;\n> \tconst char *cwd = xgetcwd();\n> +\tint has_superproject_cfg = 0;\n> \tint ret = 0;\n> \tconst char *subpath;\n> \tint code;\n> @@ -2250,6 +2251,17 @@ int get_superproject_working_tree(struct strbuf *buf)\n> \t\t */\n> \t\treturn 0;\n> \n> +\tif (git_config_get_bool(\"submodule.hassuperproject\", &has_superproject_cfg)\n> +\t || !has_superproject_cfg) {\n\ngit_config_get_bool() returns 0 when it successfully finds the\nvariable, so the above says \"If submodule.hasSuperproject is not set\nat all, or if it is set to false, then...\"\n\n> +\t\t/*\n> +\t\t * If we don't have a superproject, then we're probably not a\n> +\t\t * submodule. If this is failing and shouldn't be, investigate\n> +\t\t * why the config was never set.\n> +\t\t */\n> +\t\terror(_(\"Asked to find a superproject, but submodule.hasSuperproject != true\"));\n> +\t\treturn 0;\n\nBut given that this thing is new, I am not sure if that is a\nsensible guard to use here. Shouldn't we say \n\n - If submodule.hasSuperproject is EXPLICITLY set to false then ...\n\ninstead? I.e.\n\n\tif (!git_config_get_bool(\"submodule.hassuperproject\", &value) &&\n\t !value) {\n\t\terror(_(\"asked to ...\"));\n\t\treturn 0;\n\t}\n\n"},{"id":"450767","messageId":"YiemguagkDdoUkp2@google.com","threadId":"56920","inReplyTo":"xmqq7d9ewerw.fsf@gitster.g","subject":"Re: [PATCH v8 0/3] teach submodules to know they're submodules","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-08T18:54:58Z","receivedAt":"2022-03-08T18:55:09Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Feb 28, 2022 at 07:08:35PM -0800, Junio C Hamano wrote:\n> \n> Emily Shaffer writes:\n> \n> > One thing I'm not sure about: in the tests, I check whether the config\n> > is set, but not what the boolean value of it is. Is there a better way\n> > to do that?\n> \n> Are you looking for value normalization during both setting and\n> retrieving, i.e.\n> \n> \t$ git config vari.able 0 ;# or \"no\" or \"off\"\n> \t$ git config --type=bool vari.abble\n> \tfalse\n> \t$ git config vari.able 1 ;# or \"yes\" or \"on\"\n> \t$ git config --type=bool vari.abble\n> \ttrue\n> \n> \t$ git config --type=bool vari.able yes ;# or \"1\" or \"on\"\n> \t$ git config vari.able\n> \ttrue\n> \n\nAh, thanks! This helps. But that means I still need to check the return\nvalue, and associate \"didn't find anything\" (1) with the default as\ndocumented in Docs/config/submodule.txt, right?\n\nEither way, this is useful. Thanks!\n\n - Emily\n"},{"id":"450770","messageId":"Yie2wvbviWxhia2e@google.com","threadId":"56920","inReplyTo":"xmqqbkyqupg6.fsf@gitster.g","subject":"Re: [PATCH v8 2/3] introduce submodule.hasSuperproject record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-08T20:04:18Z","receivedAt":"2022-03-08T20:04:29Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Feb 28, 2022 at 11:00:57PM -0800, Junio C Hamano wrote:\n> \n> Emily Shaffer writes:\n> \n> > +\t/*\n> > +\t * Note location of superproject's gitdir. Because the submodule already\n> > +\t * has a gitdir and local config, we can store this pointer from\n> > +\t * worktree config to worktree config, if the submodule has\n> > +\t * extensions.worktreeConfig set.\n> > +\t */\n> \n> Probably the comment is a bit stale. There is no longer a pointer\n> or location of superproject's gitdir recorded anywhere.\n\nThanks. I considered replacing it with a new comment about \"now we'll\nnote that it's got a superproject\", but I think that's clear enough from\nthe config set line, so I deleted the comment entirely.\n\n> \n> > +\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n> > +\tgit_configset_init(&sub_cs);\n> > +\tgit_configset_add_file(&sub_cs, config_path.buf);\n> > +\n> > +\tgit_config_set_in_file(config_path.buf, \"submodule.hasSuperproject\",\n> > +\t\t\t \"true\");\n> > +\n> > +\tgit_configset_clear(&sub_cs);\n> > +\tstrbuf_release(&config_path);\n> > +\tstrbuf_release(&sb);\n> > \tfree(old_git_dir);\n> > \tfree(real_old_git_dir);\n> > \tfree(real_new_git_dir);\n> > diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\n> > index 40cf8d89aa..833fa01961 100755\n> > --- a/t/t7400-submodule-basic.sh\n> > +++ b/t/t7400-submodule-basic.sh\n> > @@ -115,6 +115,10 @@ inspect() {\n> > \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n> > \tgit -C \"$sub_dir\" update-index --refresh &&\n> > \tgit -C \"$sub_dir\" diff-files --exit-code &&\n> > +\n> > +\t# Ensure that submodule.hasSuperproject is set.\n> > +\tgit -C \"$sub_dir\" config \"submodule.hasSuperproject\"\n> \n> Are we sufficiently happy to see the variable is set to anything, or\n> do we require it to be set to boolean true?\n> \n> If the former, the above is fine, with trailing && added.\n> \n> If the latter, then something like\n> \n> \tval=$(git config --type=bool \"submodule.hasSuperproject\") &&\n> \ttest \"$val\" = true &&\n> \n> would be more appropriate, but I wonder something like\n> \n> test_config_is () {\n> \tlocal var expect val\n> \tvar=\"$1\" expect=\"$2\"\n> \tshift 2\n> val=$(git \"$@\" config --type=bool \"$var\") &&\n> \ttest \"$val\" = \"$expect\"\n> }\n> \n> would be in order. That would allow us to write\n> \n> \ttest_config_is submodule.hasSuperproject true -C \"$sub_dir\" &&\n> \n\nThis seemed neat, so I started to look into implementing it, and found\n`test_cmp_config()` which takes additional args to pass to `git config`\n- so I should be able to achieve this same thing with\n`test_config_is -C \"$sub_dir\" --type=bool true submodule.hasSuperproject`.\n\n> > \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n> > }\n> > \n> > diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\n> > index 11cccbb333..422c3cc343 100755\n> > --- a/t/t7406-submodule-update.sh\n> > +++ b/t/t7406-submodule-update.sh\n> > @@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n> > \t)\n> > '\n> > \n> > +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n> > +\t(cd super &&\n> > +\t git -C submodule config --unset submodule.hasSuperproject &&\n> \n> Are we testing that submodule.hasSuperproject is set, and that\n> it can successfully be unset? \"config --unset no.such.var\" will\n> exit with non-zero status.\n> \n> > +\t git submodule update &&\n> > +\t git -C submodule config submodule.hasSuperproject\n> \n> Ditto.\n\nAh, thanks.\n"},{"id":"450779","messageId":"kl6lee3c5bzl.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56920","inReplyTo":"20220301002613.1459916-3-emilyshaffer@google.com","subject":"Re: [PATCH v8 2/3] introduce submodule.hasSuperproject record","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-03-08T22:13:34Z","receivedAt":"2022-03-08T22:13:51Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"\nReviewing this series lightly because I will need to base\n'ar/submodule-update reroll pt 2' on this (pt.1 is at\nhttps://lore.kernel.org/git/20220305001401.20888-1-chooglen@google.com).\n\nEmily Shaffer writes:\n\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 652861aa66..59dffda775 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -449,6 +449,9 @@ cmd_update()\n> \t\t\t;;\n> \t\tesac\n> \n> +\t\t# Note that the submodule is a submodule.\n> +\t\tgit -C \"$sm_path\" config submodule.hasSuperproject \"true\"\n> +\n> \t\tif test -n \"$recursive\"\n> \t\tthen\n> \t\t\t(\n\nThis hunk has a textual conflict with 'ar/submodule-update reroll pt\n2', but the fix is easy - just teach \"git submodule--helper update\" to\nset the config in C.\n\n> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\n> index 11cccbb333..422c3cc343 100755\n> --- a/t/t7406-submodule-update.sh\n> +++ b/t/t7406-submodule-update.sh\n> @@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n> \t)\n> '\n> \n> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n> +\t(cd super &&\n> +\t git -C submodule config --unset submodule.hasSuperproject &&\n> +\t git submodule update &&\n> +\t git -C submodule config submodule.hasSuperproject\n> +\t)\n> +'\n> +\n> test_done\n\n\nI think there is a gap in the test coverage. I notice that this doesn't\ntest that we set submodule.hasSuperproject when the submodule is cloned\nfor the first time with 'git submodule update'. I thought that maybe the\ntest for this was here...\n\n> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\n> index 40cf8d89aa..833fa01961 100755\n> --- a/t/t7400-submodule-basic.sh\n> +++ b/t/t7400-submodule-basic.sh\n> @@ -115,6 +115,10 @@ inspect() {\n> \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n> \tgit -C \"$sub_dir\" update-index --refresh &&\n> \tgit -C \"$sub_dir\" diff-files --exit-code &&\n> +\n> +\t# Ensure that submodule.hasSuperproject is set.\n> +\tgit -C \"$sub_dir\" config \"submodule.hasSuperproject\"\n> +\n> \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n> }\n> \n\nBut when I removed the \"set submodule.hasSuperproject in submodule\"\nline, i.e. \n\n \t\tgit -C \"$sm_path\" config submodule.hasSuperproject \"true\"\n\nt7400 still passes.\n"},{"id":"450809","messageId":"kl6lbkyg5b8z.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56920","inReplyTo":"kl6lee3c5bzl.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v8 2/3] introduce submodule.hasSuperproject record","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-03-08T22:29:32Z","receivedAt":"2022-03-08T22:29:40Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Glen Choo writes:\n\n> Emily Shaffer writes:\n>\n>> diff --git a/git-submodule.sh b/git-submodule.sh\n>> index 652861aa66..59dffda775 100755\n>> --- a/git-submodule.sh\n>> +++ b/git-submodule.sh\n>> @@ -449,6 +449,9 @@ cmd_update()\n>> \t\t\t;;\n>> \t\tesac\n>> \n>> +\t\t# Note that the submodule is a submodule.\n>> +\t\tgit -C \"$sm_path\" config submodule.hasSuperproject \"true\"\n>> +\n>> \t\tif test -n \"$recursive\"\n>> \t\tthen\n>> \t\t\t(\n>\n> This hunk has a textual conflict with 'ar/submodule-update reroll pt\n> 2', but the fix is easy - just teach \"git submodule--helper update\" to\n> set the config in C.\n\nFrom our dicussion (offline), it turns out this statement isn't really\ncorrect because we do set the config in C, but we do it in clone_submodule():\n\n diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n index c5d3fc3817..92986646bc 100644\n --- a/builtin/submodule--helper.c\n +++ b/builtin/submodule--helper.c\n @@ -1839,6 +1839,11 @@ static int clone_submodule(struct module_clone_data *clone_data)\n git_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n error_strategy);\n\n +\t/*\n +\t * Teach the submodule that it's a submodule.\n +\t */\n +\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n +\n free(sm_alternate);\n free(error_strategy);\n\n>> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\n>> index 11cccbb333..422c3cc343 100755\n>> --- a/t/t7406-submodule-update.sh\n>> +++ b/t/t7406-submodule-update.sh\n>> @@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n>> \t)\n>> '\n>> \n>> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n>> +\t(cd super &&\n>> +\t git -C submodule config --unset submodule.hasSuperproject &&\n>> +\t git submodule update &&\n>> +\t git -C submodule config submodule.hasSuperproject\n>> +\t)\n>> +'\n>> +\n>> test_done\n>\n>\n> I think there is a gap in the test coverage. I notice that this doesn't\n> test that we set submodule.hasSuperproject when the submodule is cloned\n> for the first time with 'git submodule update'. I thought that maybe the\n> test for this was here...\n>\n>> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\n>> index 40cf8d89aa..833fa01961 100755\n>> --- a/t/t7400-submodule-basic.sh\n>> +++ b/t/t7400-submodule-basic.sh\n>> @@ -115,6 +115,10 @@ inspect() {\n>> \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n>> \tgit -C \"$sub_dir\" update-index --refresh &&\n>> \tgit -C \"$sub_dir\" diff-files --exit-code &&\n>> +\n>> +\t# Ensure that submodule.hasSuperproject is set.\n>> +\tgit -C \"$sub_dir\" config \"submodule.hasSuperproject\"\n>> +\n>> \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n>> }\n>> \n>\n> But when I removed the \"set submodule.hasSuperproject in submodule\"\n> line, i.e. \n>\n> \t\tgit -C \"$sm_path\" config submodule.hasSuperproject \"true\"\n>\n> t7400 still passes.\n\nSo we would expect that newly cloned submodules would pass even without\nthis .sh line.\n\nI don't think we need to do this twice in C and in shell. We can move\nthis line:\n\n +\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n\ninto run-update-procedure (and out of clone_submodule()). This way it's\nguaranteed to touch every submodule (newly cloned or not).\n"},{"id":"450813","messageId":"Yif3G5zxP9kj/Lkc@google.com","threadId":"56920","inReplyTo":"xmqq7d9eup7k.fsf@gitster.g","subject":"Re: [PATCH v8 3/3] rev-parse: short-circuit superproject worktree when config unset","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-09T00:38:51Z","receivedAt":"2022-03-09T01:01:16Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Mon, Feb 28, 2022 at 11:06:07PM -0800, Junio C Hamano wrote:\n> \n> Emily Shaffer writes:\n> \n> > diff --git a/submodule.c b/submodule.c\n> > index 741104af8a..463e7f0c48 100644\n> > --- a/submodule.c\n> > +++ b/submodule.c\n> > @@ -2237,6 +2237,7 @@ int get_superproject_working_tree(struct strbuf *buf)\n> > \tstruct strbuf sb = STRBUF_INIT;\n> > \tstruct strbuf one_up = STRBUF_INIT;\n> > \tconst char *cwd = xgetcwd();\n> > +\tint has_superproject_cfg = 0;\n> > \tint ret = 0;\n> > \tconst char *subpath;\n> > \tint code;\n> > @@ -2250,6 +2251,17 @@ int get_superproject_working_tree(struct strbuf *buf)\n> > \t\t */\n> > \t\treturn 0;\n> > \n> > +\tif (git_config_get_bool(\"submodule.hassuperproject\", &has_superproject_cfg)\n> > +\t || !has_superproject_cfg) {\n> \n> git_config_get_bool() returns 0 when it successfully finds the\n> variable, so the above says \"If submodule.hasSuperproject is not set\n> at all, or if it is set to false, then...\"\n> \n> > +\t\t/*\n> > +\t\t * If we don't have a superproject, then we're probably not a\n> > +\t\t * submodule. If this is failing and shouldn't be, investigate\n> > +\t\t * why the config was never set.\n> > +\t\t */\n> > +\t\terror(_(\"Asked to find a superproject, but submodule.hasSuperproject != true\"));\n> > +\t\treturn 0;\n> \n> But given that this thing is new, I am not sure if that is a\n> sensible guard to use here. Shouldn't we say \n> \n> - If submodule.hasSuperproject is EXPLICITLY set to false then ...\n\nAh, interesting. I think that makes sense. I wrote this patch hoping to\nget an additional check for completeness of the previous patch (that is\n- if I can rely on that value for this other operation, and all the\ntests still pass without me touching them, then I seem to have wired\nthe new config through everywhere) and I think it's served that purpose;\nfor the real world, that's a little more dangerous, so I think relying\non explicit false makes sense. Will do.\n\n> \n> instead? I.e.\n> \n> \tif (!git_config_get_bool(\"submodule.hassuperproject\", &value) &&\n> \t !value) {\n> \t\terror(_(\"asked to ...\"));\n> \t\treturn 0;\n> \t}\n> \n"},{"id":"450965","messageId":"20220310004423.2627181-1-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220301002613.1459916-1-emilyshaffer@google.com","subject":"[PATCH v9 0/3] teach submodules to know they're submodules","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-10T00:44:20Z","receivedAt":"2022-03-10T00:44:35Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"For the original cover letter, see\nhttps://lore.kernel.org/git/20210611225428.1208973-1-emilyshaffer%40google.com.\n\nCI run: https://github.com/nasamuffin/git/actions/runs/1954710601\n\nSince v8:\n\nOnly a couple of minor fixes.\n\nJunio pointed out that I could write the tests better using --type=bool\nand 'test_cmp_config', and that we could be a little more careful about\nwhen to give up on 'git rev-parse --show-superproject-working-dir'.\n\nGlen mentioned that builtin/submodule--helper.c:run_update_procedure() is called\nunconditionally earlier in the same function where I had added the\nconfig in git-submodule.sh. So, I moved the config set into\nsubmodule--helper.c to reduce possible edge cases where the config might\nnot be set.\n\nOtherwise, this series is pretty much unchanged.\n\nSince v7:\n\nActually a fairly large rework. Rather than keeping the path from gitdir\nto gitdir, just keep a boolean under 'submodule.hasSuperproject'. The\nidea is that from this boolean, we can decide whether to traverse the\nfilesystem looking for a superproject.\n\nBecause this simplifies the implementation, I compressed the three\nmiddle commits into one. As proof-of-concept, I added a patch at the end\nto check for this boolean when running `git rev-parse\n--show-superproject-working-tree`.\n\nOne thing I'm not sure about: in the tests, I check whether the config\nis set, but not what the boolean value of it is. Is there a better way\nto do that? For example, I could imagine someone deciding to set\n`submodule.hasSuperproject = false` and the tests would not function\ncorrectly in that case. I think we don't really normalize the value on a\nboolean config like that, so I didn't want to write a lot of comparison\nto check if the value is 1 or true or True or TRUE or Yes or .... Am I\noverthinking it?\n\nThe other thing I'm not sure about: since it's just a bool, we're not\nrestricted to setting this config only when we have both gitdir paths\navailable. That makes me want to set the config any time we are doing\nsomething with submodules anyway, like any time 'git-submodule--helper'\nis used. But that helper seems to be called in the context of the\nsuperproject, not of the submodules, so adding this config for each\nsubmodule we touch would be a second child process. Is there some other\ncommon entry point for submodules that we can use?\n\n - Emily\n\nSince v6:\n\nI've dropped the fifth commit to use this new config for `git rev-parse\n--show-superproject-working-tree`. I think it did more harm than good -\nthat tool uses an odd way to determine whether the superproject is\nactually the superproject, anyways.\n\nI poked a little bit at trying to find some benchmark to demonstrate\nthat \"submodule.superprojectGitDir\" is actually faster - but I didn't\nend up able to write one without writing a ton of new code to traverse\nthe filesystem. To be honest, I'm not all that interested in performance\n- I want the config added for correctness, instead.\n\nSo, the only real changes between v6 and v7 are some documentation\nchanges suggested by Jonathan Tan\n(https://lore.kernel.org/git/20211117234300.2598132-1-jonathantanmy%40google.com).\n\nSince v5:\n\nA couple things. Firstly, a semantics change *back* to the semantics of\nv3 - we map from gitdir to gitdir, *not* from common dir to common dir,\nso that theoretically a submodule with multiple worktrees in multiple\nsuperproject worktrees will be able to figure out which worktree of the\nsuperproject it's in. (Realistically, that's not really possible right\nnow, but I'd like to change that soon.)\n\nSecondly, a rewording of comments and commit messages to indicate that\nthis isn't a cache of some expensive operation, but rather intended to\nbe the source of truth for all submodules. I also added a fifth commit\nrewriting `git rev-parse --show-superproject-working-tree` to\ndemonstrate what that means in practice - but from a practical\nstandpoint, I'm a little worried about that fifth patch. More details in\nthe patch 5 description.\n\nI did discuss Ævar's idea of relying on in-process filesystem digging to\nfind the superproject's gitdir with the rest of the Google team, but in\nthe end decided that there are some worries about filesystem digging in\nthis way (namely, some ugly interactions with network drives that are\nactually already an issue for Googler Linux machines). Plus, the allure\nof being able to definitively know that we're a submodule is pretty\nstrong. ;) But overall, this is the direction I'd prefer to keep going\nin, rather than trying to guess from the filesystem going forward.\n\nSince v4:\n\nThe only real change here is a slight semantics change to map from\n to . In every case\n*except* for when the superproject has a worktree, this changes nothing.\nFor the case when the superproject has a worktree, this means that now\nsubmodules will refer to the general superproject common dir (e.g. no\nworktree-specific refs or configs or whatnot).\n\nI *think* that because a submodule should exist in the context of the\ncommon dir, not the worktree gitdir, that is ok. However, it does mean\nit would be difficult to do something like sharing a config specific to\nthe worktree (the initial goal of this series).\n\n$ROOT/.git\n$ROOT/.git/config.superproject <- shared by $ROOT/.git/modules/sub\n$ROOT/.git/modules/sub <- points to $ROOT/.git\n$ROOT/.git/worktrees/wt\n$ROOT/.git/worktrees/wt/config.superproject <- contains a certain config-based pre-commit hook\n\nIf the submodule only knows about the common dir, that is tough, because\nthe submodule would basically have to guess which worktree it's in from\nits own path. There would be no way for '$WT/sub' to inherit\n'$ROOT/.git/worktrees/wt/config.superproject'.\n\nThat said... right now, we don't support submodules in worktrees very\nwell at all. A submodule in a worktree will get a brand new gitdir in\n$ROOT/.git/worktrees/modules/ (and that brand new gitdir would point to\nthe super's common dir). So I think we can punt on this entire question\nuntil we teach submodules and worktrees to play more gracefully together\n(it's on my long list...), and at that time we can probably introduce a\npointer from $ROOT/.git/modules/sub/worktrees/wt/ to\n$ROOT/.git/worktrees/wt/....\n\nOr, to summarize the long ramble above: \"this is still kind of weird\nwith worktrees, but let's fix it later when we fix worktrees more\nthoroughly\".\n\n(More rambling about worktree weirdness here:\nhttps://lore.kernel.org/git/YYRaII8YWVxlBqsF%40google.com )\n\n\nSince v3, a pretty major change: the semantics of\nsubmodule.superprojectGitDir has changed, to point from the submodule's\ngitdir to the superproject's gitdir (in v3 and earlier, we kept a path\nfrom the submodule's *worktree* to the superproject's gitdir instead).\nThis cleans up some of the confusions about the behavior when a\nsubmodule worktree moves around in the superproject's tree, or in a\nfuture when we support submodules having multiple worktrees.\n\nI also tried to simplify the tests to use 'test-tool path-utils\nrelative_path' everywhere - I think that makes them much more clear for\na test reader, but if you're reviewing and it isn't obvious what we're\ntesting for, please speak up.\n\nI think this is pretty mature and there was a lot of general agreement\nthat the gitdir->gitdir association was the way to go, so please be\nbrutal and look for nits, leaks, etc. this round ;)\n[/v4 cover letter]\n\nEmily Shaffer (3):\n t7400-submodule-basic: modernize inspect() helper\n introduce submodule.hasSuperproject record\n rev-parse: short-circuit superproject worktree when config unset\n\n Documentation/config/submodule.txt | 6 ++++\n builtin/submodule--helper.c | 11 +++++++\n submodule.c | 30 ++++++++++++++++++\n t/t1500-rev-parse.sh | 10 +++++-\n t/t7400-submodule-basic.sh | 42 ++++++++++++-------------\n t/t7406-submodule-update.sh | 8 +++++\n t/t7412-submodule-absorbgitdirs.sh | 50 ++++++++++++++++++++++++++++--\n 7 files changed, 131 insertions(+), 26 deletions(-)\n\nRange-diff against v8:\n-: ---------- > 1: 251510c687 t7400-submodule-basic: modernize inspect() helper\n1: 34cbfd81ee ! 2: da01dc7c10 introduce submodule.hasSuperproject record\n @@ builtin/submodule--helper.c: static int clone_submodule(struct module_clone_data\n \tfree(sm_alternate);\n \tfree(error_strategy);\n \n -\n - ## git-submodule.sh ##\n -@@ git-submodule.sh: cmd_update()\n - \t\t\t;;\n - \t\tesac\n +@@ builtin/submodule--helper.c: static int run_update_procedure(int argc, const char **argv, const char *prefix)\n + \n + \tfree(prefixed_path);\n \n -+\t\t# Note that the submodule is a submodule.\n -+\t\tgit -C \"$sm_path\" config submodule.hasSuperproject \"true\"\n ++\t/*\n ++\t * This entry point is always called from a submodule, so this is a\n ++\t * good place to set a hint that this repo is a submodule.\n ++\t */\n ++\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n +\n - \t\tif test -n \"$recursive\"\n - \t\tthen\n - \t\t\t(\n + \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n + \t\treturn do_run_update_procedure(&update_data);\n + \n \n ## submodule.c ##\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *path)\n @@ submodule.c: static void relocate_single_git_dir_into_superproject(const char *p\n \n \trelocate_gitdir(path, real_old_git_dir, real_new_git_dir);\n \n -+\t/*\n -+\t * Note location of superproject's gitdir. Because the submodule already\n -+\t * has a gitdir and local config, we can store this pointer from\n -+\t * worktree config to worktree config, if the submodule has\n -+\t * extensions.worktreeConfig set.\n -+\t */\n +\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n +\tgit_configset_init(&sub_cs);\n +\tgit_configset_add_file(&sub_cs, config_path.buf);\n @@ t/t7400-submodule-basic.sh: inspect() {\n \tgit -C \"$sub_dir\" diff-files --exit-code &&\n +\n +\t# Ensure that submodule.hasSuperproject is set.\n -+\tgit -C \"$sub_dir\" config \"submodule.hasSuperproject\"\n ++\ttest_cmp_config -C \"$sub_dir\" true --type=bool \"submodule.hasSuperproject\"\n +\n \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n @@ t/t7406-submodule-update.sh: test_expect_success 'submodule update --quiet passe\n \n +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n +\t(cd super &&\n -+\t git -C submodule config --unset submodule.hasSuperproject &&\n ++\t test_unconfig submodule.hasSuperproject &&\n +\t git submodule update &&\n -+\t git -C submodule config submodule.hasSuperproject\n ++\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n +\t)\n +'\n +\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorb the git dir' '\n -\ttest_cmp expect.2 actual.2\n +\ttest_cmp expect.2 actual.2 &&\n +\n -+\tgit -C sub1 config submodule.hasSuperproject\n ++\ttest_cmp_config -C sub1 true --type=bool submodule.hasSuperproject\n '\n \n test_expect_success 'absorbing does not fail for deinitialized submodules' '\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorb the git dir in a\n -\ttest_cmp expect.2 actual.2\n +\ttest_cmp expect.2 actual.2 &&\n +\n -+\tgit -C sub1/nested config submodule.hasSuperproject\n ++\ttest_cmp_config -C sub1/nested true --type=bool submodule.hasSuperproject\n '\n \n test_expect_success 're-setup nested submodule' '\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorbing fails for a s\n +\tgit submodule absorbgitdirs sub4 &&\n +\n +\t# make sure the submodule noted the superproject\n -+\tgit -C sub4 config submodule.hasSuperproject\n ++\ttest_cmp_config -C sub4 true --type=bool submodule.hasSuperproject\n +\t)\n +'\n +\n @@ t/t7412-submodule-absorbgitdirs.sh: test_expect_success 'absorbing fails for a s\n +\tgit submodule absorbgitdirs sub5 &&\n +\n +\t# make sure the submodule noted the superproject\n -+\tgit -C sub5 config submodule.hasSuperproject\n ++\ttest_cmp_config -C sub5 true --type=bool submodule.hasSuperproject\n +\t)\n +'\n +\n2: c14ee8760f < -: ---------- rev-parse: short-circuit superproject worktree when config unset\n-: ---------- > 3: 1893a84fdc rev-parse: short-circuit superproject worktree when config unset\n-- \n2.35.1.616.g0bdcbb4464-goog\n\n"},{"id":"450966","messageId":"20220310004423.2627181-2-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220310004423.2627181-1-emilyshaffer@google.com","subject":"[PATCH v9 1/3] t7400-submodule-basic: modernize inspect() helper","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-10T00:44:21Z","receivedAt":"2022-03-10T00:44:36Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Since the inspect() helper in the submodule-basic test suite was\nwritten, 'git -C ' was added. By using -C, we no longer need a\nreference to the base directory for the test. This simplifies callsites,\nand will make the addition of other arguments in later patches more\nreadable.\n\nSigned-off-by: Emily Shaffer \n---\n t/t7400-submodule-basic.sh | 40 +++++++++++++++-----------------------\n 1 file changed, 16 insertions(+), 24 deletions(-)\n\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex e7cec2e457..40cf8d89aa 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -107,23 +107,15 @@ test_expect_success 'setup - repository to add submodules to' '\n # generates, which will expand symbolic links.\n submodurl=$(pwd -P)\n \n-listbranches() {\n-\tgit for-each-ref --format='%(refname)' 'refs/heads/*'\n-}\n-\n inspect() {\n-\tdir=$1 &&\n-\tdotdot=\"${2:-..}\" &&\n-\n-\t(\n-\t\tcd \"$dir\" &&\n-\t\tlistbranches >\"$dotdot/heads\" &&\n-\t\t{ git symbolic-ref HEAD || :; } >\"$dotdot/head\" &&\n-\t\tgit rev-parse HEAD >\"$dotdot/head-sha1\" &&\n-\t\tgit update-index --refresh &&\n-\t\tgit diff-files --exit-code &&\n-\t\tgit clean -n -d -x >\"$dotdot/untracked\"\n-\t)\n+\tsub_dir=$1 &&\n+\n+\tgit -C \"$sub_dir\" for-each-ref --format='%(refname)' 'refs/heads/*' >heads &&\n+\t{ git -C \"$sub_dir\" symbolic-ref HEAD || :; } >head &&\n+\tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n+\tgit -C \"$sub_dir\" update-index --refresh &&\n+\tgit -C \"$sub_dir\" diff-files --exit-code &&\n+\tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n \n test_expect_success 'submodule add' '\n@@ -146,7 +138,7 @@ test_expect_success 'submodule add' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod ../.. &&\n+\tinspect addtest/submod &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -248,7 +240,7 @@ test_expect_success 'submodule add --branch' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/submod-branch ../.. &&\n+\tinspect addtest/submod-branch &&\n \ttest_cmp expect-heads heads &&\n \ttest_cmp expect-head head &&\n \ttest_must_be_empty untracked\n@@ -264,7 +256,7 @@ test_expect_success 'submodule add with ./ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotsubmod/frotz ../../.. &&\n+\tinspect addtest/dotsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -280,7 +272,7 @@ test_expect_success 'submodule add with /././ in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/dotslashdotsubmod/frotz ../../.. &&\n+\tinspect addtest/dotslashdotsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -296,7 +288,7 @@ test_expect_success 'submodule add with // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/slashslashsubmod/frotz ../../.. &&\n+\tinspect addtest/slashslashsubmod/frotz &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -312,7 +304,7 @@ test_expect_success 'submodule add with /.. in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod ../.. &&\n+\tinspect addtest/realsubmod &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -328,7 +320,7 @@ test_expect_success 'submodule add with ./, /.. and // in path' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod2 ../.. &&\n+\tinspect addtest/realsubmod2 &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n@@ -359,7 +351,7 @@ test_expect_success 'submodule add in subdirectory' '\n \t) &&\n \n \trm -f heads head untracked &&\n-\tinspect addtest/realsubmod3 ../.. &&\n+\tinspect addtest/realsubmod3 &&\n \ttest_cmp expect heads &&\n \ttest_cmp expect head &&\n \ttest_must_be_empty untracked\n-- \n2.35.1.616.g0bdcbb4464-goog\n\n"},{"id":"450967","messageId":"20220310004423.2627181-3-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220310004423.2627181-1-emilyshaffer@google.com","subject":"[PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-10T00:44:22Z","receivedAt":"2022-03-10T00:44:43Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"Teach submodules a config variable indicating the fact that they are a\nsubmodule. If this config is set to false or unset, Git may assume the\ncurrent repo is not a submodule.\n\nGit commands can use this variable to decide whether to traverse the\nfilesystem and look for a superproject at all. 'git rev-parse\n--show-superproject-working-tree' can learn to exit early if this config\nis unset or false. Other newly added or implicit behavior - like \"git\nstatus\" showing the submodule's status in relation to the superproject,\nor a config shared between the superproject and submodule - can use this\nconfig to decide whether to search the parent directory to find a\nsuperproject.\n\nIntroduce this config everywhere we add a new submodule, or touch one\nthat already exists, so that we can proliferate it in repos which are\nalready out in the world using submodules.\n\nSigned-off-by: Emily Shaffer \nHelped-by: Junio C Hamano \n---\n Documentation/config/submodule.txt | 6 ++++\n builtin/submodule--helper.c | 11 +++++++\n submodule.c | 12 +++++++\n t/t7400-submodule-basic.sh | 4 +++\n t/t7406-submodule-update.sh | 8 +++++\n t/t7412-submodule-absorbgitdirs.sh | 50 ++++++++++++++++++++++++++++--\n 6 files changed, 89 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt\nindex ee454f8126..99d5260b8e 100644\n--- a/Documentation/config/submodule.txt\n+++ b/Documentation/config/submodule.txt\n@@ -91,3 +91,9 @@ submodule.alternateErrorStrategy::\n \t`ignore`, `info`, `die`. Default is `die`. Note that if set to `ignore`\n \tor `info`, and if there is an error with the computed alternate, the\n \tclone proceeds as if no alternate was specified.\n+\n+submodule.hasSuperproject::\n+\tIndicates whether this repository is a submodule. If this config is set\n+\tto 'true', Git may traverse the filesystem above this submodule in order\n+\tto identify the superproject. It is set automatically during submodule\n+\tcreation, update, and 'git submodule absorbgitdir'.\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex c5d3fc3817..eda9ed550e 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -1839,6 +1839,11 @@ static int clone_submodule(struct module_clone_data *clone_data)\n \t\tgit_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n \t\t\t\t error_strategy);\n \n+\t/*\n+\t * Teach the submodule that it's a submodule.\n+\t */\n+\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n+\n \tfree(sm_alternate);\n \tfree(error_strategy);\n \n@@ -2617,6 +2622,12 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n \n \tfree(prefixed_path);\n \n+\t/*\n+\t * This entry point is always called from a submodule, so this is a\n+\t * good place to set a hint that this repo is a submodule.\n+\t */\n+\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n+\n \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n \t\treturn do_run_update_procedure(&update_data);\n \ndiff --git a/submodule.c b/submodule.c\nindex c689070524..aafbd628ad 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2097,6 +2097,8 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \tchar *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL;\n \tstruct strbuf new_gitdir = STRBUF_INIT;\n \tconst struct submodule *sub;\n+\tstruct config_set sub_cs;\n+\tstruct strbuf config_path = STRBUF_INIT, sb = STRBUF_INIT;\n \n \tif (submodule_uses_worktrees(path))\n \t\tdie(_(\"relocate_gitdir for submodule '%s' with \"\n@@ -2127,6 +2129,16 @@ static void relocate_single_git_dir_into_superproject(const char *path)\n \n \trelocate_gitdir(path, real_old_git_dir, real_new_git_dir);\n \n+\tstrbuf_addf(&config_path, \"%s/config\", real_new_git_dir);\n+\tgit_configset_init(&sub_cs);\n+\tgit_configset_add_file(&sub_cs, config_path.buf);\n+\n+\tgit_config_set_in_file(config_path.buf, \"submodule.hasSuperproject\",\n+\t\t\t \"true\");\n+\n+\tgit_configset_clear(&sub_cs);\n+\tstrbuf_release(&config_path);\n+\tstrbuf_release(&sb);\n \tfree(old_git_dir);\n \tfree(real_old_git_dir);\n \tfree(real_new_git_dir);\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex 40cf8d89aa..53c8bf699d 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -115,6 +115,10 @@ inspect() {\n \tgit -C \"$sub_dir\" rev-parse HEAD >head-sha1 &&\n \tgit -C \"$sub_dir\" update-index --refresh &&\n \tgit -C \"$sub_dir\" diff-files --exit-code &&\n+\n+\t# Ensure that submodule.hasSuperproject is set.\n+\ttest_cmp_config -C \"$sub_dir\" true --type=bool \"submodule.hasSuperproject\"\n+\n \tgit -C \"$sub_dir\" clean -n -d -x >untracked\n }\n \ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex 11cccbb333..ec2397fc69 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n \t)\n '\n \n+test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n+\t(cd super &&\n+\t test_unconfig submodule.hasSuperproject &&\n+\t git submodule update &&\n+\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n+\t)\n+'\n+\n test_done\ndiff --git a/t/t7412-submodule-absorbgitdirs.sh b/t/t7412-submodule-absorbgitdirs.sh\nindex 1cfa150768..4c33a98efa 100755\n--- a/t/t7412-submodule-absorbgitdirs.sh\n+++ b/t/t7412-submodule-absorbgitdirs.sh\n@@ -30,7 +30,9 @@ test_expect_success 'absorb the git dir' '\n \tgit status >actual.1 &&\n \tgit -C sub1 rev-parse HEAD >actual.2 &&\n \ttest_cmp expect.1 actual.1 &&\n-\ttest_cmp expect.2 actual.2\n+\ttest_cmp expect.2 actual.2 &&\n+\n+\ttest_cmp_config -C sub1 true --type=bool submodule.hasSuperproject\n '\n \n test_expect_success 'absorbing does not fail for deinitialized submodules' '\n@@ -61,7 +63,9 @@ test_expect_success 'absorb the git dir in a nested submodule' '\n \tgit status >actual.1 &&\n \tgit -C sub1/nested rev-parse HEAD >actual.2 &&\n \ttest_cmp expect.1 actual.1 &&\n-\ttest_cmp expect.2 actual.2\n+\ttest_cmp expect.2 actual.2 &&\n+\n+\ttest_cmp_config -C sub1/nested true --type=bool submodule.hasSuperproject\n '\n \n test_expect_success 're-setup nested submodule' '\n@@ -130,4 +134,46 @@ test_expect_success 'absorbing fails for a submodule with multiple worktrees' '\n \ttest_i18ngrep \"not supported\" error\n '\n \n+test_expect_success 'absorbgitdirs works when called from a superproject worktree' '\n+\t# set up a worktree of the superproject\n+\tgit worktree add wt &&\n+\t(\n+\tcd wt &&\n+\n+\t# create a new unembedded git dir\n+\tgit init sub4 &&\n+\ttest_commit -C sub4 first &&\n+\tgit submodule add ./sub4 &&\n+\ttest_tick &&\n+\n+\t# absorb the git dir\n+\tgit submodule absorbgitdirs sub4 &&\n+\n+\t# make sure the submodule noted the superproject\n+\ttest_cmp_config -C sub4 true --type=bool submodule.hasSuperproject\n+\t)\n+'\n+\n+test_expect_success 'absorbgitdirs works with a submodule with worktree config' '\n+\t# reuse the worktree of the superproject\n+\t(\n+\tcd wt &&\n+\n+\t# create a new unembedded git dir\n+\tgit init sub5 &&\n+\ttest_commit -C sub5 first &&\n+\tgit submodule add ./sub5 &&\n+\ttest_tick &&\n+\n+\t# turn on worktree configs for submodule\n+\tgit -C sub5 config extensions.worktreeConfig true &&\n+\n+\t# absorb the git dir\n+\tgit submodule absorbgitdirs sub5 &&\n+\n+\t# make sure the submodule noted the superproject\n+\ttest_cmp_config -C sub5 true --type=bool submodule.hasSuperproject\n+\t)\n+'\n+\n test_done\n-- \n2.35.1.616.g0bdcbb4464-goog\n\n"},{"id":"450968","messageId":"20220310004423.2627181-4-emilyshaffer@google.com","threadId":"56920","inReplyTo":"20220310004423.2627181-1-emilyshaffer@google.com","subject":"[PATCH v9 3/3] rev-parse: short-circuit superproject worktree when config unset","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-10T00:44:23Z","receivedAt":"2022-03-10T00:44:44Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"In the previous commit, submodules learned a config\n'submodule.hasSuperproject' to indicate whether or not we should attempt\nto traverse the filesystem to find their superproject. To help test that\nthis config was added everywhere it should have been, begin using it to\ndecide whether to exit early from 'git rev-parse\n--show-superproject-working-dir'. Because that command is fairly old,\nonly short-circuit if the new config was explicitly set to false.\n\nSigned-off-by: Emily Shaffer \n---\n submodule.c | 18 ++++++++++++++++++\n t/t1500-rev-parse.sh | 10 +++++++++-\n 2 files changed, 27 insertions(+), 1 deletion(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex aafbd628ad..64760f1e3a 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -2231,6 +2231,7 @@ int get_superproject_working_tree(struct strbuf *buf)\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct strbuf one_up = STRBUF_INIT;\n \tconst char *cwd = xgetcwd();\n+\tint has_superproject_cfg = 0;\n \tint ret = 0;\n \tconst char *subpath;\n \tint code;\n@@ -2244,6 +2245,23 @@ int get_superproject_working_tree(struct strbuf *buf)\n \t\t */\n \t\treturn 0;\n \n+\t/*\n+\t * Because get_superproject_working_tree() is older than\n+\t * submodule.hasSuperproject, don't rely on the default \"unset = false\"\n+\t * - instead, only rely on if submodule.hasSuperproject was explicitly\n+\t * set to false.\n+\t */\n+\tif (! git_config_get_bool(\"submodule.hassuperproject\", &has_superproject_cfg)\n+\t && !has_superproject_cfg) {\n+\t\t/*\n+\t\t * If we don't have a superproject, then we're probably not a\n+\t\t * submodule. If this is failing and shouldn't be, investigate\n+\t\t * why the config was set to false.\n+\t\t */\n+\t\terror(_(\"Asked to find a superproject, but submodule.hasSuperproject == false\"));\n+\t\treturn 0;\n+\t}\n+\n \tif (!strbuf_realpath(&one_up, \"../\", 0))\n \t\treturn 0;\n \ndiff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\nindex 1c2df08333..dd35036bd6 100755\n--- a/t/t1500-rev-parse.sh\n+++ b/t/t1500-rev-parse.sh\n@@ -244,7 +244,15 @@ test_expect_success 'showing the superproject correctly' '\n \ttest_must_fail git -C super merge branch1 &&\n \n \tgit -C super/dir/sub rev-parse --show-superproject-working-tree >out &&\n-\ttest_cmp expect out\n+\ttest_cmp expect out &&\n+\n+\t# When submodule.hasSuperproject=false, --show-superproject-working-tree\n+\t# should fail instead of checking the filesystem.\n+\ttest_config -C super/dir/sub submodule.hasSuperproject false &&\n+\tgit -C super/dir/sub rev-parse --show-superproject-working-tree >out &&\n+\t# --show-superproject-working-tree should print an error about the\n+\t# broken config\n+\t! grep \"error:.*hasSuperproject\" out\n '\n \n # at least one external project depends on this behavior:\n-- \n2.35.1.616.g0bdcbb4464-goog\n\n"},{"id":"450976","messageId":"xmqq35jqino7.fsf@gitster.g","threadId":"56920","inReplyTo":"20220310004423.2627181-4-emilyshaffer@google.com","subject":"Re: [PATCH v9 3/3] rev-parse: short-circuit superproject worktree when config unset","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-10T01:47:20Z","receivedAt":"2022-03-10T01:47:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> +\t/*\n> +\t * Because get_superproject_working_tree() is older than\n> +\t * submodule.hasSuperproject, don't rely on the default \"unset = false\"\n> +\t * - instead, only rely on if submodule.hasSuperproject was explicitly\n> +\t * set to false.\n> +\t */\n\nThat's a round-about way to say that submodule.hassuperproject\ndefaults to true, isn't it ;-)?\n\n> +\tif (! git_config_get_bool(\"submodule.hassuperproject\", &has_superproject_cfg)\n> +\t && !has_superproject_cfg) {\n> +\t\t/*\n> +\t\t * If we don't have a superproject, then we're probably not a\n> +\t\t * submodule. If this is failing and shouldn't be, investigate\n> +\t\t * why the config was set to false.\n> +\t\t */\n> +\t\terror(_(\"Asked to find a superproject, but submodule.hasSuperproject == false\"));\n\ns/Asked to/asked to/, probably.\n\n> +\t\treturn 0;\n> +\t}\n> +\n> \tif (!strbuf_realpath(&one_up, \"../\", 0))\n> \t\treturn 0;\n> \n> diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\n> index 1c2df08333..dd35036bd6 100755\n> --- a/t/t1500-rev-parse.sh\n> +++ b/t/t1500-rev-parse.sh\n> @@ -244,7 +244,15 @@ test_expect_success 'showing the superproject correctly' '\n> \ttest_must_fail git -C super merge branch1 &&\n> \n> \tgit -C super/dir/sub rev-parse --show-superproject-working-tree >out &&\n> -\ttest_cmp expect out\n> +\ttest_cmp expect out &&\n> +\n> +\t# When submodule.hasSuperproject=false, --show-superproject-working-tree\n> +\t# should fail instead of checking the filesystem.\n> +\ttest_config -C super/dir/sub submodule.hasSuperproject false &&\n> +\tgit -C super/dir/sub rev-parse --show-superproject-working-tree >out &&\n> +\t# --show-superproject-working-tree should print an error about the\n> +\t# broken config\n> +\t! grep \"error:.*hasSuperproject\" out\n> '\n> \n> # at least one external project depends on this behavior:\n"},{"id":"450979","messageId":"xmqqtuc6h83m.fsf@gitster.g","threadId":"56920","inReplyTo":"20220310004423.2627181-3-emilyshaffer@google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-10T02:09:01Z","receivedAt":"2022-03-10T02:09:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> @@ -2617,6 +2622,12 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n> \n> \tfree(prefixed_path);\n> \n> +\t/*\n> +\t * This entry point is always called from a submodule, so this is a\n> +\t * good place to set a hint that this repo is a submodule.\n> +\t */\n> +\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n> +\n> \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n> \t\treturn do_run_update_procedure(&update_data);\n\nIn Glen's update to rewrite \"submodule update\" in C, this part is\nreplaced with a call to update_submodule2(). I am not sure what the\ncurrent repository is at this point of the code with and without\nGlen's topic, but are we sure we are in a submodule we discovered?\n\nbuiltin/submodule--helper.c::run_update_procedure() takes sm_path to\nthe path to the submodule, presumably from superproject's point of\nview, and the callchain leads to a call to run_update_command()\neventually, which uses run_command_v_opt_cd_env() to go in to the\nsubmodule repository and run an external git command (like\n\"checkout\"), so it looks like what git_config_set() updates is the\nsuperprojects' configuration, not the configuration of a particular\nsubmodule being updated.\n\nThe other one, where cmd_clone() sets the variable in submodule's\nconfiguration file, looks good, but I am not sure about this one.\n"},{"id":"450980","messageId":"xmqqpmmuh6zq.fsf@gitster.g","threadId":"56920","inReplyTo":"20220310004423.2627181-3-emilyshaffer@google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-10T02:32:57Z","receivedAt":"2022-03-10T02:33:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n> +\t(cd super &&\n> +\t test_unconfig submodule.hasSuperproject &&\n\nSo, before we run the test, we unconfig the variable in the SUPER\nrepository, and then\n\n> +\t git submodule update &&\n\nupdate the submodule, and then\n\n> +\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n\ngo in to the submodule and check the value of the variable?\n\nShouldn't the first part be more like\n\n\t(cd super &&\n\t test_unconfig -C submodule submodule.hasSuperproject &&\n\nif we want to make sure \"submodule update\" sets it there?\n\n> +\t)\n> +'\n"},{"id":"450983","messageId":"CAPig+cRsbNQpg4y=KfbF6m4PDaoQ-RdKuhEoG1oeFF48i5RWuw@mail.gmail.com","threadId":"56920","inReplyTo":"xmqq35jqino7.fsf@gitster.g","subject":"Re: [PATCH v9 3/3] rev-parse: short-circuit superproject worktree when config unset","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-03-10T04:39:42Z","receivedAt":"2022-03-10T04:39:57Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Mar 9, 2022 at 10:57 PM Junio C Hamano wrote:\n> Emily Shaffer writes:\n> > + error(_(\"Asked to find a superproject, but submodule.hasSuperproject == false\"));\n>\n> s/Asked to/asked to/, probably.\n\nThis is a user-facing error message, not just a programmer-facing\nmessage, correct? If so, then perhaps: s/==/is/\n"},{"id":"451080","messageId":"kl6lczitsdhm.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56920","inReplyTo":"xmqqtuc6h83m.fsf@gitster.g","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-03-10T21:29:25Z","receivedAt":"2022-03-10T21:29:39Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Junio C Hamano writes:\n\n> Emily Shaffer writes:\n>\n>> @@ -2617,6 +2622,12 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n>> \n>> \tfree(prefixed_path);\n>> \n>> +\t/*\n>> +\t * This entry point is always called from a submodule, so this is a\n>> +\t * good place to set a hint that this repo is a submodule.\n>> +\t */\n>> +\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n>> +\n>> \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n>> \t\treturn do_run_update_procedure(&update_data);\n>\n> In Glen's update to rewrite \"submodule update\" in C, this part is\n> replaced with a call to update_submodule2(). I am not sure what the\n> current repository is at this point of the code with and without\n> Glen's topic, but are we sure we are in a submodule we discovered?\n\nWith my topic, this call would be moved into update_submodule2(). The\nrepository at that point is the superproject.\n\n> builtin/submodule--helper.c::run_update_procedure() takes sm_path to\n> the path to the submodule, presumably from superproject's point of\n> view, and the callchain leads to a call to run_update_command()\n> eventually, which uses run_command_v_opt_cd_env() to go in to the\n> submodule repository and run an external git command (like\n> \"checkout\"), so it looks like what git_config_set() updates is the\n> superprojects' configuration, not the configuration of a particular\n> submodule being updated.\n>\n> The other one, where cmd_clone() sets the variable in submodule's\n> configuration file, looks good, but I am not sure about this one.\n\nBut in this series, the current repository is the submodule because this\npart happens in a \"run-update-procedure\" child process.\n\nSo there is a slight conflict here, but the conflict existed even before\nthis change (we used to do this twice in git-submodule.sh and\nmodule_clone()).\n\nBecause of that conflict, I was planning to base \"part2\" on this series\nanyway, and if anything, this change makes the conflict better because\nwe now set \"submodule.hasSuperproject\" in only one place\n(run_update_procedure()) instead of two.\n\nSo I think this change improves this series and improves the interaction\nwith mine.\n"},{"id":"451082","messageId":"kl6la6dxsczx.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56920","inReplyTo":"xmqqtuc6h83m.fsf@gitster.g","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-03-10T21:40:02Z","receivedAt":"2022-03-10T21:40:09Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Junio C Hamano writes:\n\n> Emily Shaffer writes:\n>\n>> @@ -2617,6 +2622,12 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n>> \n>> \tfree(prefixed_path);\n>> \n>> +\t/*\n>> +\t * This entry point is always called from a submodule, so this is a\n>> +\t * good place to set a hint that this repo is a submodule.\n>> +\t */\n>> +\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n>> +\n>> \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n>> \t\treturn do_run_update_procedure(&update_data);\n>\n> In Glen's update to rewrite \"submodule update\" in C, this part is\n> replaced with a call to update_submodule2(). I am not sure what the\n> current repository is at this point of the code with and without\n> Glen's topic, but are we sure we are in a submodule we discovered?\n\nRereading this, I realize you probably meant that this conflicts with\npart1, not part2...\n\nAt the end of part1, update_submodule2() is called from inside the\nsubmodule (specifically from run_update_procedure()). So a good merge\nconflict resolution would be to set the config _before_ calling\nupdate_submodule2(). e.g.\n\n----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex bef9ab22d4..f53808d995 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -2672,6 +2677,11 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n &update_data.update_strategy);\n\n free(prefixed_path);\n+ /*\n+ * This entry point is always called from a submodule, so this is a\n+ * good place to set a hint that this repo is a submodule.\n+ */\n+ git_config_set(\"submodule.hasSuperproject\", \"true\");\n return update_submodule2(&update_data);\n }\n"},{"id":"451083","messageId":"kl6l7d91sccb.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56920","inReplyTo":"20220310004423.2627181-3-emilyshaffer@google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-03-10T21:54:12Z","receivedAt":"2022-03-10T21:54:16Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Emily Shaffer writes:\n\n> diff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt\n> index ee454f8126..99d5260b8e 100644\n> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> index c5d3fc3817..eda9ed550e 100644\n> --- a/builtin/submodule--helper.c\n> +++ b/builtin/submodule--helper.c\n> @@ -1839,6 +1839,11 @@ static int clone_submodule(struct module_clone_data *clone_data)\n> \t\tgit_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n> \t\t\t\t error_strategy);\n> \n> +\t/*\n> +\t * Teach the submodule that it's a submodule.\n> +\t */\n> +\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n> +\n> \tfree(sm_alternate);\n> \tfree(error_strategy);\n\nThis git_config_set_* is superfluous - it sets the config in newly\ncloned submodules..\n\n> \n> @@ -2617,6 +2622,12 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n> \n> \tfree(prefixed_path);\n> \n> +\t/*\n> +\t * This entry point is always called from a submodule, so this is a\n> +\t * good place to set a hint that this repo is a submodule.\n> +\t */\n> +\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n> +\n> \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n> \t\treturn do_run_update_procedure(&update_data);\n> \n\nbut this is called over *all* submodules, so we're guaranteed to always\nset the config if \"git submodule update\" isn't interrupted halfway.\n\nI don't think we guarantee correctness if it is interrupted halfway e.g.\ncore.worktree can be unset if it is interrupted halfway (because\nensure-core-worktree is called adjacent to run-update-procedure, not\ninside of update-clone).\n\nSo I think it's better to just drop the previous hunk - it will\ndisappear anyway in gc/submodule-update-part2.\n"},{"id":"451084","messageId":"xmqqwnh1bgr4.fsf@gitster.g","threadId":"56920","inReplyTo":"kl6la6dxsczx.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-10T22:10:55Z","receivedAt":"2022-03-10T22:11:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Glen Choo writes:\n\n> Junio C Hamano writes:\n>\n>> Emily Shaffer writes:\n>>\n>>> @@ -2617,6 +2622,12 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n>>> \n>>> \tfree(prefixed_path);\n>>> \n>>> +\t/*\n>>> +\t * This entry point is always called from a submodule, so this is a\n>>> +\t * good place to set a hint that this repo is a submodule.\n>>> +\t */\n>>> +\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n>>> +\n>>> \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n>>> \t\treturn do_run_update_procedure(&update_data);\n>>\n>> In Glen's update to rewrite \"submodule update\" in C, this part is\n>> replaced with a call to update_submodule2(). I am not sure what the\n>> current repository is at this point of the code with and without\n>> Glen's topic, but are we sure we are in a submodule we discovered?\n>\n> Rereading this, I realize you probably meant that this conflicts with\n> part1, not part2...\n>\n> At the end of part1, update_submodule2() is called from inside the\n> submodule (specifically from run_update_procedure()). So a good merge\n> conflict resolution would be to set the config _before_ calling\n> update_submodule2(). e.g.\n>\n> ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----\n> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> index bef9ab22d4..f53808d995 100644\n> --- a/builtin/submodule--helper.c\n> +++ b/builtin/submodule--helper.c\n> @@ -2672,6 +2677,11 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n> &update_data.update_strategy);\n>\n> free(prefixed_path);\n> + /*\n> + * This entry point is always called from a submodule, so this is a\n> + * good place to set a hint that this repo is a submodule.\n> + */\n> + git_config_set(\"submodule.hasSuperproject\", \"true\");\n> return update_submodule2(&update_data);\n> }\n\nThat matched my tentative resolution I made last night, but what do\nyou think about this part of the test added by the patch?\n\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex 11cccbb333..ec2397fc69 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n \t)\n '\n \n+test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n+\t(cd super &&\n+\t test_unconfig submodule.hasSuperproject &&\n+\t git submodule update &&\n+\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n+\t)\n+'\n+\n test_done\n\nWe go to \"super\", make sure that superproject does not have\nsubmodule.hasSuperproject set, run \"git submodule update\", and see\nif the configuration file in \"submodule\" subdirectory has the\nvariable set. It does not clear the variable from the submodule\nbefore starting, so the variable given to the submodule when it was\ncloned would be there, even if \"git submodule update\" failed to set\nit.\n\nI am wondering if it should do something like the attached instead.\n\nWe\n\n * clear the variable from \"super\" and \"super/submodule\"\n repositories;\n\n * run \"git submodule update\";\n\n * ensure that \"git submodule update\" did not touch \"super/.git/config\";\n\n * ensure that \"git submodule update\" added the variable to\n \"super/submodule/.git/config\".\n\nClearing the variable from \"super\" is technically wrong because the\nrepository is set up as a submodule of \"recursivesuper\" and if we\nhad further tests, we should restore it in \"super\", but the point is\nthat we are makng sure \"git submodule update\" sets the variable in\nthe configuration file of the submodule, and not in the superproject's. \n\nWith the conflict resolution above, this \"corrected\" test fails and\nshows that superproject's configuration file is updated after \"git\nsubmodule update\".\n\nThis series alone, without your topic, this \"corrected\" test fails,\nand that is where my \"are we sure we are mucking with the\nconfiguration file in the submodule\"? comes from.\n\ndiff --git c/t/t7406-submodule-update.sh w/t/t7406-submodule-update.sh\nindex 000e055811..c9912bb242 100755\n--- c/t/t7406-submodule-update.sh\n+++ w/t/t7406-submodule-update.sh\n@@ -1083,4 +1083,16 @@ test_expect_success 'submodule update --filter sets partial clone settings' '\n \ttest_cmp_config -C super-filter/submodule blob:none remote.origin.partialclonefilter\n '\n \n+test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n+\t(cd super &&\n+\t test_unconfig submodule.hasSuperproject &&\n+\t test_unconfig -C submodule submodule.hasSuperproject &&\n+\t git submodule update &&\n+\t echo in super &&\n+\t test_cmp_config false --type=bool submodule.hasSuperproject &&\n+\t echo in submodule &&\n+\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n+\t)\n+'\n+\n test_done\n"},{"id":"451101","messageId":"kl6l4k45s7cb.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56920","inReplyTo":"xmqqwnh1bgr4.fsf@gitster.g","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-03-10T23:42:12Z","receivedAt":"2022-03-10T23:42:18Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Junio C Hamano writes:\n\n>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n>> index bef9ab22d4..f53808d995 100644\n>> --- a/builtin/submodule--helper.c\n>> +++ b/builtin/submodule--helper.c\n>> @@ -2672,6 +2677,11 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n>> &update_data.update_strategy);\n>>\n>> free(prefixed_path);\n>> + /*\n>> + * This entry point is always called from a submodule, so this is a\n>> + * good place to set a hint that this repo is a submodule.\n>> + */\n>> + git_config_set(\"submodule.hasSuperproject\", \"true\");\n>> return update_submodule2(&update_data);\n>> }\n>\n> That matched my tentative resolution I made last night, but what do\n> you think about this part of the test added by the patch?\n>\n> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\n> index 11cccbb333..ec2397fc69 100755\n> --- a/t/t7406-submodule-update.sh\n> +++ b/t/t7406-submodule-update.sh\n> @@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n> \t)\n> '\n> \n> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n> +\t(cd super &&\n> +\t test_unconfig submodule.hasSuperproject &&\n> +\t git submodule update &&\n> +\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n> +\t)\n> +'\n> +\n> test_done\n>\n> We go to \"super\", make sure that superproject does not have\n> submodule.hasSuperproject set, run \"git submodule update\", and see\n> if the configuration file in \"submodule\" subdirectory has the\n> variable set. It does not clear the variable from the submodule\n> before starting, so the variable given to the submodule when it was\n> cloned would be there, even if \"git submodule update\" failed to set\n> it.\n>\n> I am wondering if it should do something like the attached instead.\n>\n> We\n>\n> * clear the variable from \"super\" and \"super/submodule\"\n> repositories;\n>\n> * run \"git submodule update\";\n>\n> * ensure that \"git submodule update\" did not touch \"super/.git/config\";\n>\n> * ensure that \"git submodule update\" added the variable to\n> \"super/submodule/.git/config\".\n>\n> Clearing the variable from \"super\" is technically wrong because the\n> repository is set up as a submodule of \"recursivesuper\" and if we\n> had further tests, we should restore it in \"super\", but the point is\n> that we are makng sure \"git submodule update\" sets the variable in\n> the configuration file of the submodule, and not in the superproject's. \n\nYes, the test you've described is closer to what I thought the original\ntest was trying to do. Seeing this test pass gave me a false sense of\nconfidence hm..\n\n> With the conflict resolution above, this \"corrected\" test fails and\n> shows that superproject's configuration file is updated after \"git\n> submodule update\".\n>\n> This series alone, without your topic, this \"corrected\" test fails,\n> and that is where my \"are we sure we are mucking with the\n> configuration file in the submodule\"? comes from.\n\nYeah looks like we aren't in the submodule after all:\n\n\t\tout=$(git submodule--helper run-update-procedure \\\n\t\t\t ${wt_prefix:+--prefix \"$wt_prefix\"} \\\n\t\t\t ${GIT_QUIET:+--quiet} \\\n\t\t\t ${force:+--force} \\\n\t\t\t ${just_cloned:+--just-cloned} \\\n\t\t\t ${nofetch:+--no-fetch} \\\n\t\t\t ${depth:+\"$depth\"} \\\n\t\t\t ${update:+--update \"$update\"} \\\n\t\t\t ${prefix:+--recursive-prefix \"$prefix\"} \\\n\t\t\t ${sha1:+--oid \"$sha1\"} \\\n\t\t\t ${subsha1:+--suboid \"$subsha1\"} \\\n\t\t\t \"--\" \\\n\t\t\t \"$sm_path\")\n\nThis says \"do the update at this submodule path\", but this is being run\nfrom the superproject.\n\nSo I suppose the way forward is one of the following:\n\n- Revert my original suggestion\n- Revert my original suggestion AND remove the git_config_set from\n \"module_clone()\" (before this, we unconditionally set this value in\n git-submodule.sh anyway)\n- Set the config in the submodule even though we are running from the\n superproject (this is possible, ensure_core_worktree() does this).\n\nIn any case, sorry for the faulty suggestion :(\n"},{"id":"451102","messageId":"kl6l1qz9s6tu.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56920","inReplyTo":"kl6l4k45s7cb.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-03-10T23:53:17Z","receivedAt":"2022-03-10T23:53:35Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Glen Choo writes:\n\n> Junio C Hamano writes:\n>\n>>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n>>> index bef9ab22d4..f53808d995 100644\n>>> --- a/builtin/submodule--helper.c\n>>> +++ b/builtin/submodule--helper.c\n>>> @@ -2672,6 +2677,11 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n>>> &update_data.update_strategy);\n>>>\n>>> free(prefixed_path);\n>>> + /*\n>>> + * This entry point is always called from a submodule, so this is a\n>>> + * good place to set a hint that this repo is a submodule.\n>>> + */\n>>> + git_config_set(\"submodule.hasSuperproject\", \"true\");\n>>> return update_submodule2(&update_data);\n>>> }\n>>\n>> That matched my tentative resolution I made last night, but what do\n>> you think about this part of the test added by the patch?\n>>\n>> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\n>> index 11cccbb333..ec2397fc69 100755\n>> --- a/t/t7406-submodule-update.sh\n>> +++ b/t/t7406-submodule-update.sh\n>> @@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n>> \t)\n>> '\n>> \n>> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n>> +\t(cd super &&\n>> +\t test_unconfig submodule.hasSuperproject &&\n>> +\t git submodule update &&\n>> +\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n>> +\t)\n>> +'\n>> +\n>> test_done\n>>\n>> We go to \"super\", make sure that superproject does not have\n>> submodule.hasSuperproject set, run \"git submodule update\", and see\n>> if the configuration file in \"submodule\" subdirectory has the\n>> variable set. It does not clear the variable from the submodule\n>> before starting, so the variable given to the submodule when it was\n>> cloned would be there, even if \"git submodule update\" failed to set\n>> it.\n>>\n>> I am wondering if it should do something like the attached instead.\n>>\n>> We\n>>\n>> * clear the variable from \"super\" and \"super/submodule\"\n>> repositories;\n>>\n>> * run \"git submodule update\";\n>>\n>> * ensure that \"git submodule update\" did not touch \"super/.git/config\";\n>>\n>> * ensure that \"git submodule update\" added the variable to\n>> \"super/submodule/.git/config\".\n>>\n>> Clearing the variable from \"super\" is technically wrong because the\n>> repository is set up as a submodule of \"recursivesuper\" and if we\n>> had further tests, we should restore it in \"super\", but the point is\n>> that we are makng sure \"git submodule update\" sets the variable in\n>> the configuration file of the submodule, and not in the superproject's. \n>\n> Yes, the test you've described is closer to what I thought the original\n> test was trying to do. Seeing this test pass gave me a false sense of\n> confidence hm..\n\nCorrection, seeing the _original_ test pass gave me false sense of\nconfidence.\n\n>> With the conflict resolution above, this \"corrected\" test fails and\n>> shows that superproject's configuration file is updated after \"git\n>> submodule update\".\n>>\n>> This series alone, without your topic, this \"corrected\" test fails,\n>> and that is where my \"are we sure we are mucking with the\n>> configuration file in the submodule\"? comes from.\n> - Set the config in the submodule even though we are running from the\n> superproject (this is possible, ensure_core_worktree() does this).\n\nIf it helps, I was able to do this up by copying\nensure_core_worktree(), and this passes the amended test.\n\n----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 4d02dd05ca..3bb7a65762 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -1838,11 +1838,6 @@ static int clone_submodule(struct module_clone_data *clone_data)\n git_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n error_strategy);\n\n-\t/*\n-\t * Teach the submodule that it's a submodule.\n-\t */\n-\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n-\n free(sm_alternate);\n free(error_strategy);\n\n@@ -2560,6 +2555,20 @@ static int update_clone(int argc, const char **argv, const char *prefix)\n return update_submodules(&suc);\n}\n\n+static void set_hassuperproject(const char *sm_path)\n+{\n+\tstruct repository subrepo;\n+\tchar *cfg_file;\n+\n+\tif (repo_submodule_init(&subrepo, the_repository, sm_path, null_oid()))\n+\t\tdie(_(\"could not get a repository handle for submodule '%s'\"), sm_path);\n+\n+\tcfg_file = repo_git_path(&subrepo, \"config\");\n+\tgit_config_set_in_file(cfg_file, \"submodule.hasSuperproject\", \"true\");\n+\n+\tfree(cfg_file);\n+}\n+\nstatic int run_update_procedure(int argc, const char **argv, const char *prefix)\n{\n int force = 0, quiet = 0, nofetch = 0, just_cloned = 0;\n@@ -2622,10 +2631,9 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n free(prefixed_path);\n\n /*\n-\t * This entry point is always called from a submodule, so this is a\n-\t * good place to set a hint that this repo is a submodule.\n+\t * Teach the submodule that it's a submodule.\n */\n-\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n+\tset_hassuperproject(update_data.sm_path);\n\n if (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n return do_run_update_procedure(&update_data);\n"},{"id":"451118","messageId":"220311.8635joj0lf.gmgdl@evledraar.gmail.com","threadId":"56920","inReplyTo":"20220310004423.2627181-1-emilyshaffer@google.com","subject":"Re: [PATCH v9 0/3] teach submodules to know they're submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-03-11T09:09:50Z","receivedAt":"2022-03-11T09:32:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Mar 09 2022, Emily Shaffer wrote:\n\n> For the original cover letter, see\n> https://lore.kernel.org/git/20210611225428.1208973-1-emilyshaffer%40google.com.\n>\n> CI run: https://github.com/nasamuffin/git/actions/runs/1954710601\n>\n> Since v8:\n>\n> Only a couple of minor fixes.\n>\n> Junio pointed out that I could write the tests better using --type=bool\n> and 'test_cmp_config', and that we could be a little more careful about\n> when to give up on 'git rev-parse --show-superproject-working-dir'.\n>\n> Glen mentioned that builtin/submodule--helper.c:run_update_procedure() is called\n> unconditionally earlier in the same function where I had added the\n> config in git-submodule.sh. So, I moved the config set into\n> submodule--helper.c to reduce possible edge cases where the config might\n> not be set.\n>\n> Otherwise, this series is pretty much unchanged.\n>\n> Since v7:\n>\n> Actually a fairly large rework. Rather than keeping the path from gitdir\n> to gitdir, just keep a boolean under 'submodule.hasSuperproject'. The\n> idea is that from this boolean, we can decide whether to traverse the\n> filesystem looking for a superproject.\n>\n> Because this simplifies the implementation, I compressed the three\n> middle commits into one. As proof-of-concept, I added a patch at the end\n> to check for this boolean when running `git rev-parse\n> --show-superproject-working-tree`.\n>\n> One thing I'm not sure about: in the tests, I check whether the config\n> is set, but not what the boolean value of it is. Is there a better way\n> to do that? For example, I could imagine someone deciding to set\n> `submodule.hasSuperproject = false` and the tests would not function\n> correctly in that case. I think we don't really normalize the value on a\n> boolean config like that, so I didn't want to write a lot of comparison\n> to check if the value is 1 or true or True or TRUE or Yes or .... Am I\n> overthinking it?\n>\n> The other thing I'm not sure about: since it's just a bool, we're not\n> restricted to setting this config only when we have both gitdir paths\n> available. That makes me want to set the config any time we are doing\n> something with submodules anyway, like any time 'git-submodule--helper'\n> is used. But that helper seems to be called in the context of the\n> superproject, not of the submodules, so adding this config for each\n> submodule we touch would be a second child process. Is there some other\n> common entry point for submodules that we can use?\n\nI really don't mean to bring up the same points again, but I'm still\ngenuinely unsure what this is intended to solve in the end.\n\nI.e. from the original RFC we went from it being for optimizations for\nthe shellscript \"git rev-parse\", to suggestions that the configured path\nwould be \"canonical\" in a way we couldn't discover on-the-fly (i.e. some\nof Jonathan's noted edge cases [1]).\n\nBut now it's a boolean indicating \"it's there, discover it\", and the\nimplied (but not really explicitly stated) reason in 2/3 is that it's\npurely for optimization purposes at this point.\n\nBut it's an optimization without a benchmark.\n\nIn [1] Jonathan (if I understood it correctly, see [2]) might have\nsuggested this is important to deal with some Google in-house NFS-a-like\nauto-mounting software, i.e. the \"walking up\" is truly expensive in some\nscenarios.\n\nI do worry a bit that we'll be creating behavior edge cases related to\nthis, and if the problem being solved is for a relatively obscure setup\nis it worth it, and in that case perhaps there should be a \"I need this\noptimization\" setting guarding it?\n\nBut I don't know, a concrete case where this series makes a difference\nwould really help.\n\nI tried to come up with one before[3] and all I could find was fleeting\ncases we'd see go away with the migration of the remaining parts of\ngit-submodule.sh to C, which we already have in-flight patches for (or\nrather, Glen is AFAIK at series 1/2 of submitting those, with 1/2\nin-flight).\n\nIn any case I think lifting the bits of [3] where we assert that this\ndoesn't introduce any behavior change with a GIT_TEST_* knob would be\nvaluable.\n\nI.e. as long a the intent isn't a behavior change let's test that\nget_superproject_working_tree() doesn't need this across the entire test\nsuite, with specific tests that opt-in to the behavior (or do a whole\ntest suite run in that mode), rather than the default being\nopt-out.\n\nAn opt-out is just a recipe for growing accidental implicit\ndependencies, which explicitly isn't what we want for a \"just an\noptimization\" knob. We do the same sort of opt-in/out-out testing for\ne.g. split index, untracked cache etc (see the GIT_TEST_* bits in\nci/run-build-and-tests.sh). AFAICT a fix-up of just adding the\ngit_env_bool() here to this code in your 3/3 would do it:\n\n\tif (!git_env_bool(\"GIT_TEST_NO_SUBMODULE_HAS_SUPERPROJECT\", 0) &&\n\t !git_config_get_bool(\"submodule.hassuperproject\", &has_superproject_cfg)\n\t && !has_superproject_cfg)\n\nAnd then adding GIT_TEST_NO_SUBMODULE_HAS_SUPERPROJECT=true to\nlinux-TEST-vars in ci/run-build-and-tests.sh. The tests that do rely on\nsubmodule.hassuperproject would need to set\nGIT_TEST_NO_SUBMODULE_HAS_SUPERPROJECT=false of course...\n\n1. https://lore.kernel.org/git/YgF5V2Y0Btr8B4cd@google.com/\n2. https://lore.kernel.org/git/220212.864k53yfws.gmgdl@evledraar.gmail.com/\n3. https://lore.kernel.org/git/RFC-cover-0.2-00000000000-20211117T113134Z-avarab@gmail.com/\n"},{"id":"451233","messageId":"xmqqpmmql860.fsf@gitster.g","threadId":"56920","inReplyTo":"220311.8635joj0lf.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v9 0/3] teach submodules to know they're submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-13T05:43:03Z","receivedAt":"2022-03-13T05:43:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> But now it's a boolean indicating \"it's there, discover it\", and the\n> implied (but not really explicitly stated) reason in 2/3 is that it's\n> purely for optimization purposes at this point.\n\nYou may know that I have a separate checkout of the 'todo' branch at\npath \"Meta\" in my working tree.\n\nI could use the hasSuperproject=false setting there, to say \"this is\n*NOT* a submodule, even the parent directory is a working tree of a\ndifferent repository, it is not our superproject, so do *NOT* bother\nto go up to discover anything\".\n\nIf that configuration weren't there in the \"Meta/.git/config\", the\nparent directory of \"Meta\" (which has its own \".git\") cannot tell if\nthat \"Meta\" thing is a submodule being prepared that hasn't been\nadded yet, or it will never intended to be a submodule. I would\nimagine that \"git add X\" can later be taught to refuse to add X if\nthere is X/.git and X/.git/config says it explicitly says that it\ndoes not have a superproject.\n\nSo, I am not sure if it is a good characterization that it is for\noptimization at all.\n"},{"id":"451410","messageId":"YjDaeupPmWSp9u9w@google.com","threadId":"56920","inReplyTo":"kl6l7d91sccb.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-15T18:27:06Z","receivedAt":"2022-03-15T18:27:15Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Mar 10, 2022 at 01:54:12PM -0800, Glen Choo wrote:\n> \n> Emily Shaffer writes:\n> \n> > diff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt\n> > index ee454f8126..99d5260b8e 100644\n> > diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> > index c5d3fc3817..eda9ed550e 100644\n> > --- a/builtin/submodule--helper.c\n> > +++ b/builtin/submodule--helper.c\n> > @@ -1839,6 +1839,11 @@ static int clone_submodule(struct module_clone_data *clone_data)\n> > \t\tgit_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n> > \t\t\t\t error_strategy);\n> > \n> > +\t/*\n> > +\t * Teach the submodule that it's a submodule.\n> > +\t */\n> > +\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n> > +\n> > \tfree(sm_alternate);\n> > \tfree(error_strategy);\n> \n> This git_config_set_* is superfluous - it sets the config in newly\n> cloned submodules..\n> \n> > \n> > @@ -2617,6 +2622,12 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n> > \n> > \tfree(prefixed_path);\n> > \n> > +\t/*\n> > +\t * This entry point is always called from a submodule, so this is a\n> > +\t * good place to set a hint that this repo is a submodule.\n> > +\t */\n> > +\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n> > +\n> > \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n> > \t\treturn do_run_update_procedure(&update_data);\n> > \n> \n> but this is called over *all* submodules, so we're guaranteed to always\n> set the config if \"git submodule update\" isn't interrupted halfway.\n> \n> I don't think we guarantee correctness if it is interrupted halfway e.g.\n> core.worktree can be unset if it is interrupted halfway (because\n> ensure-core-worktree is called adjacent to run-update-procedure, not\n> inside of update-clone).\n> \n> So I think it's better to just drop the previous hunk - it will\n> disappear anyway in gc/submodule-update-part2.\n\nAh, this makes sense. Sure, will do.\n\n - Emily\n\n"},{"id":"451412","messageId":"YjDdRiRvvDtKZyq4@google.com","threadId":"56920","inReplyTo":"xmqqwnh1bgr4.fsf@gitster.g","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-15T18:39:02Z","receivedAt":"2022-03-15T18:39:12Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Mar 10, 2022 at 02:10:55PM -0800, Junio C Hamano wrote:\n> \n> Glen Choo writes:\n> \n> > Junio C Hamano writes:\n> >\n> >> Emily Shaffer writes:\n> >>\n> >>> @@ -2617,6 +2622,12 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n> >>> \n> >>> \tfree(prefixed_path);\n> >>> \n> >>> +\t/*\n> >>> +\t * This entry point is always called from a submodule, so this is a\n> >>> +\t * good place to set a hint that this repo is a submodule.\n> >>> +\t */\n> >>> +\tgit_config_set(\"submodule.hasSuperproject\", \"true\");\n> >>> +\n> >>> \tif (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)\n> >>> \t\treturn do_run_update_procedure(&update_data);\n> >>\n> >> In Glen's update to rewrite \"submodule update\" in C, this part is\n> >> replaced with a call to update_submodule2(). I am not sure what the\n> >> current repository is at this point of the code with and without\n> >> Glen's topic, but are we sure we are in a submodule we discovered?\n> >\n> > Rereading this, I realize you probably meant that this conflicts with\n> > part1, not part2...\n> >\n> > At the end of part1, update_submodule2() is called from inside the\n> > submodule (specifically from run_update_procedure()). So a good merge\n> > conflict resolution would be to set the config _before_ calling\n> > update_submodule2(). e.g.\n> >\n> > ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----\n> > diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> > index bef9ab22d4..f53808d995 100644\n> > --- a/builtin/submodule--helper.c\n> > +++ b/builtin/submodule--helper.c\n> > @@ -2672,6 +2677,11 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n> > &update_data.update_strategy);\n> >\n> > free(prefixed_path);\n> > + /*\n> > + * This entry point is always called from a submodule, so this is a\n> > + * good place to set a hint that this repo is a submodule.\n> > + */\n> > + git_config_set(\"submodule.hasSuperproject\", \"true\");\n> > return update_submodule2(&update_data);\n> > }\n> \n> That matched my tentative resolution I made last night, but what do\n> you think about this part of the test added by the patch?\n> \n> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\n> index 11cccbb333..ec2397fc69 100755\n> --- a/t/t7406-submodule-update.sh\n> +++ b/t/t7406-submodule-update.sh\n> @@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n> \t)\n> '\n> \n> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n> +\t(cd super &&\n> +\t test_unconfig submodule.hasSuperproject &&\n> +\t git submodule update &&\n> +\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n> +\t)\n> +'\n> +\n> test_done\n> \n> We go to \"super\", make sure that superproject does not have\n> submodule.hasSuperproject set, run \"git submodule update\", and see\n> if the configuration file in \"submodule\" subdirectory has the\n> variable set. It does not clear the variable from the submodule\n> before starting, so the variable given to the submodule when it was\n> cloned would be there, even if \"git submodule update\" failed to set\n> it.\n> \n> I am wondering if it should do something like the attached instead.\n> \n> We\n> \n> * clear the variable from \"super\" and \"super/submodule\"\n> repositories;\n> \n> * run \"git submodule update\";\n> \n> * ensure that \"git submodule update\" did not touch \"super/.git/config\";\n\nYeah, this is a good idea, and indeed when I add this step the bug\npointed out downthread becomes clear. Thanks.\n\n> \n> * ensure that \"git submodule update\" added the variable to\n> \"super/submodule/.git/config\".\n> \n> Clearing the variable from \"super\" is technically wrong because the\n> repository is set up as a submodule of \"recursivesuper\" and if we\n> had further tests, we should restore it in \"super\", but the point is\n> that we are makng sure \"git submodule update\" sets the variable in\n> the configuration file of the submodule, and not in the superproject's. \n> \n> With the conflict resolution above, this \"corrected\" test fails and\n> shows that superproject's configuration file is updated after \"git\n> submodule update\".\n> \n> This series alone, without your topic, this \"corrected\" test fails,\n> and that is where my \"are we sure we are mucking with the\n> configuration file in the submodule\"? comes from.\n> \n> diff --git c/t/t7406-submodule-update.sh w/t/t7406-submodule-update.sh\n> index 000e055811..c9912bb242 100755\n> --- c/t/t7406-submodule-update.sh\n> +++ w/t/t7406-submodule-update.sh\n> @@ -1083,4 +1083,16 @@ test_expect_success 'submodule update --filter sets partial clone settings' '\n> \ttest_cmp_config -C super-filter/submodule blob:none remote.origin.partialclonefilter\n> '\n> \n> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n> +\t(cd super &&\n> +\t test_unconfig submodule.hasSuperproject &&\n> +\t test_unconfig -C submodule submodule.hasSuperproject &&\n> +\t git submodule update &&\n> +\t echo in super &&\n> +\t test_cmp_config false --type=bool submodule.hasSuperproject &&\n> +\t echo in submodule &&\n> +\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n> +\t)\n> +'\n> +\n> test_done\n"},{"id":"451418","messageId":"xmqq5yofdnws.fsf@gitster.g","threadId":"56920","inReplyTo":"YjDdRiRvvDtKZyq4@google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-03-15T19:19:15Z","receivedAt":"2022-03-15T19:19:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Emily Shaffer writes:\n\n>> Clearing the variable from \"super\" is technically wrong because the\n>> repository is set up as a submodule of \"recursivesuper\" and if we\n>> had further tests, we should restore it in \"super\", but the point is\n>> that we are makng sure \"git submodule update\" sets the variable in\n>> the configuration file of the submodule, and not in the superproject's. \n\nIf we wanted to be kosher about this, we could start the test with\n\n git config submodule.hassuperproject 1\n\nin the \"super\" repository, clear the variable in the \"submodule\"\nrepository, before running the \"git submodule update\" step, which\n(1) should not touch the \"super\" configuration and (2) should touch\nthe \"submodule\" configuration.\n\nIf we inspect in the \"super\" repository after \"submodule update\"\n\n value=$(git config submodule.hassuperproject) &&\n test \"$value\" = 1\n\nI think we can tell if a buggy \"submodule update\" overwrites the\n\"super\" configuration from \"1\" to \"true\". And downstream tests\nwill take \"1\" as true just fine.\n\nAnd of course, in \"submodule\", the variable after \"submodule update\"\nmust be set to true, which can be checked with\n\n value=$(git -C submodule config --type=bool submodule.hassuperproject) &&\n test \"$value\" = true\n\nThe trick depends on the hardcoded value to represent \"true\" in the\ncode this patch adds, but that is the canonical way to spell true in\nthe config, according to \"git config --type=bool\", so the dependency\nmay not be too bad.\n\nJust a thought.\n\n\n"},{"id":"451428","messageId":"YjD7mWPEnm/0Evy1@google.com","threadId":"56920","inReplyTo":"kl6l1qz9s6tu.fsf@chooglen-macbookpro.roam.corp.google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-15T20:48:25Z","receivedAt":"2022-03-15T20:48:36Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Thu, Mar 10, 2022 at 03:53:17PM -0800, Glen Choo wrote:\n> \n> Glen Choo writes:\n> \n> > Junio C Hamano writes:\n> >\n> >>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> >>> index bef9ab22d4..f53808d995 100644\n> >>> --- a/builtin/submodule--helper.c\n> >>> +++ b/builtin/submodule--helper.c\n> >>> @@ -2672,6 +2677,11 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)\n> >>> &update_data.update_strategy);\n> >>>\n> >>> free(prefixed_path);\n> >>> + /*\n> >>> + * This entry point is always called from a submodule, so this is a\n> >>> + * good place to set a hint that this repo is a submodule.\n> >>> + */\n> >>> + git_config_set(\"submodule.hasSuperproject\", \"true\");\n> >>> return update_submodule2(&update_data);\n> >>> }\n> >>\n> >> That matched my tentative resolution I made last night, but what do\n> >> you think about this part of the test added by the patch?\n> >>\n> >> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\n> >> index 11cccbb333..ec2397fc69 100755\n> >> --- a/t/t7406-submodule-update.sh\n> >> +++ b/t/t7406-submodule-update.sh\n> >> @@ -1061,4 +1061,12 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s\n> >> \t)\n> >> '\n> >> \n> >> +test_expect_success 'submodule update adds submodule.hasSuperproject to older repos' '\n> >> +\t(cd super &&\n> >> +\t test_unconfig submodule.hasSuperproject &&\n> >> +\t git submodule update &&\n> >> +\t test_cmp_config -C submodule true --type=bool submodule.hasSuperproject\n> >> +\t)\n> >> +'\n> >> +\n> >> test_done\n> >>\n> >> We go to \"super\", make sure that superproject does not have\n> >> submodule.hasSuperproject set, run \"git submodule update\", and see\n> >> if the configuration file in \"submodule\" subdirectory has the\n> >> variable set. It does not clear the variable from the submodule\n> >> before starting, so the variable given to the submodule when it was\n> >> cloned would be there, even if \"git submodule update\" failed to set\n> >> it.\n> >>\n> >> I am wondering if it should do something like the attached instead.\n> >>\n> >> We\n> >>\n> >> * clear the variable from \"super\" and \"super/submodule\"\n> >> repositories;\n> >>\n> >> * run \"git submodule update\";\n> >>\n> >> * ensure that \"git submodule update\" did not touch \"super/.git/config\";\n> >>\n> >> * ensure that \"git submodule update\" added the variable to\n> >> \"super/submodule/.git/config\".\n> >>\n> >> Clearing the variable from \"super\" is technically wrong because the\n> >> repository is set up as a submodule of \"recursivesuper\" and if we\n> >> had further tests, we should restore it in \"super\", but the point is\n> >> that we are makng sure \"git submodule update\" sets the variable in\n> >> the configuration file of the submodule, and not in the superproject's. \n> >\n> > Yes, the test you've described is closer to what I thought the original\n> > test was trying to do. Seeing this test pass gave me a false sense of\n> > confidence hm..\n> \n> Correction, seeing the _original_ test pass gave me false sense of\n> confidence.\n> \n> >> With the conflict resolution above, this \"corrected\" test fails and\n> >> shows that superproject's configuration file is updated after \"git\n> >> submodule update\".\n> >>\n> >> This series alone, without your topic, this \"corrected\" test fails,\n> >> and that is where my \"are we sure we are mucking with the\n> >> configuration file in the submodule\"? comes from.\n> > - Set the config in the submodule even though we are running from the\n> > superproject (this is possible, ensure_core_worktree() does this).\n> \n> If it helps, I was able to do this up by copying\n> ensure_core_worktree(), and this passes the amended test.\n> \n> ----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----\n> \n> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> index 4d02dd05ca..3bb7a65762 100644\n> --- a/builtin/submodule--helper.c\n> +++ b/builtin/submodule--helper.c\n> @@ -1838,11 +1838,6 @@ static int clone_submodule(struct module_clone_data *clone_data)\n> git_config_set_in_file(p, \"submodule.alternateErrorStrategy\",\n> error_strategy);\n> \n> -\t/*\n> -\t * Teach the submodule that it's a submodule.\n> -\t */\n> -\tgit_config_set_in_file(p, \"submodule.hasSuperproject\", \"true\");\n> -\n> free(sm_alternate);\n> free(error_strategy);\n> \n> @@ -2560,6 +2555,20 @@ static int update_clone(int argc, const char **argv, const char *prefix)\n> return update_submodules(&suc);\n> }\n> \n> +static void set_hassuperproject(const char *sm_path)\n> +{\n> +\tstruct repository subrepo;\n> +\tchar *cfg_file;\n> +\n> +\tif (repo_submodule_init(&subrepo, the_repository, sm_path, null_oid()))\n> +\t\tdie(_(\"could not get a repository handle for submodule '%s'\"), sm_path);\n\nIsn't the repo_submodule_init() fairly expensive? I think this is doing\na whole repo_init() call we would not otherwise be doing.... Is it good\nenough to generate the config from sm_path, by using\nstrbuf_repo_worktree_path(), and simply be tolerant of the failure if\n/config doesn't exist?\n\nOtherwise, this is a good workaround I think. Thanks.\n\n - Emily\n"},{"id":"451429","messageId":"YjD9bgmy9ArubMoG@google.com","threadId":"56920","inReplyTo":"YjD7mWPEnm/0Evy1@google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Emily Shaffer","fromEmail":"emilyshaffer@google.com","sentAt":"2022-03-15T20:56:14Z","receivedAt":"2022-03-15T20:56:29Z","isPatch":true,"sender":{"key":"nasamuffin@google.com","avatar":"https://avatars.githubusercontent.com/u/1606826?v=4"},"body":"On Tue, Mar 15, 2022 at 01:48:25PM -0700, Emily Shaffer wrote:\n> > +static void set_hassuperproject(const char *sm_path)\n> > +{\n> > +\tstruct repository subrepo;\n> > +\tchar *cfg_file;\n> > +\n> > +\tif (repo_submodule_init(&subrepo, the_repository, sm_path, null_oid()))\n> > +\t\tdie(_(\"could not get a repository handle for submodule '%s'\"), sm_path);\n> \n> Isn't the repo_submodule_init() fairly expensive? I think this is doing\n> a whole repo_init() call we would not otherwise be doing.... Is it good\n> enough to generate the config from sm_path, by using\n> strbuf_repo_worktree_path(), and simply be tolerant of the failure if\n> /config doesn't exist?\n\nAh, I was misreading the implementation of repo_submodule_init() and I\nsee now that won't work. I guess it is fine to just invoke\nrepo_submodule_init() then, unless someone has another idea.\n"},{"id":"451439","messageId":"kl6lbky6ykuo.fsf@chooglen-macbookpro.roam.corp.google.com","threadId":"56920","inReplyTo":"YjD9bgmy9ArubMoG@google.com","subject":"Re: [PATCH v9 2/3] introduce submodule.hasSuperproject record","fromName":"Glen Choo","fromEmail":"chooglen@google.com","sentAt":"2022-03-15T21:19:43Z","receivedAt":"2022-03-15T21:19:48Z","isPatch":true,"sender":{"key":"glencbz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58092771?v=4"},"body":"Emily Shaffer writes:\n\n> On Tue, Mar 15, 2022 at 01:48:25PM -0700, Emily Shaffer wrote:\n>> > +static void set_hassuperproject(const char *sm_path)\n>> > +{\n>> > +\tstruct repository subrepo;\n>> > +\tchar *cfg_file;\n>> > +\n>> > +\tif (repo_submodule_init(&subrepo, the_repository, sm_path, null_oid()))\n>> > +\t\tdie(_(\"could not get a repository handle for submodule '%s'\"), sm_path);\n>> \n>> Isn't the repo_submodule_init() fairly expensive? I think this is doing\n>> a whole repo_init() call we would not otherwise be doing.... Is it good\n>> enough to generate the config from sm_path, by using\n>> strbuf_repo_worktree_path(), and simply be tolerant of the failure if\n>> /config doesn't exist?\n>\n> Ah, I was misreading the implementation of repo_submodule_init() and I\n> see now that won't work. I guess it is fine to just invoke\n> repo_submodule_init() then, unless someone has another idea.\n\nYes, it's difficult to avoid calling repo_submodule_init() because it's\nhard to get the gitdir using just the path to the submodule in the\nworking tree (sm_path).\n\nAre we particular about avoiding calls to repo_submodule_init()? I don't\nrecall hearing this as an objection before. If so, I'll keep this in\nmind as I work on more submodule things.\n\nAs an aside, ensure_core_worktree() already calls repo_submodule_init(),\nso this wouldn't be the first time \"submodule update\" calls\nrepo_submodule_init(), and a potential optimization might be to cache\nthe result in between invocations of repo_submodule_init().\n"}]}