{"thread":{"id":"35154","subject":"[git-users] Problem using detached worktrees with commands implemented in scripts","startedAt":"2013-10-16T20:03:30Z","lastAt":"2013-10-21T18:51:24Z","messageCount":17,"participants":["Dale R. Worley","Junio C Hamano","Philip Oakley","Jonathan Nieder"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"229085","messageId":"201310162003.r9GK3UYj014414@freeze.ariadne.com","threadId":"35154","inReplyTo":null,"subject":"[git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Dale R. Worley","fromEmail":"worley@alum.mit.edu","sentAt":"2013-10-16T20:03:30Z","receivedAt":"2013-10-16T20:03:30Z","isPatch":false,"sender":{"key":"worley@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/19911107?v=4"},"body":"In Git, one can set up a repository with a \"detached worktree\", where\nthe .git directory is not a subdirectory of the top directory of the\nwork tree.\n\nIn general, Git commands on a repository with a detached worktree can\nbe executed by cd'ing into the directory containing the .git\ndirectory, and executing the Git command there. E.g., \"git add\" and\n\"git commit\" execute as one would expect. (I think they can also be\nexecuted by cd'ing to the worktree and setting GIT_DIR.)\n\nHowever, this approach does not work with \"git filter-branch\", which\nobjects with \"You need to run this command from the toplevel of the\nworking tree.\"\n\nI suspect that it does not work with other Git commands that are\nimplemented with shell scripts. The problem appears to be in the\ngit-sh-setup script, which is called by the Git shell scripts to set\nup the environment and do preliminary tests.\n\nIt seems to me that this inconsistency between the script commands and\nthe binary commands can be fixed by updating git-sh-setup in this way:\n\n--- git-sh-setup.Custom.orig\t2013-06-20 12:59:45.000000000 -0400\n+++ git-sh-setup\t2013-10-07 22:34:06.719946134 -0400\n@@ -297,14 +297,18 @@\n # if we require to be in a git repository.\n if test -z \"$NONGIT_OK\"\n then\n-\tGIT_DIR=$(git rev-parse --git-dir) || exit\n+\texport GIT_DIR=$(git rev-parse --git-dir) || exit\n \tif [ -z \"$SUBDIRECTORY_OK\" ]\n \tthen\n-\t\ttest -z \"$(git rev-parse --show-cdup)\" || {\n-\t\t\texit=$?\n-\t\t\techo >&2 \"You need to run this command from the toplevel of the working tree.\"\n-\t\t\texit $exit\n-\t\t}\n+\t\tcdup=\"$(git rev-parse --show-cdup)\"\n+\t\tif [ -n \"$cdup\" ]\n+\t\tthen\n+\t\t\t# Current directory is not the toplevel.\n+\t\t\t# Set GIT_DIR to the absolute path of the repository.\n+\t\t\tGIT_DIR=$(cd \"$GIT_DIR\" && pwd)\n+\t\t\t# cd to the toplevel.\n+\t\t\tcd $cdup\n+\t\tfi\n \tfi\n \ttest -n \"$GIT_DIR\" && GIT_DIR=$(cd \"$GIT_DIR\" && pwd) || {\n \t\techo >&2 \"Unable to determine absolute path of git directory\"\n\nWhat this change does is, when a command is invoked from a directory\ncontaining a repository with a detached worktree, is to set GIT_DIR to\nthe directory of the repository, then cd to the top of the worktree.\nAfter that, the script command should work as expected.\n\nI am far from being an expert in Git internals, so I don't know\nwhether this is the correct approach to take to this problem or not.\n\nDoes anyone have any feedback on this?\n\nDale\n"},{"id":"229097","messageId":"xmqqeh7k51vg.fsf@gitster.dls.corp.google.com","threadId":"35154","inReplyTo":"201310162003.r9GK3UYj014414@freeze.ariadne.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-10-16T21:42:27Z","receivedAt":"2013-10-16T21:42:27Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"worley@alum.mit.edu (Dale R. Worley) writes:\n\n> In general, Git commands on a repository with a detached worktree can\n> be executed by cd'ing into the directory containing the .git\n> directory, ...\n\nEh? News to me; it might happened to have appeared to work by\naccident, but that is not by design.\n\nIIRC, the intended use pattern (i.e. the change that introduced\nGIT_DIR and GIT_WORK_TREE environment variables was designed to\nsupport) for such a working tree is to:\n\n - export GIT_DIR that points at the correct .git directory;\n\n - export GIT_WORK_TREE that points at the correct top-level of such\n a working tree; and then\n\n - run the commands anywhere in the working tree, as if you did not\n export these two environment variables and instead had the .git\n directory at the usual place in the working tree.\n\nIt _is_ possible that we may have broken this canonical use pattern\nover time with more recent updates; I do not think we have extensive\ntest coverage for \"detached worktree\" use case in the first place.\n\n> Does anyone have any feedback on this?\n\nNot exporting GIT_DIR variable in sh-setup was done not by accident\nbut as a very deliberate design choice, IIRC.\n"},{"id":"229105","messageId":"29AA597BEBC146B09E8B370949EC2CE9@PhilipOakley","threadId":"35154","inReplyTo":"xmqqeh7k51vg.fsf@gitster.dls.corp.google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2013-10-16T22:39:25Z","receivedAt":"2013-10-16T22:39:25Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Junio C Hamano\" \n> worley@alum.mit.edu (Dale R. Worley) writes:\n>\n>> In general, Git commands on a repository with a detached worktree can\n>> be executed by cd'ing into the directory containing the .git\n>> directory, ...\n>\n> Eh? News to me; it might happened to have appeared to work by\n> accident, but that is not by design.\n\nI think it is this part in Dale's original email\n\n\"However, this approach does not work with \"git filter-branch\", which\nobjects with \"You need to run this command from the toplevel of the\nworking tree.\"\n\nthat is the problem Dale has seen. IIRC there are a few commands that do \nrequire to be run from the toplevel ('git bisect' I think is another), \nand the detection process for 'toplevel' may not work properly when in a \nseparated work-tree environment.\n\nPerhaps something to consider.\n\nPhilip\n\n>\n> IIRC, the intended use pattern (i.e. the change that introduced\n> GIT_DIR and GIT_WORK_TREE environment variables was designed to\n> support) for such a working tree is to:\n>\n> - export GIT_DIR that points at the correct .git directory;\n>\n> - export GIT_WORK_TREE that points at the correct top-level of such\n> a working tree; and then\n>\n> - run the commands anywhere in the working tree, as if you did not\n> export these two environment variables and instead had the .git\n> directory at the usual place in the working tree.\n>\n> It _is_ possible that we may have broken this canonical use pattern\n> over time with more recent updates; I do not think we have extensive\n> test coverage for \"detached worktree\" use case in the first place.\n>\n>> Does anyone have any feedback on this?\n>\n> Not exporting GIT_DIR variable in sh-setup was done not by accident\n> but as a very deliberate design choice, IIRC.\n> --\n"},{"id":"229107","messageId":"xmqqk3hc3jbw.fsf@gitster.dls.corp.google.com","threadId":"35154","inReplyTo":"29AA597BEBC146B09E8B370949EC2CE9@PhilipOakley","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-10-16T23:08:19Z","receivedAt":"2013-10-16T23:08:19Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Philip Oakley\" writes:\n\n> ... and the detection process for 'toplevel' may not work\n> properly when in a separated work-tree environment.\n\nWithout GIT_WORK_TREE exported to point at the top-level, there is\nnothing that lets us \"detect\" it, as the working tree does not have\n\".git\" directory to tell us to stop, no?\n"},{"id":"229127","messageId":"201310171909.r9HJ9mxd007908@freeze.ariadne.com","threadId":"35154","inReplyTo":"xmqqeh7k51vg.fsf@gitster.dls.corp.google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Dale R. Worley","fromEmail":"worley@alum.mit.edu","sentAt":"2013-10-17T19:09:48Z","receivedAt":"2013-10-17T19:09:48Z","isPatch":false,"sender":{"key":"worley@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/19911107?v=4"},"body":"> From: Junio C Hamano \n> \n> worley@alum.mit.edu (Dale R. Worley) writes:\n> \n> > In general, Git commands on a repository with a detached worktree can\n> > be executed by cd'ing into the directory containing the .git\n> > directory, ...\n> \n> Eh? News to me; it might happened to have appeared to work by\n> accident, but that is not by design.\n\nI must admit I've never seen the design (and I personally doubt that\nthe design has ever been written down). But at least the following\ncommands work correctly on a detached worktree if the current\ndirectory contains the .git directory, because I am using them in a\nproduction manner:\n\n git add\n git cat-file\n git commit\n git commit-tree\n git config\n git gc\n git log\n git ls-tree\n git reset\n git rev-list\n git update-ref\n\nIn my situation, the worktree is not, in my mind, dependent on the\nrepository; the repository is intended to keep backups of the contents\nof the directories that are worktree. Indeed, one could establish\nseveral detached repositories to back up different subsets of the same\nworktree. So it is conceptually natural to execute Git in the\nrepository directory. And, after all, the current directory\nidentifies the repository and the repository contains a pointer to the\nworktree.\n\n> Not exporting GIT_DIR variable in sh-setup was done not by accident\n> but as a very deliberate design choice, IIRC.\n\nThe intention of my change is that it appears that all of the failures\nof my use pattern are when the command is implemented by a shell\nscript, and it appears that all shell scripts initially invoke\ngit-sh-setup.\n\nThe change specifically detects my use pattern and, for the remainder\nof the script, changes the use pattern into a pattern closely related\nto the one that Junio documents:\n\n - export GIT_DIR that points at the correct .git directory;\n\n - GIT_WORK_TREE is left unset\n\n - set the current directory to the top of the working tree\n\nPerhaps the change should also set GIT_WORK_TREE. I haven't noticed\nsetting GIT_WORK_TREE to be necessary for \"git filter-branch\", but\nperhaps that is coincidental.\n\nIt seems to me that this change would uniformly support the use\npattern I use without affecting Git's behavior in any other case.\n\nDale\n"},{"id":"229136","messageId":"xmqq4n8fzmmj.fsf@gitster.dls.corp.google.com","threadId":"35154","inReplyTo":"201310171909.r9HJ9mxd007908@freeze.ariadne.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-10-17T20:08:20Z","receivedAt":"2013-10-17T20:08:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"worley@alum.mit.edu (Dale R. Worley) writes:\n\n> I must admit I've never seen the design (and I personally doubt that\n> the design has ever been written down). But at least the following\n> commands work correctly on a detached worktree if the current\n> directory contains the .git directory, because I am using them in a\n> production manner:\n>\n> git add\n\nIf you have this:\n\n\t/repositories/proj.git/{refs,objects,...}\n\t/working/trees/proj-wt1/\n\nwhere proj-wt1 is a working tree for that proj.git repository, the\nidea was to set these:\n\n\tGIT_DIR=/repositories/proj.git\n GIT_WORK_TREE=/working/trees/proj-wt1\n export GIT_DIR GIT_WORK_TREE\n\nand then working in /working/trees/proj-wt1 or any of its\nsubdirectory should work as if you did not have these two\nenvironment variables and had /working/trees/proj-wt1/.git instead\nof /repositories/proj.git as the repository. To make that use case\nwork was the motivation behind these environment variables.\n\n\tSide note: without GIT_WORK_TREE environment (or\n\tcore.worktree), there is no way to tell where the top level\n\tis, so you were limited to always be at the top level of\n\tyour working tree if you used GIT_DIR to refer to a\n\trepository that is not embedded in your working tree. There\n\twere some changes in this area, but I do not recall the\n\tdetails offhand.\n\nNow, when you say \"the cwd contains the .git directory\", do you mean\n\n\tcd /repositories\n git add ../working/trees/proj-wt1/file\n\nupdates \"file\" in the /repositories/proj.git/index? Or do you mean\nthis?\n\n\tcd /repositories/proj.git\n git add ../../working/trees/proj-wt1/file\n\nOr this?\n\n\tcd /repositories\n\tedit ../working/trees/proj-wt1/file\n\tgit add file\n\nMost of the commands you listed do not need to look at the actual\nworking tree files, so I would expect e.g. \"git log\" or \"git log\npaths...\" to work but I am wondering what your definition of \"works\"\nwith respect to the pathspecs, especially when you talk about\nstarting Git command _outside_ the working tree (whether the working\ntree has its repository embedded in it is not very relevant).\n"},{"id":"229137","messageId":"3401D1F36F134CDDB0881B196F79CB3A@PhilipOakley","threadId":"35154","inReplyTo":"xmqqk3hc3jbw.fsf@gitster.dls.corp.google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2013-10-17T20:11:21Z","receivedAt":"2013-10-17T20:11:21Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Junio C Hamano\" \n> \"Philip Oakley\" writes:\n>\n>> ... and the detection process for 'toplevel' may not work\n>> properly when in a separated work-tree environment.\n>\n> Without GIT_WORK_TREE exported to point at the top-level, there is\n> nothing that lets us \"detect\" it, as the working tree does not have\n> \".git\" directory to tell us to stop, no?\n>\n\n\"No\", but not in that way.\n\nMy point (to Dale) was, as you state, that the \"cd to top level\" was \n(IIUC) the probable causes of the fault, and that a documentation update \nwould probably be appropriate for the discussion on exporting \nGIT_WORK_TREE, and that it would specifically mention those git commands \nthat needed to \"cd to top level\", and hence would not work in such an \nenvironment. (I wasn't sure where the appropriate \"cd to top level\" \nfunction was)\n\nAn explanation here on the list wouldn't solve the problems for others \nwho are yet to make the same mistake, hence the implied suggestion.\n\nPhilip \n"},{"id":"229140","messageId":"xmqqr4bjy63y.fsf@gitster.dls.corp.google.com","threadId":"35154","inReplyTo":"3401D1F36F134CDDB0881B196F79CB3A@PhilipOakley","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-10-17T20:50:25Z","receivedAt":"2013-10-17T20:50:25Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Philip Oakley\" writes:\n\n> From: \"Junio C Hamano\" \n>> \"Philip Oakley\" writes:\n>>\n>>> ... and the detection process for 'toplevel' may not work\n>>> properly when in a separated work-tree environment.\n>>\n>> Without GIT_WORK_TREE exported to point at the top-level, there is\n>> nothing that lets us \"detect\" it, as the working tree does not have\n>> \".git\" directory to tell us to stop, no?\n>>\n>\n> \"No\", but not in that way.\n>\n> My point (to Dale) was, as you state, that the \"cd to top level\" was\n> (IIUC) the probable causes of the fault, and that a documentation\n> update would probably be appropriate for the discussion on exporting\n> GIT_WORK_TREE, and that it would specifically mention those git\n> commands that needed to \"cd to top level\", and hence would not work in\n> such an environment. (I wasn't sure where the appropriate \"cd to top\n> level\" function was)\n>\n> An explanation here on the list wouldn't solve the problems for others\n> who are yet to make the same mistake, hence the implied suggestion.\n\nI understand what you mean by these last two lines. It was unclear\nto me which part of our documentation needs updating and how, and\nthat was (and still is) what I was primarily interested in finding\nout.\n"},{"id":"229142","messageId":"5A09FF55D37146E7A02DF2F640A46406@PhilipOakley","threadId":"35154","inReplyTo":"xmqqr4bjy63y.fsf@gitster.dls.corp.google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2013-10-17T21:14:26Z","receivedAt":"2013-10-17T21:14:26Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Junio C Hamano\" \n> \"Philip Oakley\" writes:\n>\n>> From: \"Junio C Hamano\" \n>>> \"Philip Oakley\" writes:\n>>>\n>>>> ... and the detection process for 'toplevel' may not work\n>>>> properly when in a separated work-tree environment.\n>>>\n>>> Without GIT_WORK_TREE exported to point at the top-level, there is\n>>> nothing that lets us \"detect\" it, as the working tree does not have\n>>> \".git\" directory to tell us to stop, no?\n>>>\n>>\n>> \"No\", but not in that way.\n>>\n>> My point (to Dale) was, as you state, that the \"cd to top level\" was\n>> (IIUC) the probable causes of the fault, and that a documentation\n>> update would probably be appropriate for the discussion on exporting\n>> GIT_WORK_TREE, and that it would specifically mention those git\n>> commands that needed to \"cd to top level\", and hence would not work \n>> in\n>> such an environment. (I wasn't sure where the appropriate \"cd to top\n>> level\" function was)\n>>\n>> An explanation here on the list wouldn't solve the problems for \n>> others\n>> who are yet to make the same mistake, hence the implied suggestion.\n>\n> I understand what you mean by these last two lines. It was unclear\n> to me which part of our documentation needs updating and how, and\n> that was (and still is) what I was primarily interested in finding\n> out.\n>\nI was expecting that the places would be in git(1) [git.txt] and \nconfig(1) [config.txt], in the enironment variables GIT_WORK_TREE \nsection and core.worktree sections repectively. However what the right \ntext would be hasn't been fully determined yet, as it should be clear \nabout which commands don't follow the stated 'rules'. Dale's use case \ndoes appear to be stretching...\n\nPhilip \n"},{"id":"229151","messageId":"1390B0AFBE7F4C4A875987C7469B0791@PhilipOakley","threadId":"35154","inReplyTo":"5A09FF55D37146E7A02DF2F640A46406@PhilipOakley","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2013-10-17T22:38:09Z","receivedAt":"2013-10-17T22:38:09Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Philip Oakley\" \n> From: \"Junio C Hamano\" \n>> \"Philip Oakley\" writes:\n>>\n>>> From: \"Junio C Hamano\" \n>>>> \"Philip Oakley\" writes:\n>>>>\n>>>>> ... and the detection process for 'toplevel' may not work\n>>>>> properly when in a separated work-tree environment.\n>>>>\n>>>> Without GIT_WORK_TREE exported to point at the top-level, there is\n>>>> nothing that lets us \"detect\" it, as the working tree does not have\n>>>> \".git\" directory to tell us to stop, no?\n>>>>\n>>>\n>>> \"No\", but not in that way.\n>>>\n>>> My point (to Dale) was, as you state, that the \"cd to top level\" was\n>>> (IIUC) the probable causes of the fault, and that a documentation\n>>> update would probably be appropriate for the discussion on exporting\n>>> GIT_WORK_TREE, and that it would specifically mention those git\n>>> commands that needed to \"cd to top level\", and hence would not work\n>>> in\n>>> such an environment. (I wasn't sure where the appropriate \"cd to top\n>>> level\" function was)\n>>>\n>>> An explanation here on the list wouldn't solve the problems for\n>>> others\n>>> who are yet to make the same mistake, hence the implied suggestion.\n>>\n>> I understand what you mean by these last two lines. It was unclear\n>> to me which part of our documentation needs updating and how, and\n>> that was (and still is) what I was primarily interested in finding\n>> out.\n>>\n> I was expecting that the places would be in git(1) [git.txt] and\n> config(1) [config.txt], in the enironment variables GIT_WORK_TREE\n> section and core.worktree sections repectively. However what the right\n> text would be hasn't been fully determined yet, as it should be clear\n> about which commands don't follow the stated 'rules'. Dale's use case\n> does appear to be stretching...\n>\n> Philip\n\nA bit more looking gave that the cd_to_toplevel () in git-sh-setup.sh\ndirectly uses `git rev-parse --show-toplevel`, which simply returns\nwork_tree (static char *work_tree; in environment.c, with comment /*\nThis is set by setup_git_dir_gently() and/or git_default_config() */), \napparently without a check for the GIT_WORK_TREE.\n\nOne option may be to either protect the cd_to_toplevel code with a\ncheck of `git rev-parse --local-env-vars` to see if GIT_WORK_TREE is\npresent. Or create `git rev-parse --work-dir` to match `--git-dir`. This \nwould be a code level fix. This makes the assumption that if a deteched \nGIT_WORK_TREE is set then it is the top level.\n\nIn terms of command scripts that use git-sh-setup.sh we have a longish \nlist, so a full list in the documentation is probably unreasonable \n(which suggests that a code fix would be more apprpriate)\n\ncommands:\n\ngit-am\ngit-bisect\ngit-filter-branch\ngit-instaweb\ngit-lost-found\ngit-merge-one-file\ngit-mergetool\ngit-pull\ngit-quiltimport\ngit-rebase\ngit-repack\ngit-request-pull\ngit-stash\ngit-submodule\ngit-web--browse\n\ngit\\contrib\\*various*\n\n\n\nPhilip\n"},{"id":"229153","messageId":"20131017224834.GW9464@google.com","threadId":"35154","inReplyTo":"1390B0AFBE7F4C4A875987C7469B0791@PhilipOakley","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-10-17T22:48:35Z","receivedAt":"2013-10-17T22:48:35Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Philip Oakley wrote:\n\n> A bit more looking gave that the cd_to_toplevel () in git-sh-setup.sh\n> directly uses `git rev-parse --show-toplevel`, which simply returns\n> work_tree (static char *work_tree; in environment.c, with comment /*\n> This is set by setup_git_dir_gently() and/or git_default_config()\n> */), apparently without a check for the GIT_WORK_TREE.\n\nGetting closer. :)\n\nThe usual way to use GIT_WORK_TREE is along with GIT_DIR. That takes\nyou into the setup_explicit_git_dir() codepath, which does respect\nGIT_WORK_TREE as it should. (setup_discovered_git_dir does, too.)\n\nThe strange behavior you ran into is that unlike, say, git-pull.sh and\ngit-am.sh, filter-branch does not set SUBDIRECTORY_OK, store the\nprefix from 'git rev-parse --show-prefix', and then cd_to_toplevel at\nthe top of the script. In other words, nobody bothered to make it\nwork from anywhere other than the toplevel of the worktree to begin\nwith, and nobody wanted it enough to fix it later.\n\nHope that helps,\nJonathan\n"},{"id":"229191","messageId":"03C57561C4664D048C2F28134019C391@PhilipOakley","threadId":"35154","inReplyTo":"20131017224834.GW9464@google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2013-10-18T20:40:29Z","receivedAt":"2013-10-18T20:40:29Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Jonathan Nieder\" \n> Philip Oakley wrote:\n\nIt was Dale that had the problem, I was just suggesting where he might \nwant to look... ;-)\n\n>\n>> A bit more looking gave that the cd_to_toplevel () in git-sh-setup.sh\n>> directly uses `git rev-parse --show-toplevel`, which simply returns\n>> work_tree (static char *work_tree; in environment.c, with comment /*\n>> This is set by setup_git_dir_gently() and/or git_default_config()\n>> */), apparently without a check for the GIT_WORK_TREE.\n>\n> Getting closer. :)\n>\n> The usual way to use GIT_WORK_TREE is along with GIT_DIR.\n\nWhen re-checking the manual's git(1) env variable section the comment \nthat implies this didn't read well \"The value will not be used in \ncombination...\". The section probably needs that to be stated explicitly \n(\"an exported GIT_WORK_TREE is ignored if GIT_DIR is not set\").\n\n> That takes\n> you into the setup_explicit_git_dir() codepath, which does respect\n> GIT_WORK_TREE as it should. (setup_discovered_git_dir does, too.)\n>\n> The strange behavior you ran into is that unlike, say, git-pull.sh and\n> git-am.sh, filter-branch does not set SUBDIRECTORY_OK, store the\n> prefix from 'git rev-parse --show-prefix', and then cd_to_toplevel at\n> the top of the script. In other words, nobody bothered to make it\n> work from anywhere other than the toplevel of the worktree to begin\n> with, and nobody wanted it enough to fix it later.\n>\n\nI maybe wrong, but I thought that in Dale's case he was already at the \nsame level as the GIT_WORK_TREE he had set, so may not have expected to \nneed a cd_to_toplevel\n\n\nDale did propose a patch in \nhttp://thread.gmane.org/gmane.comp.version-control.git/236260 as the \nerror was from later in the setup script.\n\n> Hope that helps,\n> Jonathan\n>\n\nPhilip \n"},{"id":"229194","messageId":"201310182225.r9IMP4i3002659@freeze.ariadne.com","threadId":"35154","inReplyTo":"xmqq4n8fzmmj.fsf@gitster.dls.corp.google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Dale R. Worley","fromEmail":"worley@alum.mit.edu","sentAt":"2013-10-18T22:25:04Z","receivedAt":"2013-10-18T22:25:04Z","isPatch":false,"sender":{"key":"worley@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/19911107?v=4"},"body":"> From: Junio C Hamano \n\n> \tSide note: without GIT_WORK_TREE environment (or\n> \tcore.worktree), there is no way to tell where the top level\n> \tis, so you were limited to always be at the top level of\n> \tyour working tree if you used GIT_DIR to refer to a\n> \trepository that is not embedded in your working tree. There\n> \twere some changes in this area, but I do not recall the\n> \tdetails offhand.\n\nThat's not true. The core.worktree config variable tells the top of\nthe worktree, so once you've located the repository, you know where\nthe worktree is.\n\nIndeed, it's not clear why GIT_WORK_TREE exists, as that allows the\nuser to set GIT_WORK_TREE inconsistently with core.worktree. (What\nhappens if you do that?)\n\nDale\n"},{"id":"229195","messageId":"xmqqy55qurmf.fsf@gitster.dls.corp.google.com","threadId":"35154","inReplyTo":"201310182225.r9IMP4i3002659@freeze.ariadne.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-10-18T22:43:52Z","receivedAt":"2013-10-18T22:43:52Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"worley@alum.mit.edu (Dale R. Worley) writes:\n\n>> From: Junio C Hamano \n>\n>> \tSide note: without GIT_WORK_TREE environment (or\n>> \tcore.worktree), there is no way to tell where the top level\n>> \tis, so you were limited to always be at the top level of\n>> \tyour working tree if you used GIT_DIR to refer to a\n>> \trepository that is not embedded in your working tree. There\n>> \twere some changes in this area, but I do not recall the\n>> \tdetails offhand.\n>\n> That's not true. The core.worktree config variable tells the top of\n> the worktree, so once you've located the repository, you know where\n> the worktree is.\n\nRead the second line again, perhaps?\n\n> ... it's not clear why GIT_WORK_TREE exists, ...\n\nThe configuration item came _way_ later than the environment, and we\nneed to keep users and scripts from old world working, that is why.\n"},{"id":"229196","messageId":"201310182250.r9IMoW57003157@freeze.ariadne.com","threadId":"35154","inReplyTo":"xmqq4n8fzmmj.fsf@gitster.dls.corp.google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Dale R. Worley","fromEmail":"worley@alum.mit.edu","sentAt":"2013-10-18T22:50:32Z","receivedAt":"2013-10-18T22:50:32Z","isPatch":false,"sender":{"key":"worley@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/19911107?v=4"},"body":"> From: Junio C Hamano \n\n> Now, when you say \"the cwd contains the .git directory\", do you mean\n> \n> \tcd /repositories\n> git add ../working/trees/proj-wt1/file\n> \n> updates \"file\" in the /repositories/proj.git/index? Or do you mean\n> this?\n\nThe pattern I use is to have this:\n\n\t/repository/.git\n\t/working/...\n\nthen\n\n\tcd /repository\n\tgit add /working/x/y/z\n\nworks as you'd expect it to. \"git rm\" seems to work correctly under\nthese circumstances as well.\n\nI seem to recall that using relative values doesn't work under\nsome conditions involving symbolic links, but I can't recall the\ndetails right now.\n\n> you talk about starting Git command _outside_ the working tree\n> (whether the working tree has its repository embedded in it is not\n> very relevant).\n\nThe above pattern is what I mean, where the cwd is not within the work\ntree.\n\nDale\n"},{"id":"229197","messageId":"201310182254.r9IMslOr003223@freeze.ariadne.com","threadId":"35154","inReplyTo":"xmqqr4bjy63y.fsf@gitster.dls.corp.google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Dale R. Worley","fromEmail":"worley@alum.mit.edu","sentAt":"2013-10-18T22:54:47Z","receivedAt":"2013-10-18T22:54:47Z","isPatch":false,"sender":{"key":"worley@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/19911107?v=4"},"body":"> From: Junio C Hamano \n\n> It was unclear to me which part of our documentation needs updating\n> and how, and that was (and still is) what I was primarily interested\n> in finding out.\n\nIt seems to me that what is missing is a description of the\ncircumstances under which Git can be run. With Subversion (the only\nother source control system I know in detail), the working tree that\nis operated on is at and below the cwd, and the working tree always\npoints to the repository. (A subdirectory of a working tree is also a\nvalid working tree.)\n\nWith Git, it seems that the basic usage is that Git searches upward\nfrom the cwd to find the top of the work tree, which is distinguished\nby having a .git subdirectory. The rules when the worktree is\ndetached are more complicated, and don't seem to be written in any\nsingle place.\n\nDale\n"},{"id":"229259","messageId":"201310211851.r9LIpOaH000372@freeze.ariadne.com","threadId":"35154","inReplyTo":"xmqqy55qurmf.fsf@gitster.dls.corp.google.com","subject":"Re: [git-users] Problem using detached worktrees with commands implemented in scripts","fromName":"Dale R. Worley","fromEmail":"worley@alum.mit.edu","sentAt":"2013-10-21T18:51:24Z","receivedAt":"2013-10-21T18:51:24Z","isPatch":false,"sender":{"key":"worley@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/19911107?v=4"},"body":"> From: Junio C Hamano \n\n> > ... it's not clear why GIT_WORK_TREE exists, ...\n> \n> The configuration item came _way_ later than the environment, and we\n> need to keep users and scripts from old world working, that is why.\n\nOK, that explains a great deal. IIRC, I first became aware that\ndetached worktrees are possible through the documentation of\ncore.worktree. As Git's architecture has a tight binding between the\nrepository and the worktree, it made a great deal of sense to me that\nthe repository points to the detached worktree. And the absence of\ncore.worktree, a non-detached worktree, is essentially equivalent to\nhaving core.worktree specify the directory containing the .git\ndirectory.\n\nSo the obvious way (to me) to invoke Git with a detached worktree is\nto set GIT_DIR to point to the repository, and the repository points\nto the root of the worktree. If the command operates on the worktree,\nGit can compare the cwd with the worktree root to determine the\nrelative path of files.\n\n(And you can see that in this situation, Git doesn't have to search\nupward to try to determine where the worktree root is.)\n\nWhat you're saying is that there's an older mode of operation where\nthe repository does not point to the worktree. Instead, the caller\nhas to set GIT_DIR to locate the repository and GIT_WORK_TREE to\nlocate the worktree. This would be prone to error, because the user\nis responsible for always pairing the repository with the correct\nworktree; it doesn't enforce the architectural assumption that the\nrepository is paired with a work tree.\n\nDale\n"}]}