{"thread":{"id":"35550","subject":"Re: [PATCH v2 00/21] Support multiple worktrees","startedAt":"2013-12-19T14:12:41Z","lastAt":"2013-12-28T02:46:57Z","messageCount":7,"participants":["Duy Nguyen","Junio C Hamano"],"isPatch":true,"patchVersion":2,"patchTotal":21},"messages":[{"id":"232234","messageId":"CACsJy8BV6uerMyHNHviL0Jy8s+_jca8NM-hVfnM=u4cr-=JX1Q@mail.gmail.com","threadId":"35550","inReplyTo":null,"subject":"Re: [PATCH v2 00/21] Support multiple worktrees","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-12-19T14:12:41Z","receivedAt":"2013-12-19T14:12:41Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"I've got a better version [1] that fixes everything I can think of\n(there's still some room for improvements). I'm going to use it a bit\nlonger before reposting again. But here's its basic design without\ngoing down to code\n\nNew .git file format includes two lines:\n-- 8< --\ngitid: \ngitdir: \n-- 8< --\n\nWhich would set $GIT_COMMON_DIR to and $GIT_DIR to\n/repos/. Repository split is the same as before, worktree\nstuff in $GIT_DIR, the rest in $GIT_COMMON_DIR. This .git file format\ntakes precedence over core.worktree but can still be overriden with\n$GIT_WORK_TREE. The main interface to create new worktree is \"git\ncheckout --to\".\n\n\"repos\" belongs to $GIT_COMMON_DIR (i.e. shared across all checkouts).\nThe new worktrees (which I call \"linked checkouts\") can also access\nHEAD of the original worktree via a virtual path \"main/HEAD\". This\nmakes it possible for a linked checkout to detach HEAD of the main\none.\n\nThere are three entries in repos/: \"gitdir\" should point to the\n.git file that points it back here. Every time a linked checkout is\nused, git should update mtime of this \"gitdir\" file to help pruning.\nIt should update the file content too if the repo is moved. \"link\" is\na hardlink to .git file, if supported, again for pruning support.\n\"locked\", if exists, means no automatic pruning (e.g. the linked\ncheckout is on a portable device).\n\nThe interesting thing is support for third party scripts (or hooks,\nmaybe) so that they could work with both old and new git versions\nwithout some sort of git version/feature detection. Of course old git\nversions will only work with ordinary worktrees. To that end, \"git\nrev-parse --git-dir\" behavior could be changed by two environment\nvariables. $GIT_ONE_PATH makes 'rev-parse --git-dir' return the .git\n_file_ in this case, which makes it much easier to pass the repo's\ncheckout view around with \"git --git-dir=... \".$GIT_COMMON_DIR_PATH\nmakes 'rev-parse --git-dir' return $GIT_COMMON_DIR if it's from a\nlinked checkout, or $GIT_DIR otherwise. This makes 'rev-parse\n--git-dir' falls back safely when running using old git versions. The\nlast patch in [1] that updates git-completion.bash could demonstrate\nhow it's used.\n\n[1] https://github.com/pclouds/git.git checkout-new-worktree\n-- \nDuy\n"},{"id":"232293","messageId":"xmqq7gaze00k.fsf@gitster.dls.corp.google.com","threadId":"35550","inReplyTo":"CACsJy8BV6uerMyHNHviL0Jy8s+_jca8NM-hVfnM=u4cr-=JX1Q@mail.gmail.com","subject":"Re: [PATCH v2 00/21] Support multiple worktrees","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-12-20T20:32:43Z","receivedAt":"2013-12-20T20:32:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen writes:\n\n> I've got a better version [1] that fixes everything I can think of\n> (there's still some room for improvements). I'm going to use it a bit\n> longer before reposting again. But here's its basic design without\n> going down to code\n>\n> New .git file format includes two lines:\n> -- 8< --\n> gitid: \n> gitdir: \n> -- 8< --\n>\n> Which would set $GIT_COMMON_DIR to and $GIT_DIR to\n> /repos/. Repository split is the same as before, worktree\n> stuff in $GIT_DIR, the rest in $GIT_COMMON_DIR. This .git file format\n> takes precedence over core.worktree but can still be overriden with\n> $GIT_WORK_TREE. The main interface to create new worktree is \"git\n> checkout --to\".\n>\n> \"repos\" belongs to $GIT_COMMON_DIR (i.e. shared across all checkouts).\n> The new worktrees (which I call \"linked checkouts\") can also access\n> HEAD of the original worktree via a virtual path \"main/HEAD\". This\n> makes it possible for a linked checkout to detach HEAD of the main\n> one.\n\nI am not happy with the choice of \"main/HEAD\" that would squat on a\ngood name for remote-tracking branch (i.e. s/origin/main/), though.\n$GIT_DIR/COMMON_HEAD perhaps?\n\n> The interesting thing is support for third party scripts (or hooks,\n> maybe) so that they could work with both old and new git versions\n> without some sort of git version/feature detection. Of course old git\n> versions will only work with ordinary worktrees. To that end, \"git\n> rev-parse --git-dir\" behavior could be changed by two environment\n> variables. $GIT_ONE_PATH makes 'rev-parse --git-dir' return the .git\n> _file_ in this case, which makes it much easier to pass the repo's\n> checkout view around with \"git --git-dir=... \".$GIT_COMMON_DIR_PATH\n> makes 'rev-parse --git-dir' return $GIT_COMMON_DIR if it's from a\n> linked checkout, or $GIT_DIR otherwise.\n\nI do not understand why you need to go such a route.\n\nExisting scripts that works only in a real repository will only know\n\"git rev-parse --git-dir\" as the way to get the real GIT_DIR and\nwould not care about the \"common\" thing. Scripts updated to work\nwell with the \"common\" thing needs to be aware of the \"common\" thing\nanyway, so adding \"git rev-parse --common-git-dir\" or somesuch that\nonly these updated knows would be sufficient, no?\n"},{"id":"232296","messageId":"CACsJy8DuXbCTjcVJNH=w6h3zgo0sxRgMfSQXZFtngYy2FLJTDg@mail.gmail.com","threadId":"35550","inReplyTo":"xmqq7gaze00k.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v2 00/21] Support multiple worktrees","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-12-21T02:00:59Z","receivedAt":"2013-12-21T02:00:59Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Dec 21, 2013 at 3:32 AM, Junio C Hamano wrote:\n> Duy Nguyen writes:\n>\n>> I've got a better version [1] that fixes everything I can think of\n>> (there's still some room for improvements). I'm going to use it a bit\n>> longer before reposting again. But here's its basic design without\n>> going down to code\n>>\n>> New .git file format includes two lines:\n>> -- 8< --\n>> gitid: \n>> gitdir: \n>> -- 8< --\n>>\n>> Which would set $GIT_COMMON_DIR to and $GIT_DIR to\n>> /repos/. Repository split is the same as before, worktree\n>> stuff in $GIT_DIR, the rest in $GIT_COMMON_DIR. This .git file format\n>> takes precedence over core.worktree but can still be overriden with\n>> $GIT_WORK_TREE. The main interface to create new worktree is \"git\n>> checkout --to\".\n>>\n>> \"repos\" belongs to $GIT_COMMON_DIR (i.e. shared across all checkouts).\n>> The new worktrees (which I call \"linked checkouts\") can also access\n>> HEAD of the original worktree via a virtual path \"main/HEAD\". This\n>> makes it possible for a linked checkout to detach HEAD of the main\n>> one.\n>\n> I am not happy with the choice of \"main/HEAD\" that would squat on a\n> good name for remote-tracking branch (i.e. s/origin/main/), though.\n> $GIT_DIR/COMMON_HEAD perhaps?\n\nIt's not just about HEAD. Anything worktree-specific of the main\ncheckout can be accessed this way, e.g. main/index,\nmain/FETCH_HEAD.... and it's not exactly \"common\" because it's\nworktree info. Maybe 1ST_ as the prefix (e.g. 1ST_HEAD, 1ST_index...)\n?\n\n>> The interesting thing is support for third party scripts (or hooks,\n>> maybe) so that they could work with both old and new git versions\n>> without some sort of git version/feature detection. Of course old git\n>> versions will only work with ordinary worktrees. To that end, \"git\n>> rev-parse --git-dir\" behavior could be changed by two environment\n>> variables. $GIT_ONE_PATH makes 'rev-parse --git-dir' return the .git\n>> _file_ in this case, which makes it much easier to pass the repo's\n>> checkout view around with \"git --git-dir=... \".$GIT_COMMON_DIR_PATH\n>> makes 'rev-parse --git-dir' return $GIT_COMMON_DIR if it's from a\n>> linked checkout, or $GIT_DIR otherwise.\n>\n> I do not understand why you need to go such a route.\n>\n> Existing scripts that works only in a real repository will only know\n> \"git rev-parse --git-dir\" as the way to get the real GIT_DIR and\n> would not care about the \"common\" thing. Scripts updated to work\n> well with the \"common\" thing needs to be aware of the \"common\" thing\n> anyway, so adding \"git rev-parse --common-git-dir\" or somesuch that\n> only these updated knows would be sufficient, no?\n\nIt simplifies the changes, if the new script is to work with both old\nand new git versions it may have to write\n\nDIR=`git rev-parse --git-common-dir 2>/dev/null || git rev-parse --git-dir`\n\nthe env way makes it\n\nDIR=`GIT_COMMON_DIR=1 git rev-parse --git-dir`\n-- \nDuy\n"},{"id":"232340","messageId":"7vvbyhjsp4.fsf@alter.siamese.dyndns.org","threadId":"35550","inReplyTo":"CACsJy8DuXbCTjcVJNH=w6h3zgo0sxRgMfSQXZFtngYy2FLJTDg@mail.gmail.com","subject":"Re: [PATCH v2 00/21] Support multiple worktrees","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-12-22T06:38:47Z","receivedAt":"2013-12-22T06:38:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen writes:\n\n>> I am not happy with the choice of \"main/HEAD\" that would squat on a\n>> good name for remote-tracking branch (i.e. s/origin/main/), though.\n>> $GIT_DIR/COMMON_HEAD perhaps?\n>\n> It's not just about HEAD. Anything worktree-specific of the main\n> checkout can be accessed this way, e.g. main/index,\n> main/FETCH_HEAD.... and it's not exactly \"common\" because it's\n> worktree info. Maybe 1ST_ as the prefix (e.g. 1ST_HEAD, 1ST_index...)\n> ?\n\nDo we even need to expose them as ref-like things as a part of the\nexternal API/UI in the first place? For end-user scripts that want\nto operate in a real or borrowing worktree, there should be no\nreason to touch this \"other\" repository directly. Things like \"if\none of the wortrees tries to check out a branch that is already\nchecked out elsewhere, error out\" policy may need to consult the\nother worktrees via $GIT_COMMON_DIR mechanism but at that level we\nhave all the control without contaminating end-user facing ref\nnamespace in a way main/FETCH_HEAD... do. You said\n\n This makes it possible for a linked checkout to detach HEAD of\n the main one.\n\nbut I think that is misguided---it just makes it easier to confuse\nusers, if done automatically and without any policy knob. It instead\nshould error out, while saying which worktree has the branch in\nquestion checked out. After all, checking out a branch that is\nchecked out in another worktree (not the common one) needs the same\ncaution, so \"main/HEAD\" is not the only special one.\n\nAnd if your updated \"git checkout 'frotz'\" gives a clear report of\nwhich worktree has the branch 'frotz' checked out, the user can go\nthere to detach the HEAD in that worktree to detach with\n\n\tgit -C $the_other_one checkout HEAD^0\n\nif he chooses to.\n"},{"id":"232349","messageId":"CACsJy8DL5=B=jch6j6g_3xj3KRsLXxwMChVHF9MUFvafhWhYag@mail.gmail.com","threadId":"35550","inReplyTo":"7vvbyhjsp4.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v2 00/21] Support multiple worktrees","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-12-22T08:44:52Z","receivedAt":"2013-12-22T08:44:52Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sun, Dec 22, 2013 at 1:38 PM, Junio C Hamano wrote:\n> Duy Nguyen writes:\n>\n>>> I am not happy with the choice of \"main/HEAD\" that would squat on a\n>>> good name for remote-tracking branch (i.e. s/origin/main/), though.\n>>> $GIT_DIR/COMMON_HEAD perhaps?\n>>\n>> It's not just about HEAD. Anything worktree-specific of the main\n>> checkout can be accessed this way, e.g. main/index,\n>> main/FETCH_HEAD.... and it's not exactly \"common\" because it's\n>> worktree info. Maybe 1ST_ as the prefix (e.g. 1ST_HEAD, 1ST_index...)\n>> ?\n>\n> Do we even need to expose them as ref-like things as a part of the\n> external API/UI in the first place? For end-user scripts that want\n> to operate in a real or borrowing worktree, there should be no\n> reason to touch this \"other\" repository directly. Things like \"if\n> one of the wortrees tries to check out a branch that is already\n> checked out elsewhere, error out\" policy may need to consult the\n> other worktrees via $GIT_COMMON_DIR mechanism but at that level we\n> have all the control without contaminating end-user facing ref\n> namespace in a way main/FETCH_HEAD... do.\n\nNo, external API/UI is just extra bonus. We need to (or should) do so\nin order to handle $GIT_COMMON_DIR/HEAD exactly like how we do normal\nrefs. Given any ref, git_path(ref) gives the path to that ref,\ngit_path(\"logs/%s\", ref) gives the path of its reflog. By mapping\nspecial names to real paths behind git_path(), We can feed\n$GIT_COMMON_DIR/HEAD (under special names) to refs.c and it'll handle\ncorrectly without any changes for special cases.\n\n> You said\n>\n> This makes it possible for a linked checkout to detach HEAD of\n> the main one.\n>\n> but I think that is misguided---it just makes it easier to confuse\n> users, if done automatically and without any policy knob. It instead\n> should error out, while saying which worktree has the branch in\n> question checked out. After all, checking out a branch that is\n> checked out in another worktree (not the common one) needs the same\n> caution, so \"main/HEAD\" is not the only special one.\n>\n> And if your updated \"git checkout 'frotz'\" gives a clear report of\n> which worktree has the branch 'frotz' checked out, the user can go\n> there to detach the HEAD in that worktree to detach with\n>\n> git -C $the_other_one checkout HEAD^0\n>\n> if he chooses to.\n\nJonathan mentions about the \"checkout in portable device\" case that\nwould make the above a bit unnatural as you just can't \"cd\" there (git\nupdate-ref still works).\n\nI don't see any problems with checking out a branch multiple times. I\nmay want to try modifying something in the branch that will be thrown\naway in the end. It's when the user updates a branch that we should\neither error+reject or detach other checkouts. I guess it's up to the\nuser to decide which way they want. The error+reject way may make the\nuser hunt through dead checkouts waiting to be pruned. But we can\nstart with error+reject then add an option to auto-detach.\n-- \nDuy\n"},{"id":"232408","messageId":"7vy537k038.fsf@alter.siamese.dyndns.org","threadId":"35550","inReplyTo":"CACsJy8DL5=B=jch6j6g_3xj3KRsLXxwMChVHF9MUFvafhWhYag@mail.gmail.com","subject":"Re: [PATCH v2 00/21] Support multiple worktrees","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-12-26T17:12:43Z","receivedAt":"2013-12-26T17:12:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen writes:\n\n> On Sun, Dec 22, 2013 at 1:38 PM, Junio C Hamano wrote:\n>\n>> Do we even need to expose them as ref-like things as a part of the\n>> external API/UI in the first place? For end-user scripts that want\n>> to operate in a real or borrowing worktree, there should be no\n>> reason to touch this \"other\" repository directly. Things like \"if\n>> one of the wortrees tries to check out a branch that is already\n>> checked out elsewhere, error out\" policy may need to consult the\n>> other worktrees via $GIT_COMMON_DIR mechanism but at that level we\n>> have all the control without contaminating end-user facing ref\n>> namespace in a way main/FETCH_HEAD... do.\n>\n> No, external API/UI is just extra bonus. We need to (or should) do so\n> in order to handle $GIT_COMMON_DIR/HEAD exactly like how we do normal\n> refs.\n\nAnd that is what I consider a confusion-trigger, not a nice bonus.\n\nI do not think it is particularly a good idea to contaminate\nend-user namespace for this kind of \"peek another repository\"\nspecial operation.\n\nIn order to handle your \"multiple worktrees manipulating the same\nbranch\" case sanely, you need to be aware of not just the real\nrepository your worktree is borrowing from, but also _all_ the other\nworktrees that borrow from that same real repository, so a single\n\"main\" virtual namespace will not cut it. You will be dealing with\nan unbounded set of HEADs, one for each such worktree.\n\nCan't we do this by adding a \"with this real repository\" layer,\ne.g. \"resolve HEAD wrt that repository\", somewhat similar to how we\npeek into submodule namespaces?\n"},{"id":"232464","messageId":"CACsJy8BV39frnQo1BW2p1J7H0pU7LifzcUey61J5Jkc6Wn=2rw@mail.gmail.com","threadId":"35550","inReplyTo":"7vy537k038.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v2 00/21] Support multiple worktrees","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-12-28T02:46:57Z","receivedAt":"2013-12-28T02:46:57Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Dec 27, 2013 at 12:12 AM, Junio C Hamano wrote:\n> Duy Nguyen writes:\n>\n>> On Sun, Dec 22, 2013 at 1:38 PM, Junio C Hamano wrote:\n>>\n>>> Do we even need to expose them as ref-like things as a part of the\n>>> external API/UI in the first place? For end-user scripts that want\n>>> to operate in a real or borrowing worktree, there should be no\n>>> reason to touch this \"other\" repository directly. Things like \"if\n>>> one of the wortrees tries to check out a branch that is already\n>>> checked out elsewhere, error out\" policy may need to consult the\n>>> other worktrees via $GIT_COMMON_DIR mechanism but at that level we\n>>> have all the control without contaminating end-user facing ref\n>>> namespace in a way main/FETCH_HEAD... do.\n>>\n>> No, external API/UI is just extra bonus. We need to (or should) do so\n>> in order to handle $GIT_COMMON_DIR/HEAD exactly like how we do normal\n>> refs.\n>\n> And that is what I consider a confusion-trigger, not a nice bonus.\n>\n> I do not think it is particularly a good idea to contaminate\n> end-user namespace for this kind of \"peek another repository\"\n> special operation.\n>\n> In order to handle your \"multiple worktrees manipulating the same\n> branch\" case sanely, you need to be aware of not just the real\n> repository your worktree is borrowing from, but also _all_ the other\n> worktrees that borrow from that same real repository, so a single\n> \"main\" virtual namespace will not cut it. You will be dealing with\n> an unbounded set of HEADs, one for each such worktree.\n\nYes. My problem is, while all secondary worktrees are in\n$GIT_DIR/repos and their HEADs can be accessed there with\n\"repos/xxx/HEAD\", the first worktree's HEAD can't be accessed this way\nbecause \"HEAD\" in a linked checkouts means repos//HEAD.\n\n> Can't we do this by adding a \"with this real repository\" layer,\n> e.g. \"resolve HEAD wrt that repository\", somewhat similar to how we\n> peek into submodule namespaces?\n\nHmm.. never thought of it like a \"submodule\". Thanks for the idea.\n-- \nDuy\n"}]}