{"thread":{"id":"11717","subject":"Multiple working trees with GIT ?","startedAt":"2008-01-24T07:49:52Z","lastAt":"2008-01-24T14:51:28Z","messageCount":8,"participants":["Willy Tarreau","Julian Phillips","Johannes Schindelin","J. Bruce Fields"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"66464","messageId":"20080124074952.GA8793@1wt.eu","threadId":"11717","inReplyTo":null,"subject":"Multiple working trees with GIT ?","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2008-01-24T07:49:52Z","receivedAt":"2008-01-24T07:49:52Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"Hi all,\n\nI'm having long thoughts about how to use GIT to manage a distro. One of\nthe aspects which comes very often is the notion of \"variant\" for a\npackaging. For instance, the whole project could consist in a list of packages\nwith their branches, but this list may vary depending on the platform, the\nmedium, etc... I was searching how to propagate common changes withing variants\nwith the least hassle.\n\nI figured out that having one file list per variant will be very annoying. In\nanother project, that's already what I have and frankly, applying the same\nchange to 10 files is counter-productive. Since the lists will often be the\nsames except for a few entries, and since most updates will be relevant to\nall variants, I thought branches will be my best friends.\n\nBut I would like to be able to always access file lists, without having to\nconstantly git-checkout <variant-X>.\n\nFinally, I found a very convenient solution, but I don't know if there are\nany risks using it. If we can conclude to a riskless usage (with/without\nsome adjustments), I can contribute a script to setup this environment.\n\nThe idea is to have multiple working trees in a subdirectory of the normal\none. Some would probably like them to be movable anywhere else, but let's\nnot complicate things first. But in order not to constantly have to pull/push\nin every tree, I set up the .git repo of each working tree with many symlinks :\n\n  project/\n          .git/ (normal repo)\n          ... master checked out there by default ...\n          worktree/\n                   variant_a/\n                             .git/ (symlinks)\n                   variant_b/\n                             .git/ (symlinks)\n                   variant_c/\n                             .git/ (symlinks)\n\n\nEach variant is set up like this :\n    4096 Jan 24 08:44 .git\n      19 Jan 24 08:27 .git/HEAD\n      28 Jan 24 08:30 .git/logs\n     419 Jan 24 08:44 .git/logs/HEAD\n      26 Jan 24 08:31 .git/logs/refs -> ../../../../.git/logs/refs\n      18 Jan 24 08:31 .git/refs -> ../../../.git/refs\n      21 Jan 24 08:31 .git/objects -> ../../../.git/objects\n      18 Jan 24 08:31 .git/info -> ../../../.git/info\n      19 Jan 24 08:31 .git/hooks -> ../../../.git/hooks\n      25 Jan 24 08:31 .git/description -> ../../../.git/description\n      20 Jan 24 08:31 .git/config -> ../../../.git/config\n     118 Jan 24 08:44 .git/index\n      22 Jan 24 08:31 .git/branches -> ../../../.git/branches\n      63 Jan 24 08:44 .git/FETCH_HEAD\n      41 Jan 24 08:44 .git/ORIG_HEAD\n\nIn fact, I found that each directory which hosts a HEAD file needs to\nremain a directory because of this head, but all other dirs can be\nsymlinks to original tree.\n\nThis works pretty well. I can simply cd worktree/variant_a and work on a\nfile, or pull master, or even git-cherry-pick from other branches (pretty\nconvenient for this usage). But I don't know what caveats I may encounter.\n\nMaybe there are other solutions too. I see that we tend to replace symlinks\neverywhere with ref files. We might as well (in a far future version) accept\na file for \".git\" which would contain a path to the central repo and the\nbranch's head.\n\nI'm open to comments. Please keep me CC, I'm not subscribed to the git list.\n\nThanks,\nWilly\n"},{"id":"66468","messageId":"Pine.LNX.4.64.0801240947230.14173@reaper.quantumfyre.co.uk","threadId":"11717","inReplyTo":"20080124074952.GA8793@1wt.eu","subject":"Re: Multiple working trees with GIT ?","fromName":"Julian Phillips","fromEmail":"julian@quantumfyre.co.uk","sentAt":"2008-01-24T09:59:05Z","receivedAt":"2008-01-24T09:59:05Z","isPatch":false,"sender":{"key":"julian@quantumfyre.co.uk","avatar":"https://avatars.githubusercontent.com/u/948888?v=4"},"body":"On Thu, 24 Jan 2008, Willy Tarreau wrote:\n\n> Hi all,\n>\n> I'm having long thoughts about how to use GIT to manage a distro. One of\n> the aspects which comes very often is the notion of \"variant\" for a\n> packaging. For instance, the whole project could consist in a list of packages\n> with their branches, but this list may vary depending on the platform, the\n> medium, etc... I was searching how to propagate common changes withing variants\n> with the least hassle.\n>\n> I figured out that having one file list per variant will be very annoying. In\n> another project, that's already what I have and frankly, applying the same\n> change to 10 files is counter-productive. Since the lists will often be the\n> sames except for a few entries, and since most updates will be relevant to\n> all variants, I thought branches will be my best friends.\n>\n> But I would like to be able to always access file lists, without having to\n> constantly git-checkout <variant-X>.\n>\n> Finally, I found a very convenient solution, but I don't know if there are\n> any risks using it. If we can conclude to a riskless usage (with/without\n> some adjustments), I can contribute a script to setup this environment.\n>\n> The idea is to have multiple working trees in a subdirectory of the normal\n> one. Some would probably like them to be movable anywhere else, but let's\n> not complicate things first. But in order not to constantly have to pull/push\n> in every tree, I set up the .git repo of each working tree with many symlinks :\n>\n>  project/\n>          .git/ (normal repo)\n>          ... master checked out there by default ...\n>          worktree/\n>                   variant_a/\n>                             .git/ (symlinks)\n>                   variant_b/\n>                             .git/ (symlinks)\n>                   variant_c/\n>                             .git/ (symlinks)\n>\n>\n> Each variant is set up like this :\n>    4096 Jan 24 08:44 .git\n>      19 Jan 24 08:27 .git/HEAD\n>      28 Jan 24 08:30 .git/logs\n>     419 Jan 24 08:44 .git/logs/HEAD\n>      26 Jan 24 08:31 .git/logs/refs -> ../../../../.git/logs/refs\n>      18 Jan 24 08:31 .git/refs -> ../../../.git/refs\n>      21 Jan 24 08:31 .git/objects -> ../../../.git/objects\n>      18 Jan 24 08:31 .git/info -> ../../../.git/info\n>      19 Jan 24 08:31 .git/hooks -> ../../../.git/hooks\n>      25 Jan 24 08:31 .git/description -> ../../../.git/description\n>      20 Jan 24 08:31 .git/config -> ../../../.git/config\n>     118 Jan 24 08:44 .git/index\n>      22 Jan 24 08:31 .git/branches -> ../../../.git/branches\n>      63 Jan 24 08:44 .git/FETCH_HEAD\n>      41 Jan 24 08:44 .git/ORIG_HEAD\n>\n> In fact, I found that each directory which hosts a HEAD file needs to\n> remain a directory because of this head, but all other dirs can be\n> symlinks to original tree.\n>\n> This works pretty well. I can simply cd worktree/variant_a and work on a\n> file, or pull master, or even git-cherry-pick from other branches (pretty\n> convenient for this usage). But I don't know what caveats I may encounter.\n\nYou might want to have a look at the git-new-workdir script in contrib, it \ndoes basically the same thing.  It's been there for about 10 months now. \nIt was based on an email from Junio:\n\nhttp://article.gmane.org/gmane.comp.version-control.git/41513/\n\nHowever, there are some caveats about using this approach, basically about \nthe fact that there is nothing stopping you from updating refs that are \ncurrently checked out in another directory and causing yourself all sorts \nof pain ... the topic has cropped up a couple of times on the list since \nthe script was added.\n\n> Maybe there are other solutions too. I see that we tend to replace symlinks\n> everywhere with ref files. We might as well (in a far future version) accept\n> a file for \".git\" which would contain a path to the central repo and the\n> branch's head.\n\nThere was a suggestion for something not too dissimilar even before the \nnew-workdir script:\n\nhttp://thread.gmane.org/gmane.comp.version-control.git/33755\n\nAFAIK it hasn't actually been implemented.\n\n-- \nJulian\n\n  ---\nMay your SO always know when you need a hug.\n"},{"id":"66473","messageId":"alpine.LSU.1.00.0801241102260.5731@racer.site","threadId":"11717","inReplyTo":"Pine.LNX.4.64.0801240947230.14173@reaper.quantumfyre.co.uk","subject":"Re: Multiple working trees with GIT ?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-01-24T11:04:42Z","receivedAt":"2008-01-24T11:04:42Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 24 Jan 2008, Julian Phillips wrote:\n\n> You might want to have a look at the git-new-workdir script in contrib, \n> it does basically the same thing.  It's been there for about 10 months \n> now. It was based on an email from Junio:\n> \n> http://article.gmane.org/gmane.comp.version-control.git/41513/\n\nFWIW I have a patch to do something like that in \"git branch\" itself.\n\n> However, there are some caveats about using this approach, basically \n> about the fact that there is nothing stopping you from updating refs \n> that are currently checked out in another directory and causing yourself \n> all sorts of pain ... the topic has cropped up a couple of times on the \n> list since the script was added.\n\nI agree; maybe we should have a telltale file \n\"refs/heads/<bla>.checkedout\" which is heeded by \"git checkout\" and \"git \nbranch -d/-D\", as well as update_ref() (should only update that ref when \nit HEAD points to it)?\n\nCiao,\nDscho\n"},{"id":"66475","messageId":"20080124125606.GB13247@1wt.eu","threadId":"11717","inReplyTo":"alpine.LSU.1.00.0801241102260.5731@racer.site","subject":"Re: Multiple working trees with GIT ?","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2008-01-24T12:56:06Z","receivedAt":"2008-01-24T12:56:06Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"On Thu, Jan 24, 2008 at 11:04:42AM +0000, Johannes Schindelin wrote:\n> Hi,\n> \n> On Thu, 24 Jan 2008, Julian Phillips wrote:\n> \n> > You might want to have a look at the git-new-workdir script in contrib, \n> > it does basically the same thing.  It's been there for about 10 months \n> > now. It was based on an email from Junio:\n> > \n> > http://article.gmane.org/gmane.comp.version-control.git/41513/\n> \n> FWIW I have a patch to do something like that in \"git branch\" itself.\n>\n> > However, there are some caveats about using this approach, basically \n> > about the fact that there is nothing stopping you from updating refs \n> > that are currently checked out in another directory and causing yourself \n> > all sorts of pain ... the topic has cropped up a couple of times on the \n> > list since the script was added.\n> \n> I agree; maybe we should have a telltale file \n> \"refs/heads/<bla>.checkedout\" which is heeded by \"git checkout\" and \"git \n> branch -d/-D\", as well as update_ref() (should only update that ref when \n> it HEAD points to it)?\n\nWhy not generalize this into HEAD.$branch (thus limiting to one checkout\nper branch) or HEAD.$checkoutdir ?\n\nBest regards,\nWilly\n"},{"id":"66476","messageId":"20080124125905.GC13247@1wt.eu","threadId":"11717","inReplyTo":"Pine.LNX.4.64.0801240947230.14173@reaper.quantumfyre.co.uk","subject":"Re: Multiple working trees with GIT ?","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2008-01-24T12:59:05Z","receivedAt":"2008-01-24T12:59:05Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"Hi Julian,\n\nOn Thu, Jan 24, 2008 at 09:59:05AM +0000, Julian Phillips wrote:\n(...)\n> >This works pretty well. I can simply cd worktree/variant_a and work on a\n> >file, or pull master, or even git-cherry-pick from other branches (pretty\n> >convenient for this usage). But I don't know what caveats I may encounter.\n> \n> You might want to have a look at the git-new-workdir script in contrib, it \n> does basically the same thing.  It's been there for about 10 months now. \n> It was based on an email from Junio:\n> \n> http://article.gmane.org/gmane.comp.version-control.git/41513/\n\nInteresting lecture, thanks for the pointer. At least now I know that it is\nnot too much exotic.\n\n> However, there are some caveats about using this approach, basically about \n> the fact that there is nothing stopping you from updating refs that are \n> currently checked out in another directory and causing yourself all sorts \n> of pain ... the topic has cropped up a couple of times on the list since \n> the script was added.\n\nhmmm good point. Given that I'm used to push into remote working dirs and\nto get caught by this problem, I think I would most often escape from the\ncaveat, but we should take care of not trapping newbies.\n\n> >Maybe there are other solutions too. I see that we tend to replace symlinks\n> >everywhere with ref files. We might as well (in a far future version) \n> >accept\n> >a file for \".git\" which would contain a path to the central repo and the\n> >branch's head.\n> \n> There was a suggestion for something not too dissimilar even before the \n> new-workdir script:\n> \n> http://thread.gmane.org/gmane.comp.version-control.git/33755\n\nOK, thank you for your links. I still think I will wo the easy way for now,\nprobably using git-new-workdir, waiting for a general consensus on the subject.\n\nRegards,\nWilly\n"},{"id":"66477","messageId":"alpine.LSU.1.00.0801241336510.5731@racer.site","threadId":"11717","inReplyTo":"20080124125606.GB13247@1wt.eu","subject":"Re: Multiple working trees with GIT ?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-01-24T13:38:45Z","receivedAt":"2008-01-24T13:38:45Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 24 Jan 2008, Willy Tarreau wrote:\n\n> On Thu, Jan 24, 2008 at 11:04:42AM +0000, Johannes Schindelin wrote:\n> \n> > On Thu, 24 Jan 2008, Julian Phillips wrote:\n> > \n> > > You might want to have a look at the git-new-workdir script in \n> > > contrib, it does basically the same thing.  It's been there for \n> > > about 10 months now. It was based on an email from Junio:\n> > > \n> > > http://article.gmane.org/gmane.comp.version-control.git/41513/\n> > \n> > FWIW I have a patch to do something like that in \"git branch\" itself.\n> >\n> > > However, there are some caveats about using this approach, basically \n> > > about the fact that there is nothing stopping you from updating refs \n> > > that are currently checked out in another directory and causing \n> > > yourself all sorts of pain ... the topic has cropped up a couple of \n> > > times on the list since the script was added.\n> > \n> > I agree; maybe we should have a telltale file \n> > \"refs/heads/<bla>.checkedout\" which is heeded by \"git checkout\" and \n> > \"git branch -d/-D\", as well as update_ref() (should only update that \n> > ref when it HEAD points to it)?\n> \n> Why not generalize this into HEAD.$branch (thus limiting to one checkout \n> per branch) or HEAD.$checkoutdir ?\n\nBecause multiple working trees for the same repository will always be a \nsecond-class citizen.  And I would rather not affect the common case too \nmuch.\n\nHaving a \"lock\" file which is heeded by just a few places which are \nsupposed to update refs (thinking about it, just update_ref() should be \nenough), is at least a well-contained change.\n\nCiao,\nDscho\n"},{"id":"66478","messageId":"20080124141041.GF13247@1wt.eu","threadId":"11717","inReplyTo":"alpine.LSU.1.00.0801241336510.5731@racer.site","subject":"Re: Multiple working trees with GIT ?","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2008-01-24T14:10:41Z","receivedAt":"2008-01-24T14:10:41Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"On Thu, Jan 24, 2008 at 01:38:45PM +0000, Johannes Schindelin wrote:\n> Hi,\n> \n> On Thu, 24 Jan 2008, Willy Tarreau wrote:\n> \n> > On Thu, Jan 24, 2008 at 11:04:42AM +0000, Johannes Schindelin wrote:\n> > \n> > > On Thu, 24 Jan 2008, Julian Phillips wrote:\n> > > \n> > > > You might want to have a look at the git-new-workdir script in \n> > > > contrib, it does basically the same thing.  It's been there for \n> > > > about 10 months now. It was based on an email from Junio:\n> > > > \n> > > > http://article.gmane.org/gmane.comp.version-control.git/41513/\n> > > \n> > > FWIW I have a patch to do something like that in \"git branch\" itself.\n> > >\n> > > > However, there are some caveats about using this approach, basically \n> > > > about the fact that there is nothing stopping you from updating refs \n> > > > that are currently checked out in another directory and causing \n> > > > yourself all sorts of pain ... the topic has cropped up a couple of \n> > > > times on the list since the script was added.\n> > > \n> > > I agree; maybe we should have a telltale file \n> > > \"refs/heads/<bla>.checkedout\" which is heeded by \"git checkout\" and \n> > > \"git branch -d/-D\", as well as update_ref() (should only update that \n> > > ref when it HEAD points to it)?\n> > \n> > Why not generalize this into HEAD.$branch (thus limiting to one checkout \n> > per branch) or HEAD.$checkoutdir ?\n> \n> Because multiple working trees for the same repository will always be a \n> second-class citizen.  And I would rather not affect the common case too \n> much.\n\nOK.\n\n> Having a \"lock\" file which is heeded by just a few places which are \n> supposed to update refs (thinking about it, just update_ref() should be \n> enough), is at least a well-contained change.\n\nindeed, with the appropriate warnings/error messages, that makes a lot of sense.\n\nCheers,\nWilly\n"},{"id":"66479","messageId":"20080124145128.GA26164@fieldses.org","threadId":"11717","inReplyTo":"20080124074952.GA8793@1wt.eu","subject":"Re: Multiple working trees with GIT ?","fromName":"J. Bruce Fields","fromEmail":"bfields@fieldses.org","sentAt":"2008-01-24T14:51:28Z","receivedAt":"2008-01-24T14:51:28Z","isPatch":false,"sender":{"key":"bfields@citi.umich.edu","avatar":null},"body":"On Thu, Jan 24, 2008 at 08:49:52AM +0100, Willy Tarreau wrote:\n> Hi all,\n> \n> I'm having long thoughts about how to use GIT to manage a distro. One of\n> the aspects which comes very often is the notion of \"variant\" for a\n> packaging. For instance, the whole project could consist in a list of packages\n> with their branches, but this list may vary depending on the platform, the\n> medium, etc... I was searching how to propagate common changes withing variants\n> with the least hassle.\n> \n> I figured out that having one file list per variant will be very annoying. In\n> another project, that's already what I have and frankly, applying the same\n> change to 10 files is counter-productive. Since the lists will often be the\n> sames except for a few entries, and since most updates will be relevant to\n> all variants, I thought branches will be my best friends.\n> \n> But I would like to be able to always access file lists, without having to\n> constantly git-checkout <variant-X>.\n\nCould just read-only access with git-show be enough for your purposes?:\n\n\tgit show <branch-name>:\t\t# show top-level directory,\n\tgit show <branch-name>:lib/\t# show lib/ directory, etc....\n\tgit show <branch-name>:lib/Makefile\n\n--b.\n"}]}