{"thread":{"id":"60556","subject":"\"git overlay\" - command for overlaying branches","startedAt":"2023-11-24T16:47:12Z","lastAt":"2023-12-01T00:11:45Z","messageCount":5,"participants":["Oliver Bandel","Michal Suchánek"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"485132","messageId":"n46dskd3hudzssaam56jesxr5elzdvs3asurqvbxkgsdcqitjs@qlufclnnjpsq","threadId":"60556","inReplyTo":null,"subject":"\"git overlay\" - command for overlaying branches","fromName":"Oliver Bandel","fromEmail":"oliver@first.in-berlin.de","sentAt":"2023-11-24T16:39:12Z","receivedAt":"2023-11-24T16:47:12Z","isPatch":false,"sender":{"key":"oliver@first.in-berlin.de","avatar":null},"body":"Hello,\n\nI'm adressing the problem with files separated from the main branch(es),\nwhich currently might be (more or less) solved with either submodules or subtrees.\nI want to suggest a new command here.\n\nAs usecase-example I assume a project that has 'branch_a' and 'branch_b'\nwith some files may be identical, some different between them.\nI assume that the (classical) way the files are handled\nby belonging to those branches is intended.\n\nThen say later I want to add more files to these branches,\nbut don't want to commit them in either of these existing branches.\nInstead a branch 'branch_addons' is created, which solely contains\nfiles that are used in 'branch_a' as well as 'branch_b'.\n\nAssume there is a command 'git overlay' which overlays one branch on top of another branch,\nso that the working dir contains files of both branches (with the overlayed on-top\nhaving priority in hiding all the other files of same name in the working dir).\nThe files in the working dir are regarded as being members of the branches,\nthey are coming from initially (with the (last) overlayed branch as priority).\n\nThis, I guess, would solve the use case of the submodules/subtree problem.\n\nExample:\n\nAssuming only the files of the according branches are in the working dir.\n\nbranch_a has these files:\n  a1.txt\n  a2.txt\n  common.txt\n\n\nbranch_b has these files:\n  b1.txt\n  b2.txt\n  common.txt\n\n\nbranch_addons has these files:\n\n  addon_1.txt\n  addon_2.txt\n\n\n\nCASE_1:\nWhen sitting in 'branch_a' and typing 'git overlay branch_addons' I see these files:\n\n  a1.txt\n  a2.txt\n  common.txt\n  addon_1.txt\n  addon_2.txt\n\n\nCASE_2:\nWould I instead be sitting in 'branch_b' and typing 'git overlay branch_addons' I see these files:\n\n  b1.txt\n  b2.txt\n  common.txt\n  addon_1.txt\n  addon_2.txt\n\nChanging files and then adding/committing them would apply add/commit on the branches,\nwhere these files are comming from.\n\nThis means, in CASE_1, adding/committing changes of files from 'branch_a' would work on 'branch_a',\nadding/committing changes of files from 'branch_addons' would work on 'branch_addons'.\n\nIn CASE_2, adding/committing changes of files from 'branch_b' would work on 'branch_b',\nadding/committing changes of files from 'branch_addons' would work on 'branch_addons'.\n\n\nIf 'branch_addons' would also contain a file common.txt, then this file would be\nresiding in the working dir (hiding all other common.txt) and\nchanging/adding/committing it, would result in the 'branch_addons' common.txt\nbeing changed.\n\nA 'git unoverlay' or 'git unoverlay branch_addons' or 'git overlay --rm'\nor 'git overlay --off' would disable the overlay, which means only the main branch\n(here 'branch_a' or 'branch_b') would be checked out.\n\nA switch of the branch (checkout/switch) might automatically un-overlay all of\nthe overlayed branches, or might be forbidden until an explicit un-overlay\nwould be done. (maybe configurable behaviour)\n\nI assume that more than one overlay would be possible.\nOf course the order of applying the overlays determenines, what branches the\nfiles in the working dir are associated with (and checked out from).\n\n\nOther use cases of course could use programming files.\nThe main branch might contain some interface files (*.h, *.mli, ...)\nand the overlays the implementation (*.c, *.ml, ...).\n\nThen switching the implementation for the same interface\n(assuming no interface files are in the overlay-branches)\nwould just be one git command.\n\n\nAdditionally it would be good to have an easy way to move a file from one branch to another\nbranch (for example from one of the main branches to an overlay or vice versa).\nIf common.txt in the above example does not differ between 'branch_a' and\n'branch_b', it might make sense to move it to maybe a branch 'branch_common', which\nmight be used as overlay as well (or as a base branch instead, making branhc_a and 'branch_b'\npossible candidates for being overlayed).\n\n\nI suggest considering such an overlay feature for git in the future.\nIf there are problems lurking in the background here\n(for example when considering merges, rebasing and such stuff),\nor if anything would fit in nicely, I can't see right now.\n\n\nCiao,\n  Oliver\n\nP.S.:\n\n  Instead of \"overlay\" I first had in mind \"mount\" and \"unmount\" as name,\n  but the name \"overlay\" is better choice, even though \"unoverlay\" is somewhat intricate.\n"},{"id":"485137","messageId":"20231124205945.GA9696@kitsune.suse.cz","threadId":"60556","inReplyTo":"n46dskd3hudzssaam56jesxr5elzdvs3asurqvbxkgsdcqitjs@qlufclnnjpsq","subject":"Re: \"git overlay\" - command for overlaying branches","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2023-11-24T20:59:45Z","receivedAt":"2023-11-24T20:59:50Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Fri, Nov 24, 2023 at 05:39:12PM +0100, Oliver Bandel wrote:\n> Hello,\n> \n> I'm adressing the problem with files separated from the main branch(es),\n> which currently might be (more or less) solved with either submodules or subtrees.\n> I want to suggest a new command here.\n> \n> As usecase-example I assume a project that has 'branch_a' and 'branch_b'\n> with some files may be identical, some different between them.\n> I assume that the (classical) way the files are handled\n> by belonging to those branches is intended.\n> \n> Then say later I want to add more files to these branches,\n> but don't want to commit them in either of these existing branches.\n> Instead a branch 'branch_addons' is created, which solely contains\n> files that are used in 'branch_a' as well as 'branch_b'.\n\nNow you can merge branch_addons onto branch_a and branch_b and be done.\n\nUnfortunately, git does not provide 'theirs' merge strategy while 'ours'\nis provided - somewhat asymmetrical.\n\nThanks\n\nMichal\n"},{"id":"485139","messageId":"647focpqwyild7dbmw7dloc5q2irijk7z77ymmfut5zdjrqhzy@xsle27m6flun","threadId":"60556","inReplyTo":"20231124205945.GA9696@kitsune.suse.cz","subject":"Re: \"git overlay\" - command for overlaying branches","fromName":"Oliver Bandel","fromEmail":"oliver@first.in-berlin.de","sentAt":"2023-11-24T23:51:06Z","receivedAt":"2023-11-24T23:51:09Z","isPatch":false,"sender":{"key":"oliver@first.in-berlin.de","avatar":null},"body":"Quoting  Michal Suchánek <msuchanek@suse.de> (snt: 2023-11-24 21:59 +0100 CET) (rcv: 2023-11-24 21:59 +0100 CET):\n> On Fri, Nov 24, 2023 at 05:39:12PM +0100, Oliver Bandel wrote:\n> > Hello,\n> > \n> > I'm adressing the problem with files separated from the main branch(es),\n> > which currently might be (more or less) solved with either submodules or subtrees.\n> > I want to suggest a new command here.\n> > \n> > As usecase-example I assume a project that has 'branch_a' and 'branch_b'\n> > with some files may be identical, some different between them.\n> > I assume that the (classical) way the files are handled\n> > by belonging to those branches is intended.\n> > \n> > Then say later I want to add more files to these branches,\n> > but don't want to commit them in either of these existing branches.\n> > Instead a branch 'branch_addons' is created, which solely contains\n> > files that are used in 'branch_a' as well as 'branch_b'.\n> \n> Now you can merge branch_addons onto branch_a and branch_b and be done.\n\nMaybe my example or the explanation was not clear enough or stopped to early.\n\nAfter a merge, I may have added files from one branch to another branch\nand some files might be changed during the merge operation and this is\npersistent then.\n\nBut the overlay would be a temporal situation and so no \"pollution\" with files from\nother branches would occur and no changes of the files with same names\n(some were just temporarily hidden).\n\nIn an overlay situation, all involved branches are checked out into the\nworking dir (the non-hidden files are checked out) at the same time.\nAll changes (change/add/commit) could then be done in one go (with 'git\nadd' and 'git commit' as in the usual way), and all commits will only\naffect those branches where the changed files are comming from.\nThis means editing more than one branch at the same time,\ncommitting to more than one branch at the same time,\nbut not merging them.\n\nAfter un-overloading, the temporarily overlayed files from other\nbranches will be removed from the working dir by git.\nNo merge has been done, but changes to more than one branch might have\nhappened.\n\nI hope it's more clear now, how that differs from normal merge-branches situation.\n\nCiao,\n  Oliver\n\nP.S.: Changing sources and tests together but only publishing the code would be\n      easy this way.\n      Just check out the sources, overlay the tests, do the work, commit\n      the changes and after un-overlaying the sources branch has no\n      \"tests pollution\", and no other tricks are needed then.\n\n      Also useful, when working with LaTeX documents and you temporarily\n      want to change the included extra files (or use the other layoutet\n      old version together with the current one) from a different\n      branch, but don't want the changes be permanently in your\n      currently preferred main branch (nevertheless change the main-doc\n      and the extra files in one go).\n\n      For those people who commit their passwords or private keys into git,\n      this also could increase security ;-)\n      For testing the code, the secrets are just overlayed.\n"},{"id":"485145","messageId":"20231125105046.GB9696@kitsune.suse.cz","threadId":"60556","inReplyTo":"647focpqwyild7dbmw7dloc5q2irijk7z77ymmfut5zdjrqhzy@xsle27m6flun","subject":"Re: \"git overlay\" - command for overlaying branches","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2023-11-25T10:50:46Z","receivedAt":"2023-11-25T10:50:50Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Sat, Nov 25, 2023 at 12:51:06AM +0100, Oliver Bandel wrote:\n> Quoting  Michal Suchánek <msuchanek@suse.de> (snt: 2023-11-24 21:59 +0100 CET) (rcv: 2023-11-24 21:59 +0100 CET):\n> > On Fri, Nov 24, 2023 at 05:39:12PM +0100, Oliver Bandel wrote:\n> > > Hello,\n> > > \n> > > I'm adressing the problem with files separated from the main branch(es),\n> > > which currently might be (more or less) solved with either submodules or subtrees.\n> > > I want to suggest a new command here.\n> > > \n> > > As usecase-example I assume a project that has 'branch_a' and 'branch_b'\n> > > with some files may be identical, some different between them.\n> > > I assume that the (classical) way the files are handled\n> > > by belonging to those branches is intended.\n> > > \n> > > Then say later I want to add more files to these branches,\n> > > but don't want to commit them in either of these existing branches.\n> > > Instead a branch 'branch_addons' is created, which solely contains\n> > > files that are used in 'branch_a' as well as 'branch_b'.\n> > \n> > Now you can merge branch_addons onto branch_a and branch_b and be done.\n> \n> Maybe my example or the explanation was not clear enough or stopped to early.\n> \n> After a merge, I may have added files from one branch to another branch\n> and some files might be changed during the merge operation and this is\n> persistent then.\n> \n> But the overlay would be a temporal situation and so no \"pollution\" with files from\n> other branches would occur and no changes of the files with same names\n> (some were just temporarily hidden).\n> \n> In an overlay situation, all involved branches are checked out into the\n> working dir (the non-hidden files are checked out) at the same time.\n> All changes (change/add/commit) could then be done in one go (with 'git\n> add' and 'git commit' as in the usual way), and all commits will only\n> affect those branches where the changed files are comming from.\n> This means editing more than one branch at the same time,\n> committing to more than one branch at the same time,\n> but not merging them.\n> \n> After un-overloading, the temporarily overlayed files from other\n> branches will be removed from the working dir by git.\n> No merge has been done, but changes to more than one branch might have\n> happened.\n> \n> I hope it's more clear now, how that differs from normal merge-branches situation.\n\nIt's not clear why would anyone do that, though.\n\n> \n> Ciao,\n>   Oliver\n> \n> P.S.: Changing sources and tests together but only publishing the code would be\n>       easy this way.\n\nAnd why would you do that?\n\nEven if you wanted the git archive has option to exclude directories so\nyou could publish separate releases from one branch.\n\n>       Just check out the sources, overlay the tests, do the work, commit\n>       the changes and after un-overlaying the sources branch has no\n>       \"tests pollution\", and no other tricks are needed then.\n\nSince you detail how 'do the work' would commit the changes to both\nbranches at the same time that means that specific revision of the code\nis tied to specific revision of the tests, different revisions would not\nwork together. If the overlay branch is used with multiple different\nbranches it will diverge, and will be compatible only with one of them\nat any time.\n\nMerging a specific revision of the overlay branch solves this problem.\nIt ties the correct revision of the shared branch together with the\ncorrect revision of the separate files.\n\nIt's the same with the interface-implementation example you gave\nearlier.\n\n>       Also useful, when working with LaTeX documents and you temporarily\n>       want to change the included extra files (or use the other layoutet\n>       old version together with the current one) from a different\n>       branch, but don't want the changes be permanently in your\n>       currently preferred main branch (nevertheless change the main-doc\n>       and the extra files in one go).\n\nDon't really understand what you are trying to do here.\n\n>       For those people who commit their passwords or private keys into git,\n>       this also could increase security ;-)\n>       For testing the code, the secrets are just overlayed.\n\nOr not put those into project git in the first place. When the secrets\nare in an 'overlay branch' in the same repository they are published\nanyway.\n\nThanks\n\nMichal\n"},{"id":"485295","messageId":"6wyjngq772jjy5hbydegblbrviuvama4avyxof2olkjpey3ohi@drpwcb2eothu","threadId":"60556","inReplyTo":"20231125105046.GB9696@kitsune.suse.cz","subject":"Re: \"git overlay\" - command for overlaying branches","fromName":"Oliver Bandel","fromEmail":"oliver@first.in-berlin.de","sentAt":"2023-12-01T00:11:41Z","receivedAt":"2023-12-01T00:11:45Z","isPatch":false,"sender":{"key":"oliver@first.in-berlin.de","avatar":null},"body":"What I basically want to do is:\n\n- Working on more than one branch at the same time.\n- Having all checked-out branches together in one working dir.\n- When adding/committing, all checked-out files will be added/committed\n  to those branches, from where they were checked out.\n\nThis way I could edit multiple branches simultaneously independently without the need of merging.\nMerging can be done later on, if needed.\n\nPossible alternatives?\n\nFor the multiple checkouts I could do a workaround and use a local bare-repo\nand check out each branch of interest into a seperate directory.\nI could then edit the files from the different dirs.\n\nBut for the overlaying thing, to get the files into one dir\n(with the automatic hiding of equally named files from branches that\nwere overlayed before) it would need something like a FUSE-based\ntool, so that all the branches could be blended into one working dir.\n\nA \"git overlay\" would make this easier.\nNo extra bare-repo needed, no additional tool for blending the dirs\nneeded.\n\nCiao,\n  Oliver\n"}]}