{"thread":{"id":"51169","subject":"Need help merging unrelated histories","startedAt":"2019-05-24T14:05:11Z","lastAt":"2019-05-29T13:56:07Z","messageCount":7,"participants":["Robert Dailey","Ævar Arnfjörð Bjarmason","Andreas Schwab","Philip Oakley"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"376190","messageId":"CAHd499AidrL4jCG_WytPJF9yHfJJJvWJ5fpjKN6sGBY5Weu8Hg@mail.gmail.com","threadId":"51169","inReplyTo":null,"subject":"Need help merging unrelated histories","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2019-05-24T14:04:57Z","receivedAt":"2019-05-24T14:05:11Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"Everything I'm going to describe is related to this repository:\n\nhttps://github.com/powervr-graphics/Native_SDK\n\nThis repo has several distinct branches. None of them seem to be tied\nto each other. Instead of having a `master` where they branched off\neach of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a\ncopy of previous branches with no ancestry and then continued coding\non that.\n\nWhat I'm trying to do is go back and rebase all of the X.X branches\nonto a new `master` branch. I started off with this:\n\n$ git checkout --orphan master\n\nThen I committed a `.gitattributes`. Next, I rebased the first\n(oldest) release branch:\n\n$ git rebase -i --onto master origin/3.0 --root\n\nThen I merged:\n\n$ git checkout master && git merge --no-ff -\n\nNext, I did 3.1:\n\n$ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs\n\nUsing interactive mode, Git is smart enough to detect duplicate\ncommits and eliminates those, even though the 2 branches do not share\na merge base. I continued doing this all the way up to rebasing 4.3,\nbut that's when things got tough. I see a lot of 'UD', 'UA', 'AU',\n'AA' merge conflicts. These are obviously due to the fact that the\nbranches aren't connected. But I expected was that `-X theirs` would\nalways favor what's on the branch being rebased. However, it seems\nthis only affects modified conflicts, not adds or deletes.\n\nI was trying to find a way to bulk-resolve these. I mean, if git sees\na file added on the left AND the right, I want the right one (theirs;\nthe one coming from the 4.3 branch). Even though the branches are\nunrelated in terms of their history, I want the net effect of the\nrebase to essentially reflect the files on 4.3 itself. If a file isn't\npresent on HEAD, delete it. If a file exists on REBASE_HEAD but not on\nHEAD, then add it. If the same file exists on both, favor the one on\nREBASE_HEAD.\n\nBut I don't see a way of doing that. I tried `git checkout --theirs .`\nand `git checkout REBASE_HEAD -- .` but this doesn't work with all\nconflict types.\n\nCan anyone provide some advice on how to properly restructure this\nrepository to create some ancestry, as if all along a `master` existed\nand all release branches were based on this in a linear fashion?\n"},{"id":"376191","messageId":"CAHd499ArS4h=V=407D70uXhtAguvVmcogHVS3J199ECJAhgfgw@mail.gmail.com","threadId":"51169","inReplyTo":"CAHd499AidrL4jCG_WytPJF9yHfJJJvWJ5fpjKN6sGBY5Weu8Hg@mail.gmail.com","subject":"Re: Need help merging unrelated histories","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2019-05-24T14:08:13Z","receivedAt":"2019-05-24T14:08:28Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Fri, May 24, 2019 at 9:04 AM Robert Dailey wrote:\n>\n> Everything I'm going to describe is related to this repository:\n>\n> https://github.com/powervr-graphics/Native_SDK\n>\n> This repo has several distinct branches. None of them seem to be tied\n> to each other. Instead of having a `master` where they branched off\n> each of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a\n> copy of previous branches with no ancestry and then continued coding\n> on that.\n>\n> What I'm trying to do is go back and rebase all of the X.X branches\n> onto a new `master` branch. I started off with this:\n>\n> $ git checkout --orphan master\n>\n> Then I committed a `.gitattributes`. Next, I rebased the first\n> (oldest) release branch:\n>\n> $ git rebase -i --onto master origin/3.0 --root\n>\n> Then I merged:\n>\n> $ git checkout master && git merge --no-ff -\n>\n> Next, I did 3.1:\n>\n> $ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs\n>\n> Using interactive mode, Git is smart enough to detect duplicate\n> commits and eliminates those, even though the 2 branches do not share\n> a merge base. I continued doing this all the way up to rebasing 4.3,\n> but that's when things got tough. I see a lot of 'UD', 'UA', 'AU',\n> 'AA' merge conflicts. These are obviously due to the fact that the\n> branches aren't connected. But I expected was that `-X theirs` would\n> always favor what's on the branch being rebased. However, it seems\n> this only affects modified conflicts, not adds or deletes.\n>\n> I was trying to find a way to bulk-resolve these. I mean, if git sees\n> a file added on the left AND the right, I want the right one (theirs;\n> the one coming from the 4.3 branch). Even though the branches are\n> unrelated in terms of their history, I want the net effect of the\n> rebase to essentially reflect the files on 4.3 itself. If a file isn't\n> present on HEAD, delete it. If a file exists on REBASE_HEAD but not on\n> HEAD, then add it. If the same file exists on both, favor the one on\n> REBASE_HEAD.\n>\n> But I don't see a way of doing that. I tried `git checkout --theirs .`\n> and `git checkout REBASE_HEAD -- .` but this doesn't work with all\n> conflict types.\n>\n> Can anyone provide some advice on how to properly restructure this\n> repository to create some ancestry, as if all along a `master` existed\n> and all release branches were based on this in a linear fashion?\n\nBy the way, everything I've done so far (up to 4.2) is located here:\n\nhttps://github.com/rcdailey/Native_SDK\n\nThere is a `master` branch in my fork of that repo. I am not sure if\nI've done it correctly so far, or if how I've done this up to 4.2\nexplains why 4.3 is so hard.\n"},{"id":"376192","messageId":"87blzrgbjj.fsf@evledraar.gmail.com","threadId":"51169","inReplyTo":"CAHd499AidrL4jCG_WytPJF9yHfJJJvWJ5fpjKN6sGBY5Weu8Hg@mail.gmail.com","subject":"Re: Need help merging unrelated histories","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-05-24T14:41:20Z","receivedAt":"2019-05-24T14:41:25Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, May 24 2019, Robert Dailey wrote:\n\n> Everything I'm going to describe is related to this repository:\n>\n> https://github.com/powervr-graphics/Native_SDK\n>\n> This repo has several distinct branches. None of them seem to be tied\n> to each other. Instead of having a `master` where they branched off\n> each of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a\n> copy of previous branches with no ancestry and then continued coding\n> on that.\n>\n> What I'm trying to do is go back and rebase all of the X.X branches\n> onto a new `master` branch. I started off with this:\n>\n> $ git checkout --orphan master\n>\n> Then I committed a `.gitattributes`. Next, I rebased the first\n> (oldest) release branch:\n>\n> $ git rebase -i --onto master origin/3.0 --root\n>\n> Then I merged:\n>\n> $ git checkout master && git merge --no-ff -\n>\n> Next, I did 3.1:\n>\n> $ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs\n>\n> Using interactive mode, Git is smart enough to detect duplicate\n> commits and eliminates those, even though the 2 branches do not share\n> a merge base. I continued doing this all the way up to rebasing 4.3,\n> but that's when things got tough. I see a lot of 'UD', 'UA', 'AU',\n> 'AA' merge conflicts. These are obviously due to the fact that the\n> branches aren't connected. But I expected was that `-X theirs` would\n> always favor what's on the branch being rebased. However, it seems\n> this only affects modified conflicts, not adds or deletes.\n>\n> I was trying to find a way to bulk-resolve these. I mean, if git sees\n> a file added on the left AND the right, I want the right one (theirs;\n> the one coming from the 4.3 branch). Even though the branches are\n> unrelated in terms of their history, I want the net effect of the\n> rebase to essentially reflect the files on 4.3 itself. If a file isn't\n> present on HEAD, delete it. If a file exists on REBASE_HEAD but not on\n> HEAD, then add it. If the same file exists on both, favor the one on\n> REBASE_HEAD.\n>\n> But I don't see a way of doing that. I tried `git checkout --theirs .`\n> and `git checkout REBASE_HEAD -- .` but this doesn't work with all\n> conflict types.\n>\n> Can anyone provide some advice on how to properly restructure this\n> repository to create some ancestry, as if all along a `master` existed\n> and all release branches were based on this in a linear fashion?\n\nWe don't have a merge strategy to do this, but should. I had a WIP patch\nfor this here that I haven't picked up:\nhttps://public-inbox.org/git/87sh7sdtc1.fsf@evledraar.gmail.com/\n\nBut you can emulate the same thing with \"read-tree\" and\n\"commit\". I.e. manually craft a commit with plumbing that diverges from\nthe \"master\" branch, then instead of stock \"rebase\" write some loop\nwhere you keep using \"read-tree\" to read what you want to stage into the\nindex from the existing branched commits, then the equivalent of \"git\ncommit -c\" etc. to commit it.\n"},{"id":"376194","messageId":"87ftp3lqvw.fsf@igel.home","threadId":"51169","inReplyTo":"CAHd499AidrL4jCG_WytPJF9yHfJJJvWJ5fpjKN6sGBY5Weu8Hg@mail.gmail.com","subject":"Re: Need help merging unrelated histories","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2019-05-24T17:10:59Z","receivedAt":"2019-05-24T17:11:04Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"On Mai 24 2019, Robert Dailey wrote:\n\n> Can anyone provide some advice on how to properly restructure this\n> repository to create some ancestry, as if all along a `master` existed\n> and all release branches were based on this in a linear fashion?\n\nHow about using git replace --graft, then git filter-branch to make it\npermanent?\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1\n\"And now for something completely different.\"\n"},{"id":"376300","messageId":"CAHd499BQO8E99Y4=T9V6xXDuDyHdqMJ_sQM4DTraAxLzKf+Z-Q@mail.gmail.com","threadId":"51169","inReplyTo":"87ftp3lqvw.fsf@igel.home","subject":"Re: Need help merging unrelated histories","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2019-05-28T13:35:55Z","receivedAt":"2019-05-28T13:36:10Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Fri, May 24, 2019 at 12:11 PM Andreas Schwab wrote:\n>\n> On Mai 24 2019, Robert Dailey wrote:\n>\n> > Can anyone provide some advice on how to properly restructure this\n> > repository to create some ancestry, as if all along a `master` existed\n> > and all release branches were based on this in a linear fashion?\n>\n> How about using git replace --graft, then git filter-branch to make it\n> permanent?\n\nI unfortunately have little-to-no experience with low level plumbing\ncommands. Could you provide an example of a series of commands to run?\nIt will help me to figure out how to do it on my own for my specific\nuse case. Thanks in advance.\n"},{"id":"376336","messageId":"CAHd499CZnhzw+sKM_Nx4_SvUT5YWaLWr9SmBKCkOcH4=wSdaqA@mail.gmail.com","threadId":"51169","inReplyTo":"CAHd499BQO8E99Y4=T9V6xXDuDyHdqMJ_sQM4DTraAxLzKf+Z-Q@mail.gmail.com","subject":"Re: Need help merging unrelated histories","fromName":"Robert Dailey","fromEmail":"rcdailey.lists@gmail.com","sentAt":"2019-05-28T18:26:34Z","receivedAt":"2019-05-28T18:26:49Z","isPatch":false,"sender":{"key":"rcdailey.lists@gmail.com","avatar":null},"body":"On Tue, May 28, 2019 at 8:35 AM Robert Dailey wrote:\n>\n> On Fri, May 24, 2019 at 12:11 PM Andreas Schwab wrote:\n> >\n> > On Mai 24 2019, Robert Dailey wrote:\n> >\n> > > Can anyone provide some advice on how to properly restructure this\n> > > repository to create some ancestry, as if all along a `master` existed\n> > > and all release branches were based on this in a linear fashion?\n> >\n> > How about using git replace --graft, then git filter-branch to make it\n> > permanent?\n>\n> I unfortunately have little-to-no experience with low level plumbing\n> commands. Could you provide an example of a series of commands to run?\n> It will help me to figure out how to do it on my own for my specific\n> use case. Thanks in advance.\n\nToyed around with it a bit, and tried with the script below. There\nwere no errors, but I noticed that it erased my `.gitattributes` file\non master that I committed before performing the graft. Why did it\nremove the file? I assume I'm not doing this correctly. Please advise.\nThank you.\n\n```sh\n#!/usr/bin/env bash\nset -ex\n\nif [[ ! -d Native_SDK.git ]]; then\n git clone --bare git@github.com:powervr-graphics/Native_SDK.git\nelse\n cd Native_SDK.git\n git fetch\n cd -\nfi\n\nrm -rf test_repo\ngit clone Native_SDK.git test_repo\ncd test_repo\n\ngit checkout -f --orphan new_master\necho '* text=auto' > .gitattributes\ngit add .gitattributes\ngit commit -m 'Add gitattributes file'\n\nc=$(git log --oneline origin/3.4 | tail -1 | cut -d ' ' -f 1)\ngit replace --graft $c new_master\ngit filter-branch HEAD..origin/3.4\ngit merge --no-edit --no-ff origin/3.4\n```\n"},{"id":"376399","messageId":"4e1ab4ad-f883-263a-2826-5a79bf75494b@iee.org","threadId":"51169","inReplyTo":"CAHd499CZnhzw+sKM_Nx4_SvUT5YWaLWr9SmBKCkOcH4=wSdaqA@mail.gmail.com","subject":"Re: Need help merging unrelated histories","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2019-05-29T13:56:03Z","receivedAt":"2019-05-29T13:56:07Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"Hi Robert,\nA few comments to look at, which may help.\nOn 28/05/2019 19:26, Robert Dailey wrote:\n> On Tue, May 28, 2019 at 8:35 AM Robert Dailey wrote:\n>> On Fri, May 24, 2019 at 12:11 PM Andreas Schwab wrote:\n>>> On Mai 24 2019, Robert Dailey wrote:\n>>>\n>>>> Can anyone provide some advice on how to properly restructure this\n>>>> repository to create some ancestry, as if all along a `master` existed\n>>>> and all release branches were based on this in a linear fashion?\n>>> How about using git replace --graft, then git filter-branch to make it\n>>> permanent?\n>> I unfortunately have little-to-no experience with low level plumbing\n>> commands. Could you provide an example of a series of commands to run?\n>> It will help me to figure out how to do it on my own for my specific\n>> use case. Thanks in advance.\n> Toyed around with it a bit, and tried with the script below. There\n> were no errors, but I noticed that it erased my `.gitattributes` file\n> on master that I committed before performing the graft. Why did it\n> remove the file? I assume I'm not doing this correctly. Please advise.\n> Thank you.\n>\n> ```sh\n> #!/usr/bin/env bash\n> set -ex\n>\n> if [[ ! -d Native_SDK.git ]]; then\n> git clone --bare git@github.com:powervr-graphics/Native_SDK.git\n> else\n> cd Native_SDK.git\n> git fetch\n> cd -\n> fi\n>\n> rm -rf test_repo\n> git clone Native_SDK.git test_repo\n> cd test_repo\n>\n> git checkout -f --orphan new_master\nCheck here what is in the files system and index - see the caveat in the \nman page for the --orphan start point and potential use of `git rm -rf .`\n> echo '* text=auto' > .gitattributes\nMaybe also add some mega comment line to really see it..\n> git add .gitattributes\n> git commit -m 'Add gitattributes file'\nCheck what is actually committed, in case you have more/less than expected.\n> c=$(git log --oneline origin/3.4 | tail -1 | cut -d ' ' -f 1)\nIs this the one you expected (maybe gitk viewer )\n> git replace --graft $c new_master\nTyping \"git replace\" without arguments, also lists all replace refs. - \ncheck what the new one is. Investigate.\n> git filter-branch HEAD..origin/3.4\nHmm is this the right range for re-write (not checked, just feels as if \nit's wrong). Maybe also need a local branch name for origin/3.4, just \nfor comparing.\n> git merge --no-edit --no-ff origin/3.4\n> ```\nsame again\n\nHopefully the random thoughts may prompt something.\nPhilip\n"}]}