# Need help merging unrelated histories 7 messages from 2019-05-24 to 2019-05-29. Participants: Robert Dailey, Ævar Arnfjörð Bjarmason, Andreas Schwab, Philip Oakley. Thread: https://gitlist.dev/t/51169 ## Robert Dailey, 2019-05-24 14:04 Subject: Need help merging unrelated histories Message-ID: URL: https://gitlist.dev/e/CAHd499AidrL4jCG_WytPJF9yHfJJJvWJ5fpjKN6sGBY5Weu8Hg%40mail.gmail.com ``` Everything I'm going to describe is related to this repository: https://github.com/powervr-graphics/Native_SDK This repo has several distinct branches. None of them seem to be tied to each other. Instead of having a `master` where they branched off each of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a copy of previous branches with no ancestry and then continued coding on that. What I'm trying to do is go back and rebase all of the X.X branches onto a new `master` branch. I started off with this: $ git checkout --orphan master Then I committed a `.gitattributes`. Next, I rebased the first (oldest) release branch: $ git rebase -i --onto master origin/3.0 --root Then I merged: $ git checkout master && git merge --no-ff - Next, I did 3.1: $ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs Using interactive mode, Git is smart enough to detect duplicate commits and eliminates those, even though the 2 branches do not share a merge base. I continued doing this all the way up to rebasing 4.3, but that's when things got tough. I see a lot of 'UD', 'UA', 'AU', 'AA' merge conflicts. These are obviously due to the fact that the branches aren't connected. But I expected was that `-X theirs` would always favor what's on the branch being rebased. However, it seems this only affects modified conflicts, not adds or deletes. I was trying to find a way to bulk-resolve these. I mean, if git sees a file added on the left AND the right, I want the right one (theirs; the one coming from the 4.3 branch). Even though the branches are unrelated in terms of their history, I want the net effect of the rebase to essentially reflect the files on 4.3 itself. If a file isn't present on HEAD, delete it. If a file exists on REBASE_HEAD but not on HEAD, then add it. If the same file exists on both, favor the one on REBASE_HEAD. But I don't see a way of doing that. I tried `git checkout --theirs .` and `git checkout REBASE_HEAD -- .` but this doesn't work with all conflict types. Can anyone provide some advice on how to properly restructure this repository to create some ancestry, as if all along a `master` existed and all release branches were based on this in a linear fashion? ``` ## Robert Dailey, 2019-05-24 14:08 Subject: Re: Need help merging unrelated histories Message-ID: URL: https://gitlist.dev/e/CAHd499ArS4h%3DV%3D407D70uXhtAguvVmcogHVS3J199ECJAhgfgw%40mail.gmail.com In-Reply-To: ``` On Fri, May 24, 2019 at 9:04 AM Robert Dailey wrote: > > Everything I'm going to describe is related to this repository: > > https://github.com/powervr-graphics/Native_SDK > > This repo has several distinct branches. None of them seem to be tied > to each other. Instead of having a `master` where they branched off > each of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a > copy of previous branches with no ancestry and then continued coding > on that. > > What I'm trying to do is go back and rebase all of the X.X branches > onto a new `master` branch. I started off with this: > > $ git checkout --orphan master > > Then I committed a `.gitattributes`. Next, I rebased the first > (oldest) release branch: > > $ git rebase -i --onto master origin/3.0 --root > > Then I merged: > > $ git checkout master && git merge --no-ff - > > Next, I did 3.1: > > $ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs > > Using interactive mode, Git is smart enough to detect duplicate > commits and eliminates those, even though the 2 branches do not share > a merge base. I continued doing this all the way up to rebasing 4.3, > but that's when things got tough. I see a lot of 'UD', 'UA', 'AU', > 'AA' merge conflicts. These are obviously due to the fact that the > branches aren't connected. But I expected was that `-X theirs` would > always favor what's on the branch being rebased. However, it seems > this only affects modified conflicts, not adds or deletes. > > I was trying to find a way to bulk-resolve these. I mean, if git sees > a file added on the left AND the right, I want the right one (theirs; > the one coming from the 4.3 branch). Even though the branches are > unrelated in terms of their history, I want the net effect of the > rebase to essentially reflect the files on 4.3 itself. If a file isn't > present on HEAD, delete it. If a file exists on REBASE_HEAD but not on > HEAD, then add it. If the same file exists on both, favor the one on > REBASE_HEAD. > > But I don't see a way of doing that. I tried `git checkout --theirs .` > and `git checkout REBASE_HEAD -- .` but this doesn't work with all > conflict types. > > Can anyone provide some advice on how to properly restructure this > repository to create some ancestry, as if all along a `master` existed > and all release branches were based on this in a linear fashion? By the way, everything I've done so far (up to 4.2) is located here: https://github.com/rcdailey/Native_SDK There is a `master` branch in my fork of that repo. I am not sure if I've done it correctly so far, or if how I've done this up to 4.2 explains why 4.3 is so hard. ``` ## Ævar Arnfjörð Bjarmason, 2019-05-24 14:41 Subject: Re: Need help merging unrelated histories Message-ID: <87blzrgbjj.fsf@evledraar.gmail.com> URL: https://gitlist.dev/e/87blzrgbjj.fsf%40evledraar.gmail.com In-Reply-To: ``` On Fri, May 24 2019, Robert Dailey wrote: > Everything I'm going to describe is related to this repository: > > https://github.com/powervr-graphics/Native_SDK > > This repo has several distinct branches. None of them seem to be tied > to each other. Instead of having a `master` where they branched off > each of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a > copy of previous branches with no ancestry and then continued coding > on that. > > What I'm trying to do is go back and rebase all of the X.X branches > onto a new `master` branch. I started off with this: > > $ git checkout --orphan master > > Then I committed a `.gitattributes`. Next, I rebased the first > (oldest) release branch: > > $ git rebase -i --onto master origin/3.0 --root > > Then I merged: > > $ git checkout master && git merge --no-ff - > > Next, I did 3.1: > > $ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs > > Using interactive mode, Git is smart enough to detect duplicate > commits and eliminates those, even though the 2 branches do not share > a merge base. I continued doing this all the way up to rebasing 4.3, > but that's when things got tough. I see a lot of 'UD', 'UA', 'AU', > 'AA' merge conflicts. These are obviously due to the fact that the > branches aren't connected. But I expected was that `-X theirs` would > always favor what's on the branch being rebased. However, it seems > this only affects modified conflicts, not adds or deletes. > > I was trying to find a way to bulk-resolve these. I mean, if git sees > a file added on the left AND the right, I want the right one (theirs; > the one coming from the 4.3 branch). Even though the branches are > unrelated in terms of their history, I want the net effect of the > rebase to essentially reflect the files on 4.3 itself. If a file isn't > present on HEAD, delete it. If a file exists on REBASE_HEAD but not on > HEAD, then add it. If the same file exists on both, favor the one on > REBASE_HEAD. > > But I don't see a way of doing that. I tried `git checkout --theirs .` > and `git checkout REBASE_HEAD -- .` but this doesn't work with all > conflict types. > > Can anyone provide some advice on how to properly restructure this > repository to create some ancestry, as if all along a `master` existed > and all release branches were based on this in a linear fashion? We don't have a merge strategy to do this, but should. I had a WIP patch for this here that I haven't picked up: https://public-inbox.org/git/87sh7sdtc1.fsf@evledraar.gmail.com/ But you can emulate the same thing with "read-tree" and "commit". I.e. manually craft a commit with plumbing that diverges from the "master" branch, then instead of stock "rebase" write some loop where you keep using "read-tree" to read what you want to stage into the index from the existing branched commits, then the equivalent of "git commit -c" etc. to commit it. ``` ## Andreas Schwab, 2019-05-24 17:10 Subject: Re: Need help merging unrelated histories Message-ID: <87ftp3lqvw.fsf@igel.home> URL: https://gitlist.dev/e/87ftp3lqvw.fsf%40igel.home In-Reply-To: ``` On Mai 24 2019, Robert Dailey wrote: > Can anyone provide some advice on how to properly restructure this > repository to create some ancestry, as if all along a `master` existed > and all release branches were based on this in a linear fashion? How about using git replace --graft, then git filter-branch to make it permanent? Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ``` ## Robert Dailey, 2019-05-28 13:35 Subject: Re: Need help merging unrelated histories Message-ID: URL: https://gitlist.dev/e/CAHd499BQO8E99Y4%3DT9V6xXDuDyHdqMJ_sQM4DTraAxLzKf%2BZ-Q%40mail.gmail.com In-Reply-To: <87ftp3lqvw.fsf@igel.home> ``` On Fri, May 24, 2019 at 12:11 PM Andreas Schwab wrote: > > On Mai 24 2019, Robert Dailey wrote: > > > Can anyone provide some advice on how to properly restructure this > > repository to create some ancestry, as if all along a `master` existed > > and all release branches were based on this in a linear fashion? > > How about using git replace --graft, then git filter-branch to make it > permanent? I unfortunately have little-to-no experience with low level plumbing commands. Could you provide an example of a series of commands to run? It will help me to figure out how to do it on my own for my specific use case. Thanks in advance. ``` ## Robert Dailey, 2019-05-28 18:26 Subject: Re: Need help merging unrelated histories Message-ID: URL: https://gitlist.dev/e/CAHd499CZnhzw%2BsKM_Nx4_SvUT5YWaLWr9SmBKCkOcH4%3DwSdaqA%40mail.gmail.com In-Reply-To: ``` On Tue, May 28, 2019 at 8:35 AM Robert Dailey wrote: > > On Fri, May 24, 2019 at 12:11 PM Andreas Schwab wrote: > > > > On Mai 24 2019, Robert Dailey wrote: > > > > > Can anyone provide some advice on how to properly restructure this > > > repository to create some ancestry, as if all along a `master` existed > > > and all release branches were based on this in a linear fashion? > > > > How about using git replace --graft, then git filter-branch to make it > > permanent? > > I unfortunately have little-to-no experience with low level plumbing > commands. Could you provide an example of a series of commands to run? > It will help me to figure out how to do it on my own for my specific > use case. Thanks in advance. Toyed around with it a bit, and tried with the script below. There were no errors, but I noticed that it erased my `.gitattributes` file on master that I committed before performing the graft. Why did it remove the file? I assume I'm not doing this correctly. Please advise. Thank you. ˋˋˋsh #!/usr/bin/env bash set -ex if [[ ! -d Native_SDK.git ]]; then git clone --bare git@github.com:powervr-graphics/Native_SDK.git else cd Native_SDK.git git fetch cd - fi rm -rf test_repo git clone Native_SDK.git test_repo cd test_repo git checkout -f --orphan new_master echo '* text=auto' > .gitattributes git add .gitattributes git commit -m 'Add gitattributes file' c=$(git log --oneline origin/3.4 | tail -1 | cut -d ' ' -f 1) git replace --graft $c new_master git filter-branch HEAD..origin/3.4 git merge --no-edit --no-ff origin/3.4 ˋˋˋ ``` ## Philip Oakley, 2019-05-29 13:56 Subject: Re: Need help merging unrelated histories Message-ID: <4e1ab4ad-f883-263a-2826-5a79bf75494b@iee.org> URL: https://gitlist.dev/e/4e1ab4ad-f883-263a-2826-5a79bf75494b%40iee.org In-Reply-To: ``` Hi Robert, A few comments to look at, which may help. On 28/05/2019 19:26, Robert Dailey wrote: > On Tue, May 28, 2019 at 8:35 AM Robert Dailey wrote: >> On Fri, May 24, 2019 at 12:11 PM Andreas Schwab wrote: >>> On Mai 24 2019, Robert Dailey wrote: >>> >>>> Can anyone provide some advice on how to properly restructure this >>>> repository to create some ancestry, as if all along a `master` existed >>>> and all release branches were based on this in a linear fashion? >>> How about using git replace --graft, then git filter-branch to make it >>> permanent? >> I unfortunately have little-to-no experience with low level plumbing >> commands. Could you provide an example of a series of commands to run? >> It will help me to figure out how to do it on my own for my specific >> use case. Thanks in advance. > Toyed around with it a bit, and tried with the script below. There > were no errors, but I noticed that it erased my `.gitattributes` file > on master that I committed before performing the graft. Why did it > remove the file? I assume I'm not doing this correctly. Please advise. > Thank you. > > ˋˋˋsh > #!/usr/bin/env bash > set -ex > > if [[ ! -d Native_SDK.git ]]; then > git clone --bare git@github.com:powervr-graphics/Native_SDK.git > else > cd Native_SDK.git > git fetch > cd - > fi > > rm -rf test_repo > git clone Native_SDK.git test_repo > cd test_repo > > git checkout -f --orphan new_master Check here what is in the files system and index - see the caveat in the man page for the --orphan start point and potential use of `git rm -rf .` > echo '* text=auto' > .gitattributes Maybe also add some mega comment line to really see it.. > git add .gitattributes > git commit -m 'Add gitattributes file' Check what is actually committed, in case you have more/less than expected. > c=$(git log --oneline origin/3.4 | tail -1 | cut -d ' ' -f 1) Is this the one you expected (maybe gitk viewer ) > git replace --graft $c new_master Typing "git replace" without arguments, also lists all replace refs. - check what the new one is. Investigate. > git filter-branch HEAD..origin/3.4 Hmm is this the right range for re-write (not checked, just feels as if it's wrong). Maybe also need a local branch name for origin/3.4, just for comparing. > git merge --no-edit --no-ff origin/3.4 > ˋˋˋ same again Hopefully the random thoughts may prompt something. Philip ```