Possible issue with rebase's --rebase-merges option

> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
> 
> What did you do before the bug happened? (Steps to reproduce your issue)
> This is a difficult one to give proper steps to reproduce. The issue
> is with rebase's --rebase-merges flag. We recently switched from using
> rebase with the --preserve-merges option to --rebase-merges. Most of
> the time the output is the same, but sometimes it is very different.
> I'm unable to determine whether this is by design or a bug.

> What did you expect to happen? (Expected behavior)
> Resulting graph after running rebase --rebase-merges is the same as
> running rebase --preserve-merges.
> 
> What happened instead? (Actual behavior)
> Using --rebase merges tries to pick substantially more commits and
> results in merge commits with no parent commit when viewing log in
> reverse chronological order.
> 
> What's different between what you expected and what actually happened?
> When the issue does occur (it doesn't for all rebases) it results in
> two completely different logs and picks commits that are apparently
> not part of the branch being rebased. eg, for a branch with 128
> commits including merges, --preserve-merges picks 128 commits and the
> resulting topology matches the original branch's topology.
> --rebase-merges picked 183(?) commits in v2.24 and 202 commits in
> v2.27, and in both cases resulted in a very strange topology.

> I
> would love to understand if this is by design or a legitimate bug.
> 
> Please review the rest of the bug report below.
> You can delete any lines you don't wish to share.
> 
> 
> [System Info]
> git version:
> git version 2.27.0.windows.1
> cpu: x86_64
> built from commit: 907ab1011dce9112700498e034b974ba60f8b407
> sizeof-long: 4
> sizeof-size_t: 8
> uname: Windows 10.0 18363
> compiler info: gnuc: 10.1
> libc info: no libc information available
> 
> 
> [Enabled Hooks]
> 

>
> Hi Joel
>
> On 21/07/2020 22:20, Joel Marshall wrote:
> > Thank you for filling out a Git bug report!
> > Please answer the following questions to help us understand your issue.
> >
> > What did you do before the bug happened? (Steps to reproduce your issue)
> > This is a difficult one to give proper steps to reproduce. The issue
> > is with rebase's --rebase-merges flag. We recently switched from using
> > rebase with the --preserve-merges option to --rebase-merges. Most of
> > the time the output is the same, but sometimes it is very different.
> > I'm unable to determine whether this is by design or a bug.
>
> --preserve-merges is buggy if you rearrange the commits (this is way
> --rebase-merges was added), if you're just rebasing without reordering
> anything then I think the result should be the same though there may be
> some corner cases I'm not aware of. Are you able to share the topology
> before rebasing and after with --preserve-merges and --rebase-merges?
>
> > What did you expect to happen? (Expected behavior)
> > Resulting graph after running rebase --rebase-merges is the same as
> > running rebase --preserve-merges.
> >
> > What happened instead? (Actual behavior)
> > Using --rebase merges tries to pick substantially more commits and
> > results in merge commits with no parent commit when viewing log in
> > reverse chronological order.
> >
> > What's different between what you expected and what actually happened?
> > When the issue does occur (it doesn't for all rebases) it results in
> > two completely different logs and picks commits that are apparently
> > not part of the branch being rebased. eg, for a branch with 128
> > commits including merges, --preserve-merges picks 128 commits and the
> > resulting topology matches the original branch's topology.
> > --rebase-merges picked 183(?) commits in v2.24 and 202 commits in
> > v2.27, and in both cases resulted in a very strange topology.
>
> That's interesting there were some changes to how empty commits and
> upstreamed commits are handled between v2.24 and v2.17, without seeing
> the staring point and the results it's hard to tell what is going on though.
>
> Without seeing some examples it's hard to tell if there is a bug here or
> not though it does sound a bit suspicious.
>
> > Anything else you want to add:
> > Feel free to contact me at joel@tusksoft.com for additional details.
>
> You email me directly if there are things you don't want to share on the
> list
>
> Best wishes
>
> Phillip
>
> > I
> > would love to understand if this is by design or a legitimate bug.
> >
> > Please review the rest of the bug report below.
> > You can delete any lines you don't wish to share.
> >
> >
> > [System Info]
> > git version:
> > git version 2.27.0.windows.1
> > cpu: x86_64
> > built from commit: 907ab1011dce9112700498e034b974ba60f8b407
> > sizeof-long: 4
> > sizeof-size_t: 8
> > uname: Windows 10.0 18363
> > compiler info: gnuc: 10.1
> > libc info: no libc information available
> >
> >
> > [Enabled Hooks]
> >

> I've attached a couple of screenshots from tortoisegit. The branches
> are too long to show in their entirety, but this should give you an
> idea of what I'm talking about. The highlighted commit is the head of
> the branch that I'm rebasing onto. Interestingly, I just noticed that
> while --rebase merges reports that it's rebasing 202 commits
> intitially, upon manually reconciling the first conflict it reports
> that it is rebasing 183 commits.

> On Wed, Jul 22, 2020 at 10:22 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>>
>> Hi Joel
>>
>> On 21/07/2020 22:20, Joel Marshall wrote:
>>> Thank you for filling out a Git bug report!
>>> Please answer the following questions to help us understand your issue.
>>>
>>> What did you do before the bug happened? (Steps to reproduce your issue)
>>> This is a difficult one to give proper steps to reproduce. The issue
>>> is with rebase's --rebase-merges flag. We recently switched from using
>>> rebase with the --preserve-merges option to --rebase-merges. Most of
>>> the time the output is the same, but sometimes it is very different.
>>> I'm unable to determine whether this is by design or a bug.
>>
>> --preserve-merges is buggy if you rearrange the commits (this is way
>> --rebase-merges was added), if you're just rebasing without reordering
>> anything then I think the result should be the same though there may be
>> some corner cases I'm not aware of. Are you able to share the topology
>> before rebasing and after with --preserve-merges and --rebase-merges?
>>
>>> What did you expect to happen? (Expected behavior)
>>> Resulting graph after running rebase --rebase-merges is the same as
>>> running rebase --preserve-merges.
>>>
>>> What happened instead? (Actual behavior)
>>> Using --rebase merges tries to pick substantially more commits and
>>> results in merge commits with no parent commit when viewing log in
>>> reverse chronological order.
>>>
>>> What's different between what you expected and what actually happened?
>>> When the issue does occur (it doesn't for all rebases) it results in
>>> two completely different logs and picks commits that are apparently
>>> not part of the branch being rebased. eg, for a branch with 128
>>> commits including merges, --preserve-merges picks 128 commits and the
>>> resulting topology matches the original branch's topology.
>>> --rebase-merges picked 183(?) commits in v2.24 and 202 commits in
>>> v2.27, and in both cases resulted in a very strange topology.
>>
>> That's interesting there were some changes to how empty commits and
>> upstreamed commits are handled between v2.24 and v2.17, without seeing
>> the staring point and the results it's hard to tell what is going on though.
>>
>> Without seeing some examples it's hard to tell if there is a bug here or
>> not though it does sound a bit suspicious.
>>
>>> Anything else you want to add:
>>> Feel free to contact me at joel@tusksoft.com for additional details.
>>
>> You email me directly if there are things you don't want to share on the
>> list
>>
>> Best wishes
>>
>> Phillip
>>
>>> I
>>> would love to understand if this is by design or a legitimate bug.
>>>
>>> Please review the rest of the bug report below.
>>> You can delete any lines you don't wish to share.
>>>
>>>
>>> [System Info]
>>> git version:
>>> git version 2.27.0.windows.1
>>> cpu: x86_64
>>> built from commit: 907ab1011dce9112700498e034b974ba60f8b407
>>> sizeof-long: 4
>>> sizeof-size_t: 8
>>> uname: Windows 10.0 18363
>>> compiler info: gnuc: 10.1
>>> libc info: no libc information available
>>>
>>>
>>> [Enabled Hooks]
>>>

>
> Hi Joel
>
> On 22/07/2020 18:09, Joel Marshall wrote:
> > I've attached a couple of screenshots from tortoisegit. The branches
> > are too long to show in their entirety, but this should give you an
> > idea of what I'm talking about. The highlighted commit is the head of
> > the branch that I'm rebasing onto. Interestingly, I just noticed that
> > while --rebase merges reports that it's rebasing 202 commits
> > intitially, upon manually reconciling the first conflict it reports
> > that it is rebasing 183 commits.
>
> Thanks, the rebased topologies are certainly quite different, what is
> the topology before the rebase? Looking at the --rebase-merges result it
> looks like the second parents of merges that are being rebased have
> ancestors in the upstream branch. If that is the case then I think it is
> working as intended.
>
>  Are you able to run
>
>   git log --format=%ad --graph $upstream..HEAD
>
> before and after the rebase to check that?
>
> Best Wishes
>
> Phillip
>
> > On Wed, Jul 22, 2020 at 10:22 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
> >>
> >> Hi Joel
> >>
> >> On 21/07/2020 22:20, Joel Marshall wrote:
> >>> Thank you for filling out a Git bug report!
> >>> Please answer the following questions to help us understand your issue.
> >>>
> >>> What did you do before the bug happened? (Steps to reproduce your issue)
> >>> This is a difficult one to give proper steps to reproduce. The issue
> >>> is with rebase's --rebase-merges flag. We recently switched from using
> >>> rebase with the --preserve-merges option to --rebase-merges. Most of
> >>> the time the output is the same, but sometimes it is very different.
> >>> I'm unable to determine whether this is by design or a bug.
> >>
> >> --preserve-merges is buggy if you rearrange the commits (this is way
> >> --rebase-merges was added), if you're just rebasing without reordering
> >> anything then I think the result should be the same though there may be
> >> some corner cases I'm not aware of. Are you able to share the topology
> >> before rebasing and after with --preserve-merges and --rebase-merges?
> >>
> >>> What did you expect to happen? (Expected behavior)
> >>> Resulting graph after running rebase --rebase-merges is the same as
> >>> running rebase --preserve-merges.
> >>>
> >>> What happened instead? (Actual behavior)
> >>> Using --rebase merges tries to pick substantially more commits and
> >>> results in merge commits with no parent commit when viewing log in
> >>> reverse chronological order.
> >>>
> >>> What's different between what you expected and what actually happened?
> >>> When the issue does occur (it doesn't for all rebases) it results in
> >>> two completely different logs and picks commits that are apparently
> >>> not part of the branch being rebased. eg, for a branch with 128
> >>> commits including merges, --preserve-merges picks 128 commits and the
> >>> resulting topology matches the original branch's topology.
> >>> --rebase-merges picked 183(?) commits in v2.24 and 202 commits in
> >>> v2.27, and in both cases resulted in a very strange topology.
> >>
> >> That's interesting there were some changes to how empty commits and
> >> upstreamed commits are handled between v2.24 and v2.17, without seeing
> >> the staring point and the results it's hard to tell what is going on though.
> >>
> >> Without seeing some examples it's hard to tell if there is a bug here or
> >> not though it does sound a bit suspicious.
> >>
> >>> Anything else you want to add:
> >>> Feel free to contact me at joel@tusksoft.com for additional details.
> >>
> >> You email me directly if there are things you don't want to share on the
> >> list
> >>
> >> Best wishes
> >>
> >> Phillip
> >>
> >>> I
> >>> would love to understand if this is by design or a legitimate bug.
> >>>
> >>> Please review the rest of the bug report below.
> >>> You can delete any lines you don't wish to share.
> >>>
> >>>
> >>> [System Info]
> >>> git version:
> >>> git version 2.27.0.windows.1
> >>> cpu: x86_64
> >>> built from commit: 907ab1011dce9112700498e034b974ba60f8b407
> >>> sizeof-long: 4
> >>> sizeof-size_t: 8
> >>> uname: Windows 10.0 18363
> >>> compiler info: gnuc: 10.1
> >>> libc info: no libc information available
> >>>
> >>>
> >>> [Enabled Hooks]
> >>>
>

> I saved the state of the repo in a copy so I could come back to it if
> additional examples were needed but I had to clean up my live copy so
> I could get back to work. I'll get you some additional screenshots in
> the next few days. In the meantime, I'll try to give you some context
> around what I'm doing here. The parent branch is my main dev branch
> which consists of a series of clean branches and merges- the dev
> branch basically looks like what you're seeing in the
> --preserve-merges screenshot. I've also got a long running feature
> branch that branches off of dev, and it also consists of many branches
> and merges, each a subtask of the story related to the feature branch
> as a whole. Occasionally to get the feature branch up to date with the
> newest features I'll rebase the whole thing on top of dev, which
> should result in an unbroken chain of branches and merges as seen in
> the --preserve-merges screenshot. While you can't see it in the
> --rebase-merges screenshot, those merges show no ancestors when viewed
> in reverse chronological order- they just trail off into oblivion.

>
> Hi Joel,
>
> On Thu, 23 Jul 2020, Joel Marshall wrote:
>
> > I saved the state of the repo in a copy so I could come back to it if
> > additional examples were needed but I had to clean up my live copy so
> > I could get back to work. I'll get you some additional screenshots in
> > the next few days. In the meantime, I'll try to give you some context
> > around what I'm doing here. The parent branch is my main dev branch
> > which consists of a series of clean branches and merges- the dev
> > branch basically looks like what you're seeing in the
> > --preserve-merges screenshot. I've also got a long running feature
> > branch that branches off of dev, and it also consists of many branches
> > and merges, each a subtask of the story related to the feature branch
> > as a whole. Occasionally to get the feature branch up to date with the
> > newest features I'll rebase the whole thing on top of dev, which
> > should result in an unbroken chain of branches and merges as seen in
> > the --preserve-merges screenshot. While you can't see it in the
> > --rebase-merges screenshot, those merges show no ancestors when viewed
> > in reverse chronological order- they just trail off into oblivion.
>
> I could imagine that you might want to try this rebase with
> `--rebase-merges=rebase-cousins`.
>
> Otherwise, you might want to export your use case with `git fast-export
> --anonymize` so that others (such as myself) have a chance of helping you.
>
> Ciao,
> Johannes

>
> Hi all,
>
> Sorry to drop off on this for so long. I think this is still a
> possible outstanding issue, yes? If so I will work on getting you a
> copy of the repo as I did archive a copy at the state originally
> mentioned in this issue.
>
> On Mon, Aug 10, 2020 at 10:46 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > Hi Joel,
> >
> > On Thu, 23 Jul 2020, Joel Marshall wrote:
> >
> > > I saved the state of the repo in a copy so I could come back to it if
> > > additional examples were needed but I had to clean up my live copy so
> > > I could get back to work. I'll get you some additional screenshots in
> > > the next few days. In the meantime, I'll try to give you some context
> > > around what I'm doing here. The parent branch is my main dev branch
> > > which consists of a series of clean branches and merges- the dev
> > > branch basically looks like what you're seeing in the
> > > --preserve-merges screenshot. I've also got a long running feature
> > > branch that branches off of dev, and it also consists of many branches
> > > and merges, each a subtask of the story related to the feature branch
> > > as a whole. Occasionally to get the feature branch up to date with the
> > > newest features I'll rebase the whole thing on top of dev, which
> > > should result in an unbroken chain of branches and merges as seen in
> > > the --preserve-merges screenshot. While you can't see it in the
> > > --rebase-merges screenshot, those merges show no ancestors when viewed
> > > in reverse chronological order- they just trail off into oblivion.
> >
> > I could imagine that you might want to try this rebase with
> > `--rebase-merges=rebase-cousins`.
> >
> > Otherwise, you might want to export your use case with `git fast-export
> > --anonymize` so that others (such as myself) have a chance of helping you.
> >
> > Ciao,
> > Johannes

> I have confirmed that this is still an issue under certain
> circumstances. --rebase-merges works as expected if it is being used
> with simple feature branches, ie one commit after the other, no
> merges. Where things go off of the rails is when there are branches
> and merges coming off of and going into a feature branch. At that
> point using the --rebase-merges flag with rebase will create a similar
> mess to the images of the logs I attached back in July of 2022.

>
> On Wed, Dec 8, 2021 at 3:46 PM Joel Marshall <joelmdev@gmail.com> wrote:
> >
> > Hi all,
> >
> > Sorry to drop off on this for so long. I think this is still a
> > possible outstanding issue, yes? If so I will work on getting you a
> > copy of the repo as I did archive a copy at the state originally
> > mentioned in this issue.
> >
> > On Mon, Aug 10, 2020 at 10:46 AM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > Hi Joel,
> > >
> > > On Thu, 23 Jul 2020, Joel Marshall wrote:
> > >
> > > > I saved the state of the repo in a copy so I could come back to it if
> > > > additional examples were needed but I had to clean up my live copy so
> > > > I could get back to work. I'll get you some additional screenshots in
> > > > the next few days. In the meantime, I'll try to give you some context
> > > > around what I'm doing here. The parent branch is my main dev branch
> > > > which consists of a series of clean branches and merges- the dev
> > > > branch basically looks like what you're seeing in the
> > > > --preserve-merges screenshot. I've also got a long running feature
> > > > branch that branches off of dev, and it also consists of many branches
> > > > and merges, each a subtask of the story related to the feature branch
> > > > as a whole. Occasionally to get the feature branch up to date with the
> > > > newest features I'll rebase the whole thing on top of dev, which
> > > > should result in an unbroken chain of branches and merges as seen in
> > > > the --preserve-merges screenshot. While you can't see it in the
> > > > --rebase-merges screenshot, those merges show no ancestors when viewed
> > > > in reverse chronological order- they just trail off into oblivion.
> > >
> > > I could imagine that you might want to try this rebase with
> > > `--rebase-merges=rebase-cousins`.
> > >
> > > Otherwise, you might want to export your use case with `git fast-export
> > > --anonymize` so that others (such as myself) have a chance of helping you.
> > >
> > > Ciao,
> > > Johannes
>

> Hi Joel,
>
> please reply inline. What you did is called "top-posting" on this list and
> is regularly discouraged.
>
> On Tue, 22 Mar 2022, Joel Marshall wrote:
>
>> I have confirmed that this is still an issue under certain
>> circumstances. --rebase-merges works as expected if it is being used
>> with simple feature branches, ie one commit after the other, no
>> merges. Where things go off of the rails is when there are branches
>> and merges coming off of and going into a feature branch. At that
>> point using the --rebase-merges flag with rebase will create a similar
>> mess to the images of the logs I attached back in July of 2022.
> I wonder what happened to my suggestions to use
> `--rebase-merges=rebase-cousins` or `git fast-export --anonymize`. They
> seem to have faded without any echo.
>
> Ciao,
> Johannes
>

>> On Wed, Dec 8, 2021 at 3:46 PM Joel Marshall <joelmdev@gmail.com> wrote:
>>> Hi all,
>>>
>>> Sorry to drop off on this for so long. I think this is still a
>>> possible outstanding issue, yes? If so I will work on getting you a
>>> copy of the repo as I did archive a copy at the state originally
>>> mentioned in this issue.
>>>
>>> On Mon, Aug 10, 2020 at 10:46 AM Johannes Schindelin
>>> <Johannes.Schindelin@gmx.de> wrote:
>>>> Hi Joel,
>>>>
>>>> On Thu, 23 Jul 2020, Joel Marshall wrote:
>>>>
>>>>> I saved the state of the repo in a copy so I could come back to it if
>>>>> additional examples were needed but I had to clean up my live copy so
>>>>> I could get back to work. I'll get you some additional screenshots in
>>>>> the next few days. In the meantime, I'll try to give you some context
>>>>> around what I'm doing here. The parent branch is my main dev branch
>>>>> which consists of a series of clean branches and merges- the dev
>>>>> branch basically looks like what you're seeing in the
>>>>> --preserve-merges screenshot. I've also got a long running feature
>>>>> branch that branches off of dev, and it also consists of many branches
>>>>> and merges, each a subtask of the story related to the feature branch
>>>>> as a whole. Occasionally to get the feature branch up to date with the
>>>>> newest features I'll rebase the whole thing on top of dev, which
>>>>> should result in an unbroken chain of branches and merges as seen in
>>>>> the --preserve-merges screenshot. While you can't see it in the
>>>>> --rebase-merges screenshot, those merges show no ancestors when viewed
>>>>> in reverse chronological order- they just trail off into oblivion.
>>>> I could imagine that you might want to try this rebase with
>>>> `--rebase-merges=rebase-cousins`.
>>>>
>>>> Otherwise, you might want to export your use case with `git fast-export
>>>> --anonymize` so that others (such as myself) have a chance of helping you.
>>>>
>>>> Ciao,
>>>> Johannes