git merge and cherry-pick and duplicated commits?

I created a branch from master, did a commit (8e9fdd), then did 2 more commits (11c59c and 7024d), then did another commit (2daf23). From master, I did a commit (47bd1b) then cherry-pick'd 2 commits from the branch (11c59c and 7024d). When merged the branch into master, I see the 2 cherry-picked commits twice in the log (once from the original cherry-pick's and again from the merge).

I thought git would realize that master already had those 2 commits and not add them again when merging?

On Jan 13, 2009, at 9:40 PM, skillzero@gmail.com wrote:

Before the cherry-picks, your repository looks like this

o-o (master: 47bd1b)
  \
   o-A-B-o (branch:2daf23)

A and B are the two commits you cherry-picked (11c59c and 7024d)

After the cherry-picks, the repo looks like this:

o-o-A'-B' (master)
  \
   o-A-B-o (branch:2daf23)

A and A' are different commits. Same with B and B'. If you check the SHA1 of master at this point, it will NOT be 702fd... (B). Cherry pick creates a new commit that (as far as git is concerned) is totally unrelated.

After the merge, you get:

o-o-A'-B'-o (master)
  \       /
   o-A-B-o

Since git has no knowledge that the cherry-picked (A' B') commits are related to their originals (A B), it displays both to you. If you want, you can use the -x flag when you use "git cherry-pick" to add a line that describes the original source of the patch in the new commit which eases confusion when you look at the history, but will not stop them from being displayed.

(The reason git will still display them is that the cherry-picked commits may be different if there were conflicting changes on the branches. Also, hiding those commits would give a false view of history since the changes were actually added to the repository twice. Using gitk or "git log --graph" will show the commits on two different lines of development.)

The simplest method is to rebase branch after doing the cherry-picks. This should only be done if your branch has not been published. From after the cherry-picks:

o-o-A'-B' (master)
  \
   o-A-B-o (branch:2daf23)

"git rebase master branch" would give you

o-o-A'-B' (master)
         \
          o'-o' (branch)

Git should detect that the changes from A and B were already present in master during the rebase and skip the commits.

~~ Brian Gernhardt

On Tue, Jan 13, 2009 at 9:31 PM, Brian Gernhardt <benji@silverinsanity.com> wrote:

That's what I was somewhat disappointed by. Even though the result of the commit had a different hash, I assumed git would keep some kind of internal per-commit hash so it could tell later that two commits were the same and not re-apply them.

The problem is, by the time I wanted to do the cherry-pick, I had already committed other stuff to the branch. I tried doing 'git rebase master branch' when on master and it just applied all the stuff from master to branch.

Is there any way to apply a commit to 2 different branches (which have diverged) in a way that git will remember so that when the 2 branches merge later, it won't result in duplicate commits? I find that I often make changes that days or weeks later find out that some other branch needs that change and by then, there have been lots of commits to both branches after the commit I want.

skillzero@gmail.com schrieb:

Well, the way to do it is "careful planning".

If you have a *slight* suspicion that some change *might* be needed on a different branch, then:

1. you commit the change on a branch of its own that forks off of the
merge-base of *all* the branches that *might* need it;

2. next, you merge this fix-up branch into the branch where you need it
first, which is very likely your current topic-under-development.

3. Later you can merge the branch into the other branches if you find that
it is really needed.

If you don't have the slight suspicion, then you have to take the second-best route, namely to cherry-pick the commit onto a branch just like in 1. above, and continue with 2. and 3. In this case you have the commit twice, but not more than that.

-- Hannes

On Tue, Jan 13, 2009 at 11:34 PM, Johannes Sixt <j.sixt@viscovery.net> wrote:

If I create a separate bug-fix-only branch X that forks from the latest common commit of all the branches that might need it and some of those branches already have commits after that merge base (e.g. branch Z is 5 commits after the common merge base by the time I fix the bug), will git be able to merge the new branch X into Z in a way that will allow me to also merge branch X into my original feature branch A and then later merge A into Z without duplicating the commit that is now in both branch X and Z?

It seems like I'd run into my original duplicate commit problem because even though branch X was originally based off the same parent commit, it will have a different parent when it is merged into Z because Z is no longer at that common merge commit (it's 5 commits beyond it).

[Please reply-to-all on this list, to keep Cc: list]

skillzero@gmail.com schrieb:

After you created the fixup, you have this situation:

    o--o--o   <- A (feature branch)
   /
--o--x        <- X (the fix-up branch)
   \
    o--o--o   <- Z (probably your master)

You merge the fix-up into the feature branch and continue developing the feature:

    o--o--o--M--o--o   <- A
   /        /
--o--x-----'           <- X
   \
    o--o--o            <- Z

Other people need the fix in Z right now, so you merge it into Z as well:

    o--o--o--M--o--o   <- A
   /        /
--o--x-----<           <- X
   \        \
    o--o--o--N         <- Z

You complete your feature and merge it into Z:

    o--o--o--M--o--o     <- A
   /        /       \
--o--x-----<         \   <- X
   \        \         \
    o--o--o--N---------O <- Z

The fix-up commit is only once in your history.

-- Hannes

On Wed, Jan 14, 2009 at 12:34 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:

Thanks for the info. That's what I was hoping, but I was thinking that I'd get duplicate commits if I did that. I'll have to try it out when I run into this situation again.

Related to this, is there a way to easily find the common merge base given a bunch of a branches? When I want to fix a bug, I want to say "Given branches A, B, C, D, and E, where should I fork my bug fix branch from so that I can merge this branch into all those branches without getting duplicate commits?".

On Wed, Jan 14, 2009 at 10:33:02AM -0800, skillzero@gmail.com wrote:

[ ... ]
  

You should have a look at 'git help merge-base'

-Peter

skillzero@gmail.com writes:

You do not necessarily have to fork from, nor merge into, any of them.

If you fixed a bug, you would hopefully know where the bug was injected at into your history. You may have bisected it down to one commit $BAD. You can fork your fix on top of that $BAD commit:

	$ git checkout -b fix-bug-foo $BAD

All of the branches that share the commit have the bug, so your fix could be merged to all of them if you really wanted to, and you should do so if these A...E branches are meant to be consumed on their own.

But if the branches A...E you are about are for developing independent topics, and if their theme won't get affected by the bug, it is much better not to merge the fix in. You will have the merge for the fix in your integration branch anyway. It is preferable not to contaminate an independent topic branch whose purpose is to cook its own theme with an unrelated bugfix, even if it is brought in as a merge.

skillzero@gmail.com, 14.01.2009:

Note, that you'd get 2 merge commits for this fix-up commit into branch Z. The first from merging X into Z, the second is created from merging X into A and occures in Z when merging A into it.

Markus

skillzero@gmail.com wrote:

No, if you use merges it will not duplicate. It will know exactly what to do because it is the same commit in all branches. Only git-cherry-pick will duplicate the same patch, but as a different new commit. Then when merging the merge sees a merge conflict but since it is exactly the same change it will accept it. The same happens if two different patches have exact same hunk, the merge is smart to accept the same change from two sources. What happen with cherry-pick is that all the hunks match.

Boaz

Quoting skillzero@gmail.com:

Johennes Sixt gave a good answer. You need to think before making your commits while you are developing to separate what change belongs to common code base and what change belongs to a specific feature.

I was reading gitster's blog (he has several "git tutorial" articles recently) and he talks about how to commit a change to a branch different from what you originally developed it on in today's entry. You may find it instructive, but the procedure is after you think about it and decide what change goes where.

skillzero@gmail.com wrote:

[later]

I think there's an important misunderstanding here: merging A into B does *not* have anything to do with commits, or history for that matter, beyond the differences from $(git merge-base A B) to A and B.[*]

Along the same lines, nothing is ever re-applied during merging. git-merge just figures out that you made the same change on both sides, so it must have been a good change, so it must go into the end result. *How* you arrived at the same change---say, by cherry-picking, or by getting the same result in that region from otherwise different commits, or even from several commits---does *not* matter in any way.

You can use 'git cherry', 'git log --left-right --cherry-pick', and similar tools to find commits that are cherry-picked "duplicates", but unless you rewrite history, they are there to stay.

[*] This is a simplification since as soon as the merge-base is not unique, merge-recursive will actually start looking into history further back.

2009/1/14 Thomas Rast <trast@student.ethz.ch>:

Yes, merge only considers what bytes (aka contents of trees-directories and blobs-files) do the branches to be merged have, compares them (by comparing their hashes) and if there are differences tries to mix them together according to the merge rules described somewhere in Documentation.

So this all is really just about what the branches contain, not how they got it. It is the conflict resolution algorithm which uses the history to find the best possible source blob or tree which was changed by conflicting branches so the "mix" can be prepared as close as possible to what would we do if we went looking for the pieces manually.

I guess maybe a better question is how do people normally handle situations like mine where I did some work on branch X and I later realize I need only a portion of that work on branch Y? I'm not sure how I can change my workflow to completely eliminate these situations. For example, I often start a branch to add a new feature and I end up fixing bug A on that branch. Then other people on my team decide they need the fix for bug A immediately and can't wait for me to finish my feature branch and do a full merge.

Is there some way I can change my workflow such that I can fix bug A (maybe on a separate branch?) and somehow apply it to both both branches in a way that won't result in duplicate commits?

Does this kind of thing ever happen with the Linux kernel or git itself: somebody does a fix as part of their topic branch and the Linux kernel or git master wants that particular fix now, but is not ready for the full topic branch? Would they just suggest the fix be separated into its own topic branch and that merged? If so, how would that new topic branch merge into the original topic branch without resulting in a duplicate commit when it's later merged into master?

On 2009-01-14, skillzero@gmail.com <skillzero@gmail.com> wrote:

If I understand you right, you're talking about a situation where a particular change is in two different branches, but the SHA is different.

And yet, in your scenario, both of these branches somehow get merged into the final branch.

In other words, a DAG of branches merging itself has a merge, if that makes sense :-)

You may need to think about how commits flow from branch to branch, and what branches are private, and therefore can be rebased or change history, and what are public, and how often and when the private ones rebase off of the public ones.