Cherry picking instead of merges.

We have a branch coming from "company B" who is also doing development on the same tree as us. We would like to do regular merges with them, but unfortunately, there are quite a few changes in between.

First we tried a git-merge and resolved the conflicts. The problem here is that the resultant code didn't work. git-bisect wasn't very useful because the intermediate versions don't have resolved conflicts.

Yesterday, one developer cherry picked company B's changes into a branch. It appears he resolved the conflicts for each commit, which should make bisecting easier.

The problem is that we now have very divergent history.

Any advice on how to make use of how he resolved conflicts in order to merge company B's changes in using git-merge. I could always use the "ours" strategy to just kind of force the merge, so at least future merges would work, but it'd kind of be nice to have the proper history.

There are about 110 commits in question.

Thanks, David Brown

David Brown, Thu, Jul 03, 2008 20:26:50 +0200:

...and have absolutely no idea on what were the changes of the company B were made.

Let that developer do the merge next time? Or let the B do a merge with a commonly accepted base?

On 7/3/08, David Brown <git@davidb.org> wrote:

Unfortunately, since your mismerged branches are already published, rewriting history would cause a lot of pain for everyone. It would be better to avoid doing that entirely. However, I can see why you'd want to do that in order to make future git-bisect easier.

Basically, if you're going to try to fix the git-bisect intermediate versions, you're going to have to rewrite history anyway; in which case, why not just make your developer's cherry-picked branch the official one? Then your problems are solved, other than getting all your developers onto the new history.

Alternatively, if you just want to fix your main development tree so that it's "correct", then you could do this:

- Go to a point in time where both branches (main and developer) have
exactly the same set of patches: that is, your company + company B.
The exact history (ie. commit ids) will look different, because the
two branches took different paths to get there, but the code *should*
have been identical at those two times, since you have the same set of
patches.  Call the two points X and Y.

- X (on the main branch) is actually different from Y (on your
developer's branch) because someone mis-resolved the conflicts on X.
However, the only difference between X and Y should be the mis-merge.
Thus:

- git diff X..Y >fix-mis-merge.patch

- git checkout HEAD

- git apply fix-mis-merge.patch

This will apply the correct conflict resolution to the tip of your newest branch. All the revisions between X and HEAD will still be broken, but that's usually better than trying to rewrite history and pretend the broken revisions never existed. You can always use "git bisect skip" for cases like that.

Have fun,

Avery

On Thu, Jul 03, 2008 at 04:15:22PM -0400, Avery Pennarun wrote:

It's only sitting in a private developer's branch. I want to do the merge properly, but I'm just trying to figure out how to get the conflict resolution out of his work.

Once we start cherry picking the changes from Company B, we have a different set of changes from them, and future merges will get harder and harder.

Except we already know that the broken change is inside of the broken revisions.

It turns out that things are more messed up than I thought. This developer had done a 'git push' with some manual refs and pushed what was supposed to be a merge into an unmerged branch.

I've spoken with all of the developers who use this tree, and everyone agrees that rewinding the tree is the best way to go. Now just time to trudge forward and learn.

But, it makes for a good new rule: no cherry-picking other people's changes.

David

David> It's only sitting in a private developer's branch. I want to David> do the merge properly, but I'm just trying to figure out how to David> get the conflict resolution out of his work.

Why don't you create a commit which has both your (A) and company B branch head (B) as parent with the same content as the private developer's branch (P) which contains the succesful merge?

Something like that: (totally untested)

% git checkout A % git merge --no-commit -s ours B % git checkout P . % git commit -a

On Thu, 3 Jul 2008, David Brown wrote:

What I would suggest is actually a stupid combination of the two. The advantage and disadvantage of the cherry-picking is:

 - the cherry-picking obviously gave you the end result you wanted, and is 
   bisectable at a per-commit level (ie each cherry-pick was a 
   "micro-merge", and if there was a merge problem, it would show up 
   exactly in _that_ one)

BUT

 - the cherry-picking doesn't help future merges, since the history is 
   disjoint, and as such it will always diverge and just cause more and 
   more problems down the road to keep on doing this.

so what I would suggest is

 - generate the *state* of the tree by cherry-picking each commit one by 
   one, and verify that state.

 - but then do a merge that picks that one state as the merge result 
   (this is all assuming you cherry-picked every single commit, of 
   course!)

The latter thing is really easy to do. Just do the merge, and ignore the conflicts that happen entirely, because you then just use the tree from the temporary cherry-picking branch

	# Try to merge the other branch ..
	git merge otherbranch

	# .. but ignore conflicts entirely by then
	# setting the index and checked-out state to
	# the known-good end result
	git read-tree -u --reset cherry-picked-branch

	# .. and commit that instead
	git commit

	# and you can now remove the cherry-picked branch
	# that was only used for the tree
	git branch -D cherry-picked-branch

to actually commit that merge and get rid of the temporary cherry-picking branch.

(Of course, if the merge actually doesn't generate any conflicts, and thus commits it, but doesn't actually _work_, then that "git commit" needs to be a "git commit --amend" instead, in order to actually replace the merge commit rather than add a fixup commit afterwards)

End result: you have a nice merge with nice history that actually converges at a common point, but you effectively did the merge resolution one commit at a time with cherry-picking (or "git rebase", which is obviously just a convenient shorthand for cherry-picking everything).

This actually has a few other advantages too: "git show --cc" on that merge will also show the conflicts the way they got resolved, so you actually end up with some useful information this way too.

			Linus

On Thu, Jul 03, 2008 at 02:18:53PM -0700, Linus Torvalds wrote:

I'm still not clear how the one-commit-at-a-time resolution gets recorded anywhere (except in the cherry-picking branch).

It seems to be that I would need to do multiple merges, one at each point where there is a conflict that I had to resolved. I would remember this as I did each cherry picked change, but after the fact, I would have to compare the cherry picked change with the one it came from, and figure out where conflicts had to be resolved.

Thanks, David

On 2008.07.03 15:39:49 -0700, David Brown wrote:

They don't get recorded. Git does not store how merges happened in some diffs or whatever, it just stores the end result. So what Linus suggested was to reuse the end result that you already have to create the merge commit.

Say you have 5 commits like this:

  A---B---C
   \
    D---E

Now there are two ways to get the changes from D and E ontop of C. The first one is cherry-picking, which leads to:

  A---B---C---D'---E'
   \
    D---E

The other one is merging:

  A---B---C---M
   \         /
    D---E---/

Of course, you should end up with the same tree either way. It's just a different way of getting towards that final state. So commit E' and commit M, while different commits, would point to the same tree object.

Now if you want to create that merge commit M, and already have E' (like you do, as you already did all those cherry-picks), you can use what Linus suggested. You start the merge, ignore any conflicts and just tell git to use the tree that E' is pointing to instead.

Björn

On Fri, Jul 04, 2008 at 02:10:03AM +0200, Björn Steinbrink wrote:

Ok. I guess I need to explain a little further. The advice I've gotten is good, BTW.

After we've resolved the merge and committed it, we've then discovered that it doesn't work. However, there are around 100 commits on both branches of the merges and it would be nice to come up with some way of doing something like a bisect.

The trick is that the branch being merged in doesn't actually work on our platform, so I can't just test the alternate branch.

But, git-bisect isn't being all that helpful here. The problem is that the only conflicts we resolved is how the two trees were put together. Picking points in the middle seem to generate lots of similar, but not quite the same conflicts.

A cherry-picked tree would allow for an easy bisect, since all of the intermediary versions would work. If I somehow knew magical points within the other tree I could do some number of merges and the bisect would still work. I suppose I could do the merges one at a time, but it would make the history rather verbose.

Thanks, David

On Thu, 3 Jul 2008, David Brown wrote:

Why do you want to worry so much about bisecting the merge?

You should basically expect that normally, a merge doesn't cause new bugs in itself. And if it does, the most common case is a fairly obvious mismerge (we've had a number of those in the kernel, but they weren't all that mysterious). So _planning_ for a subtle bisection failure sounds a bit strange and counter-productive.

So don't plan for it, and don't make it a primary issue for your workflow. 

Then, *after* you have a merge, and the unlucky thing happens where the merge is actually the thing that introduces the bug, and it's not obvious what the exact interaction is, and bisection didn't help you pinpoint it (because it really was the merge), *that* is when you might then choose to re-do the rebase in a temporary branch, and then bisect that re-linearized history.

So basically:

 - keep the history clean and obvious

 - make it fairly straightforward and don't worry about the unlikely case.

 - don't do something painful just because bad things _can_ happen (they 
   can happen regardless of what you do)

 - because in the unlikely situation that bad things happen, you can 
   always then go back and do things the other way around and pinpoint 
   where the fault is that way.

IOW, let's say that you really do bisect things down to a merge and cannot see what the fault in that merge is, you can literally do

	# create a test-branch with the 'remote' side of the merge
	git checkout -b test-branch merge^2

	# rebase that remote side on top of the local side
	git rebase merge^

and now you've linearized the merge temporarily just to be able to bisect in that temporary branch what the bad interaction is. But once you've bisected it, the temporary branch is again just junk - there's no real value in saving it, because once you know _why_ the bug happened, you're just better off going back to the original history and just fixing it (and documenting the bug through the fix, rather than by addign extra-ugly history).

		Linus

Linus Torvalds schrieb:

FWIW, the same thing in different words is written in section

"Why bisecting merge commits can be harder than bisecting linear history"

of Documentation/user-manual.txt.

-- Hannes

On Fri, 4 Jul 2008, Johannes Sixt wrote:

I don't think that's the same thign at all.

That section basically says "just keep things linear". Which I very much disagree with. Trying to keep things linear just doesn't work if you work together with other people - since you have to rewrite history.

So my argument was the exact _reverse_. Don't try to keep things linear, because it doesn't _work_ right. Do the merges. They will very seldom cause subtle merge problems (non-subtle ones are much more common, but trivial to handle), and they will mean that you can work effectively togethr with other people.

And then, _if_ you have a merge that you really cannot figure out why it breaks, at that point you can _temporarily_ linearize the git history after-the-fact just as easily as you would ever have done before-the-fact.

In other words: linearization throws away real and useful information. You can always linearize afterwards for bisection purposes or whatever, but you can never _un_linearize because you've thrown away the information.

So it's much better to just do merges and keep the history, and then there are ways to rewrite the history later if you really need it.

That said - I think it's good practice and perfectly sane to do things like git-rebase to rewrite the history in a _private_ tree that contains only your own modifications and has never been public (where "applied emailed patches from others" still counts as your own work).

The "don't linearize" mantra really only is about commits that have ever been in anybody elses repository (and whether they _came_ from there, or whether they came from you but were public to others is immaterial).

			Linus

On Thu, 3 Jul 2008, David Brown wrote:

For merges, the _only_ thing that matters from a "resolution" standpoint is that the history joins together. And it doesn't matter if it joins together fifty times or once - only the last point is relevant, since that becomes the last point of shared state.

No, because you simply don't care. You only care about the final result. With a single merge, there is only a single merge-point, and a single result. The advantage of doing the cherry-pick is just that it splits the decision on how to merge up into many smaller decisions (that is often the _dis_advantage of cherry-picking too - it often makes for more work, even if the individual work may be simpler).

But once you've done all the small cherry-pick decisions, you'd only make one final merge that takes that final result as-is. Any future work will then know about the fact that you have a new common base point.

		Linus