interactive rebase not rebasing

Hello,

Per the emails from me last week, I'm working in an environment with shared topic branches and am trying to find a bullet-proof way for devs to rebase their local commits after the topic branch has moved.

The easy approach would be to just let `git pull` create merge commits, and I would have been done with this long ago, but I'm striving to get rebasing to "just work" and avoid the ugliness of same-branch merge commits. Cross-branch merge commits are cool, but not same-branch.

So, here's a crazy scenario we've ran into--a new release has hit stable, with two devs working on the same topic branch, and both of them merge. One wins, and the other has to rebase. Previously, this was replaying commits, but with great tips from the list last week, `rebase -i -p` is handling most scenarios.

However, not this one:

# A --C------ <-- origin/stable # \ | \ # B -- D -- E -- F <-- origin/topic2 # \| # g -- h <-- topic2

topic2 is a dev that has locally merged stable in g, moved on in h, is ready to push, but another dev already merged stable in E, and also committed F.

If we do a `git pull --no-rebase`, the result is:

# A --C------ <-- origin/stable # \ | \ # B -- D -- E -- F <-- origin/topic2 # \| \ # g -- h ------- i <-- topic2

But i is a same-branch merge, and we'd rather rebase to something like:

# A --C------ <-- origin/stable # \ \ # B -- D -- E -- F <-- origin/topic2 # \ # h' <-- topic2

(...maybe g' in there if g resolved stable conflicts differently E did them. I'm not sure, I haven't gotten there yet.)

However, currently, `git rebase -i -p origin/topic2` results in:

# A --C------ <-- origin/stable # \ | \ # B -- D -- E -- F <-- origin/topic2 # \| # g -- h <-- topic2

Nothing has changed. g & h haven't moved...I can keep executing this operation and the commits never make it on top of origin/topic2's F.

Frustratingly, if I run non-interactive rebase, it works perfectly. But I've got other cases (see earlier posts) that do need the interactive rebase. Personally, I could probably make do with trying each and seeing what happened, but I'm really trying to find a bullet proof command/alias/script for devs to run and have it "just work".

I've attached a test that sets up the DAG as above and currently passes by asserting the not-desired/unchanged-DAG result. The assert for the desired result is commented out at the end.

Am I correct in asserting this is some sort of bug in the interactive rebase such that g & h are not ending up on top of F?

Thanks, Stephen

#!/bin/sh

test_description='rebase interactive does not rebase'

. ./test-lib.sh

test_expect_success 'setup' '
	echo "setup" >a &&
	git add a &&
	git commit -m "setup" &&
	git clone ./. server &&
	rm -fr server/.git/hooks &&
	git remote add origin ./server &&
	git config --add branch.master.remote origin &&
	git config --add branch.master.merge refs/heads/master &&
	git fetch &&

	git checkout -b stable master &&
	echo "setup.stable" >a &&
	git commit -a -m "stable" &&
	git push origin stable
'
#
# A --C------            <-- origin/stable
#  \  |      \
#   B -- D -- E -- F     <-- origin/topic2
#    \|             \
#     g -- h ------- i   <-- topic2
#
# Trying to push F..i
#
# merge-base(F, h) has two options: B and C
#
test_expect_success 'merging in stable with tricky double baserev does not fool the script' '
	# B: start our topic2 branch, and share it
	git checkout -b topic2 origin/stable &&
	git config --add branch.topic2.merge refs/heads/topic2 &&
	echo "commit B" >a.topic2 &&
	git add a.topic2 &&
	git commit -m "commit B created topic2" &&
	git push origin topic2 &&

	# C: now, separately, move ahead stable, and share it
	git checkout stable
	echo "commit C" >a &&
	git commit -a -m "commit C moved stable" &&
	git push origin stable &&

	# D: have another client commit (in this case, it is the server, but close enough) moves topic2
	cd server &&
	git checkout topic2 &&
	echo "commit D continuing topic2" >a.client2 &&
	git add a.client2 &&
	git commit -m "commit D by client2" &&

	# E: the same other client merges the moved stable
	git merge stable &&

	# F: the same other client moves topic2 again
	echo "commit F" >a.client2 &&
	git commit -a -m "commit F by client2" &&
	F_hash=$(git rev-parse HEAD) &&
	cd .. &&

	# g: now locally merge in the moved stable (even though our topic2 is out of date)
	git checkout topic2 &&
	git merge stable &&
	g_hash=$(git rev-parse HEAD) &&

	# h: advance local topic2
	echo "commit H" >a.topic2 &&
	git commit -a -m "commit H continues local fork" &&
	h_hash=$(git rev-parse HEAD) &&

	# i: make a new merge commit
	git pull --no-rebase &&
	i_hash=$(git rev-parse HEAD) &&

	# Watch merge rejected as something that should get rebased
	# ! git push origin topic2
	test "$i_hash $h_hash $F_hash" = "$(git rev-list --parents --no-walk HEAD)"

	# Now fix it the merge by rebasing it
	git reset --hard ORIG_HEAD &&
	GIT_EDITOR=: git rebase -i -p origin/topic2 &&
	h2_hash=$(git rev-parse HEAD) &&

	# Should be:
	# test "$h2_hash $F_hash" = "$(git rev-list --parents --no-walk HEAD)"
	# But is just:
	test "$h_hash $g_hash" = "$(git rev-list --parents --no-walk HEAD)"
	# Where did $F_hash go?
'

test_done

Stephen Haberman wrote:

I can imagine. Since you don't want to preserve the merges in this case, you shouldn't be using the -p flag.

In fact, for this particular scenario (assuming "h" is really the only commit on topic2), you probably want to just cherry-pick that commit into origin/topic2:

   git checkout topic2
   git reset --hard origin/topic2
   git cherry-pick ORIG_HEAD

would work nicely for that. If topic2 and topic1 resolved merge conflicts differently, cherry-picking topic2 will give conflicts again, so they'll have to be resolved. rebase (without -p) will have the same effect, ofcourse.

I don't think you can have a single command that does all the things you want, because the possible differences in input makes it very nearly impossible to always do "the right thing". For this case, you don't want to preserve merges, so "git rebase" without -p does the right thing. For that other case earlier this week, you wanted to preserve merges, so "git rebase -p" does the right thing there. It's complex to try to dwim things like that, and complexity will cause very weird and surprising errors when it *does* fail.

Instead, I think you should give your developers some training in the more advanced parts of git's integration tools and also look over your workflow. Frequently merging 'master' into a topic is very rarely the right thing to do. It's usually better to rebase the topic onto master or cherry-pick relevant bugfixes from 'master' into the topic. If two devs need to work on one topic, then that topic probably needs breaking up into a sub-topic. However, sub-topics should never merge 'master' into them themselves; only 'topic'. Otherwise you'll get more merges than work done.

Or you could sit down and think about what you want to happen and then run the appropriate command. ;-)

Thanks for this. If all bug-reports had this, bugfixing would be a lot easier. However, for this particular case, I'm not so sure it's a bug.

Assuming you're passing a correct input file to rebase -i; yes. At the very least, "h" should be moved to the tip of origin/topic2.

No, I do want to preserve most merges. This "most" qualification is because the merge "g", if rebased, would have been a no-op, so `rebase -i -p` correctly kept it out of the TODO file.

Which is cool, except that later on, when rewriting the other TODO commits, some of which were children of "g", it did not remember that "g" had gone away, so did nothing to take "g" out of the rewritten children's parent list.

Agreed. This makes a lot of sense for me, who has been hacking around in git-rebase--interactive fixing things, but I'd really like the other people on my team to just have to run `git rebase -i -p`.

Ah, you are too pessimistic. :-)

Cool, agreed. I've got a patch that gets `rebase -i -p` to do this. I'll send it to the list soon.

- Stephen

Stephen Haberman wrote:

Perhaps, although I think you're being overly optimistic if you think rebase can do all of this intelligently while still retaining clear semantics. I'd start with writing a separate tool for it, probably based on git sequencer. When that works out ok, get it to do what rebase does today and then incorporate the new tool as an option to rebase and get ready to answer complex questions for the cases where it doesn't do what the user wanted it to do.

Git is stupid very, very much by design. Linus constantly says that he prefers tools that he can figure out why they did something stupid over tools that try really hard to get it right, and I agree with him 100%, as do most of the core contributors (insofar as I've understood it). What you're suggesting is that a git command should try hard to dwim a lot of complexity about choosing tools away, and that goes right against the KISS principle which serves git so well. I'm not saying you shouldn't do it. Merely that you should think hard about it and then make sure it doesn't break anything people are already doing today with the current toolset.

I would agree with this, except that --preserve-merges is already in the codebase and does what we want. I'm not fundamentally changing how it works (e.g. how it decides what commits to keep/rewrite/etc.), I just found and patched a bug in it.

Yeah, I'm sorry, I did not mean my "pessimistic" comment that seriously. I understand `git rebase` can never do what everyone wants in all scenarios.

But given /this/ scenario (hehe), with the implementation's existing explicit usage of "--left-right --cherry-pick" to drop no-op commits, but then it's forgetting of this information later, leading to `git rebase` not performing a rebase at all, I think it is an obvious bug, and one that can be fixed without changing any of `git rebase`s existing semantics.

I've attempted to do that. Now that I sent in the patch, if you could review it, I would appreciate your feedback. I do need to rework the test case because I realized including the origin/pull aspects (which is what caused us to find it) is just noise and the bug can be reproduced with just local branches.

Thanks, Stephen

Stephen Haberman wrote:

Agreed.

I'm heading home from work now. I'll look it over tonight or tomorrow morning. Thanks for the confidence :-)

Cool, thanks.

Question: how taboo is it to just add another test file?

I'm attempting to integrate my test into t3404, which is the existing interactive rebase test. The two test_expect_success's I added worked when I ran them at the start of the test and then reset --hard the branches back for the other tests, but if I paste my tests where they should probably be, in the middle after the other -p tests, they break because the 10 or tests before this have screwed with the DAG already.

I can suffer through getting it to work, but a t3409 would be much easier, and probably easier to read as well as a I could setup my own DAG instead of hacking onto 3404s.

No problem. :-)

- Stephen

Stephen Haberman <stephen@exigencecorp.com> wrote:

Usually folks prefer to add stuff to an existing test file, but if the DAG is already a mess and you need a different DAG I find it easier to just add new test file. Thus far Junio hasn't pushed back when I've done that. Maybe I'm just lucky. :-)

Stephen Haberman wrote:

Not so taboo. Especially not if there are compelling technical reasons not to add stuff to an existing one.

Where you add the tests doesn't matter much. Many tests are grouped by feature simply because they were added along with the feature they're testing. There's no other value of grouping tests together.

t3409 is already in spearce.git's next branch. You should be able to add stuff to that, or add t3410 if that doesn't work so good.