Itches with the current rev spec

Hi,

So, I have three serious itches that would be nice to address:

1. git reset --hard HEAD~1/ git show HEAD~1 is a very common idiom
that's unnecessarily cumbersome to type out.  We can make the <rev>
part of <rev>~<n> optional without being ambiguous: you might argue
that ~<n> normally refers to a /home/<n>, but who uses numbers in
place of usernames?  Even if they do, how can that path possibly be
inside our repository?

2. git rebase -i master fails unless I've rebased my branch on top of
master.  I always wished I could do the equivalent of 'git rebase -i
master..', but I can't.  Can we give the A..B syntax a new meaning in
the context of rebase, namely $(git merge-base A B)?  No, this is not
similar to the current diff A..B at all: first, we don't operate on
two endpoints (so 'git rebase -i A B' is nonsensical, and only the
'git rebase -i A ^B'/ 'git rebase -i ^A B' should be handled as
special cases); second, we're trying to be consistent with the
end-result meaning of A..B in ranged-commands like log (as opposed to
diff, which is being inconsistent).

3. Even though I lashed out strongly against 'git diff A..B' because
of inconsistency, I can't say that it's not useful (omit specifying
HEAD on one side).  If we were to start over today, I would argue that
'git diff A ^B' and 'git diff B ^A' be handled as special cases to
mean 'git diff B $(git merge-base A B)' and 'git diff $(git merge-base
A B) B' respectively.  The normal 'git diff A B' should have nothing
to do with this.  Plus, 'git diff A...B' is really an eyesore.  So I
ask again: what can be done to improve the situation?

Thanks.

Ramkumar Ramachandra wrote:

Okay, so my solution to this is:

1. Change the meaning of 'git diff A..B' (and A ^B, ^A B for
consistency).  Existing users might be using 'git diff master..' on
their feature branches to get meaningful output, and this will not
change.  'git diff featurebranch1..' on another feature branch doesn't
give meaningful output anyway, and I find it hard to believe that
users will complain if we change the meaning of this.  Okay, maybe we
want to do it in git 2.0?

2. Document (1) properly in gitrevisions.txt.

3. Deprecate 'git diff A...B'.

What do you think?

Ramkumar Ramachandra <artagnon@gmail.com> writes:

It's a bit more complex than that: the ~<username> is expanded by the shell, before Git has any opportunity to guess anything.

~1 would be unusable for zsh users and tcsh users at least by default:

zsh% echo ~1 zsh: not enough directory stack entries.

tcsh% echo ~1 Unknown user: 1.

(An obvious workaround is to shell-quote it, but as the goal is to have something easy to type, \~1 or '~1' do not give so much benefit over HEAD~1)

That said, it seems to work fine for bash (even if the number is a PID, it's not expanded), so it may be a good idea to add it as a shortcut, with a warning in the doc about shell expansion.

On Thu, Apr 25, 2013 at 3:22 AM, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:

Yeah, probably fine, but I would also like H~1.

Felipe Contreras wrote:

You can get that now using `git symbolic ref H HEAD`. I'm wondering if we can do better than hard-interpreting HEAD as H at the rev-parse level.

Matthieu Moy venit, vidit, dixit 25.04.2013 10:22:

I've been using a patch for that for ages without problems; it had been rejected because of the reasons above, plus:

Note that even in bash ~1 has a different meaning when your directory stack is non-empty. It's just that I don't use that feature, and bash leaves '~1' as is when there is no stack (you haven't used pushd), whereas zsh errors out.

So, I do understand that some consider this semi-broken, even though it's not. But we avoid clashes with shell expansion in most cases for most shells.

As for rebase, I still have to look up what "git rebase A B" means. This would be much clearer with a range notation. I seem to recall I even suggested it, but that might have been in a parallel universe.

Michael

Ramkumar Ramachandra <artagnon@gmail.com> writes:

~<n> expands to the <n>th element of the dir stack.

Andreas.

Andreas Schwab wrote:

Oh, ouch.  And this is bash.

Have to think of something else.

On Thu, Apr 25, 2013 at 1:07 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

In what way does it fail? It seems to work ok for me. Do you mean that it chooses extra commits you did not want?

Maybe you expect rebase--interactive will only result in changes to commits you touch in the todo list and it will not actually rebase anything. Is that the goal?

I have been thinking of adding a targeted "rebase -i" extension. I often use rebase -i to change one commit in recent history or to squash some fixup into place. The trip through $EDITOR to do this seems disruptive to my thinking. So I would like to be able to do this:

   git rebase --edit $REF

which should act the same as

  GIT_EDITOR='sed -i "s/^pick $REF/edit $REF/"' \
  git rebase -i $REF^

Except that $REF could be any ref and not just the exact SHA1-abbreviation given in todo.

The change I imagine allows --fixup, --reword, --squash, etc. It might even allow multiple instances of each.

I haven't thought through how to handle the case where there are merges in the way, but I do already suppose that the command will simply fail if a ref is not an ancestor of HEAD.

Maybe this is too simple for your workflow, though. As I said, I did not understand your itch.

Phil

If I understand well, you're refering to a problem that also annoys me, ie. using "rebase -i" to just edit your local commits, without rebasing onto the lastest revision on the upstream branch, right ? That is, just another wart of having a single command for arguably-different use cases, or of having the single-argument version of rebase use that argument for 2 very different things (cut-off point and destination), but I won't try to address either of these today :)

In that case, what about just adding a new flag to "rebase -i", that would prevent the single-argument to be interpreted as destination ? I really consider this a workaround for a suboptimal CLI, but since we don't want to change the rebase CLI before at least 2.0, that could fill the gap for now.

As for the flag itself, what about --here ? Obviously it would only be meaninglful together with -i, and be exclusive with --onto.

Am 4/26/2013 10:19, schrieb Yann Dirson:

How about this:

Allow alternative spelling of

   git rebase -i master topic

like this:

   git rebase -i master..topic

(as always, the default for topic is HEAD).

Then by extension (cf. git diff, where A...B shows the diff between the mergebase and B)

   git rebase -i master...topic

would rebase onto the mergebase, which in practice will be the fork point of topic, i.e., a "non-rebasing rebase".

-- Hannes

Johannes Sixt wrote:

We absolutely don't want to go down the inconsistent diff UI route.  See [1].

[1]: http://thread.gmane.org/gmane.comp.version-control.git/222248

Johannes Sixt <j.sixt@viscovery.net> writes:

I actually made this typo a few times in the past.

In a single-strand-of-pearls history, what rebase operates on is clearly a _range_ with a defined linear order of commits, and master..topic is a natural way to express it.

And rebase not just needs the range that defines the set of commits to be replayed, but also needs the commit on top of which they are replayed. It is natural to take ^master as that commit, and it is useful when you are trying to catch up with that branch.

The reason you would use "rebase -i" is not for catching up [*1*], so defaulting to replay onto ^master is not useful. You want the command to replay on top of the stable same base, so that you can compare the result with the previous version in order to verify. Often, the fork-point with master is a good choice for that.

It is tempting to say that "rebase -i" and normal catch-up "rebase" (e.g. "pull --rebase") should have designed to behave differently. "git rebase -i master" perhaps should have made to rebase the current work on top of the fork-point from master, not on top of it, and require an explict --onto master if the user does want to also catch up.

But the above is orthogonal to the syntax "../..." issue.

[Footnote]

*1* "rebase" and "rebase -i" already behave slightly differently with respect to $onto" in that a catch-up rebase that is already up to date notices the situation and turns into a no-op, but it does not turn "rebase -i" into a no-op for this exact reason.

On Fri, Apr 26, 2013 at 12:49 PM, Junio C Hamano <gitster@pobox.com> wrote:

I agree, but I think there are other unexpected things.

I don't know what 'git rebase master' does, but I would expect 'git rebase --onto=master' to do the same thing. Then, if 'git rebase --onto=next master..topic' makes sense, so should 'git rebase next master..topic'.

Moreover, it often annoys me that 'git rebase master' does exactly what I want, but 'git rebase --onto=master previous' doesn't find the commits that are already into 'master'. One would expect the more defined version to work better, but it doesn't =/

Cheers.

Felipe Contreras <felipe.contreras@gmail.com> writes:

That all stems from the fact that rebase (not -i variant) predates these nice A..B, A...B, and $(merge-base A B) concepts have been ingrained in the user's mindset as the primary UI language of Git.

 - The UI language of "rebase origin" comes more from the "workflow"
   school.  "I have built on 'origin'; I want to catch up with its
   current state".  To support that workflow, 'origin' is the only
   thing you need to say, and "rebase origin" matches that nicely.
   If you then add "By the way, that statement expresses my wish for
   the 'topic' branch, not my current one", you get "rebase origin
   topic".

 - If the UI language for "rebase" were designed following the
   "composition using common elements like ranges and revisions"
   school, it would have started from "rebase --onto=X A..B".

Back then, we did not know which principle to design the UI language would prevail, but we needed something that works to support the end users. So "git log" spoke "A..B" but "git rebase" took "origin".

Over time, the "composition" school prevailed and these days we see many commands accept and act on revision ranges or discrete revisions.

The same thing happened to format-patch, whose original syntax was

    format-patch origin

which is still accepted, but we have adjusted it to understand the more prevalent

    format-patch origin..

because it is far more understandable if you know other commands that are based on "composition" UI language. That adjustment started making sense after it has become clear that "composition" school of UI language is the way forward.

It's just that "rebase" is waiting for the same kind of adjustment.

Hint, hint.

Junio C Hamano wrote:

I think you're looking at the whole issue backwards from the way I look at it. Let's try to lay out some fundamental principles and build a representations on top of that:

1. All rev specs (those specified in revisions.txt) either emit a
single positive/ negative (^) commit or multiple positive/ negative
commits (where the ordering does not matter).

2. Fundamentally, all commands require single/ multiple commits to
operate on.  They might also require some additional information.

rebase requires three pieces of information: the commit onto which to replay, a list of commits to replay, and a refspec to update once the replaying is done.

log requires one piece of information: the list of commits.

diff requires two pieces of information: two commits to diff.

3. "Range" is not an inherent property of A..B or A...B.  There are no
"revision ranges".

4. Every command is free to interpret positive and negative commits as
it sees fit.  Since there is no ordering, it must never treat one
negative commit differently from another negative commit, or one
positive commit differently from another positive commit.

show takes a list of positive commits and shows all of them.

log will show all the commits reachable from positive commits, and exclude all the commits reachable from negative commits. Here, the "list of commits" are interpreted differently from the show case.

diff can either take two positive commits or one positive + one negative commit. In the latter case, it swaps the arguments and treats both as positive commits.

rebase can take one negative commit and one positive commit. The commits reachable from the positive commit, but not from the negative commit are replayed onto the negative commit. Now, we can use --onto= to override the commit onto which to replay. But the fundamental constraint remains: rebase _cannot_ make this --onto= parameter part of the normal rev spec (we only have two types of commits: positive and negative to which we can assign different meanings). --

This, I think, is the way forward. In any command, forcing the user to differentiate between the two commits only using argv[0] and argv[1] is just horrible (diff with two positive commits is the only necessary exception to this rule).

Further, what I think is of utmost importance is consistency. Inventing loose mnemonics like in the diff case is the road to insanity. All commands _must_ behave exactly the same way with all the different rev specs (or error out when the particular rev spec emits more commits than the command needs/ the wrong number of positive-negative commits).

What's more? I have a solution. A brand new revspec is the _only_ way to solve our problems without breaking consistency, or trading off terseness [Who wants to do git rebase --onto master $(git merge-base master topic)..topic every single time?]. I mentioned it on the other thread, but didn't get feedback :(

On Mon, 29 Apr 2013 20:38:03 +0530 Ramkumar Ramachandra <artagnon@gmail.com> wrote:

Well, that could be seen as a problem, the .. syntax being commonly associated with the concept of range.

Don't forget the particular situation of cherry-pick, which shows a situation where we may want to specify a set of single commits and ranges, but for which the current mechanisms cause a problem.

See: http://thread.gmane.org/gmane.comp.version-control.git/199994/focus=200058

Ramkumar Ramachandra <artagnon@gmail.com> writes:

I am not "looking at" anything. I was giving the historical background to explain how the current UI language came to be, but that was not to argue for keeping it to be the way it is, or even to justify that it is the right UI.

Very interesting.

That world view is broken, isn't it? Perhaps you forgot to consider symmetric differences, where left positives and right positives have to be treated differently. "diff A B" and "diff B A" mean very different things, for that matter. A line of thought that begins with "there is no ordering" may be a "brave proposal", perhaps, but it is not "fundamental principles".

If you do not like the word "range", read it as a DAG.

"rebase requires three: onto, list and a ref" (by the way, it is not refspec, which has a specific meaning) is trapped by the limitation of its current UI language that came from the "workflow" school and missing what the operation really means.

rebase takes a DAG with one negative commit, and replays it to form an isomorphic DAG on top of another commit. At the essential level, it takes two pieces of information, such a DAG and an "onto" commit.

But in the current UI, the way to specify these two things are by giving three commits, i.e. --onto=ONTO ONE_NEGATIVE ONE_POSITIVE. The positive is used to specify which ref to update.

It is not far-fetched to allow rebase to handle a history with two branches A and B that share the common initial part (i.e. ^X A B) and replay that history on top of an unrelated point in history Y to transform:

             o---o---Y
            /
    ---o---X---C---C---A---A---A (tip of branch A)
                    \
                     B---B---B (tip of branch B)

into

             o---o---Y---C'--C'--A'--A'--A' (updated tip of branch A)
            /         \
    ---o---X           B'--B'--B' (updated tip of branch B)

But the "rebase one branch on a new base" UI that came from the "workflow" school is unable to express such an operation. The pieces of information we are using in the above are:

 * Where the bottom of the DAG being replayed is (i.e. X);
 * What refs are the top of the DAG (i.e. A and B);
 * Where the new bottom of the replayed DAG (i.e. Y).

So if we are refining the rebase UI, while making sure we can later extend it, we shouldn't start from "onto, list and a ref". We should start from "a single onto, a single bottom, and one or more refs that define tops".

So what? Why do you even _need_ to mix up all positive revisions, some of which mean different things from others, into a single bag, only to later differenciate some as special (i.e. used as the onto commit) from the others (i.e. the tips in the DAG)? If something is special, you can say not just it is special and can say what it means by saying "this is where I want to replay the DAG on top".

A much larger issue is that the current setup_revisions() infrastructure does not let us express an operation that involves two or more DAGs. People sometimes wish to say an equivalent of

    git show $(git rev-list A..B) $(git rev-list C..D)

but obviously

    git show A..B C..D

is not the way to say it, and this limitation comes from it.

Junio C Hamano wrote:

No, I did consider symmetric difference. How is git log A B --not $(git merge-base --all A B) different from git log B A --not $(git merge-base --all A B)?

In fact, I would go so far as to claim that git diff A B is broken. diff should be forbidden from taking two positives (since it has no way to differentiate between them but for the ordering). It should diff a positive against a negative.

My claim is very simple. If a command _depends_ on A..B being resolved as ^A B, and not B ^A, we have have a big problem. Why? Because we've already established that git log A B is exactly the same thing as git log B A. To maintain consistency with this, ordering should never matter.

Sorry about that thinko: yes, I meant ref.

After working on the implicit-push proposal for so long, I think I can tell the difference between a ref and refspec ;)

I wholeheartedly agree.

However, I think you've misunderstood what I said: my goal is to define _everything_ in terms of how different commands interpret a list of positive and negative commits. It's not that some commands take DAGs, other ranges, and yet others lists; all of them take rev specs that resolve to a list of positive-negative commits. What to do with that information is up to the command (erroring out is a valid response). In my above proposal, I'd like to change "rebase can take one negative commit and one positive commit" to "rebase can take one negative commit and multiple positive commits" (in fact, this was my original sentence, but I went back to "one positive commit" before sending out the email because I thought I was being crazy).

Um, my point was again that "ordering does not matter"; therefore for a third type of commit, you need a command-line parameter.

This is seriously bad.  We'll have to think about fixing this along the way.

Ramkumar Ramachandra <artagnon@gmail.com> writes:

Compare these (gitk will give you nicer picture):

   $ git log --oneline --graph --left-right A...B
   $ git log --oneline --graph --left-right B...A

For the purpose of "doing one thing and well", we have drawn the line at "we operate on at most one DAG and specify what happens to it with various other parameters, which may include commits" long time ago. If you want to operate on more than one DAG, the cleanest way is to do the set computation for A..B and C..D separately and combine them yourself (which is the example you omitted from the quote).

The setup_revisions() machinery that is the foundation of the current codebase has this design decision ingrained in it. That is where the marking of commits with only two primary colors (i.e. the UNINTERESTING bit) comes from, and where the "single DAG" limitation originates. You can extending it a little bit (e.g. by introducing a secondary color left/right) to enrich it, but fundamentally the infrastructure pretty much assumes we operate on one DAG and a commit is either outside or inside (or at the boundary) of it.

It may be nice if the low-level operated on more than one DAG, but it is very close to a proposition to throw the baby with the bathwater and restart from scratch. It is a lot more than a little "as an aside" task.

Junio C Hamano wrote:

Darn. I didn't realize that rev-list had a --left-right to mark commits with <, >, or - before giving it to the command. So, that's one more thing to note: there are positive and negative commits, as well as commits marked with a direction (in the special case of --left-right). Have we missed anything?

I know too little to comment on the issue.  I was merely musing.

Junio C Hamano wrote:

Okay, so can I start writing a series that will make git rebase accept one negative commit (N) and one positive commit (P) in any order? A git rebase N..P should rebase the DAG defined by P ^N onto $(git merge-base N P). Does that make sense?

(two positive commits are a special case for backward compatibility?)

Junio C Hamano <gitster@pobox.com> writes:

Just a clarification. Technically, this is _not_ impossible. With some (read: quite a lot of) work to move objects.flags out of the object and to allow unbounded number of flag bits, you could support arbitrary number of ranges that are UNION'ed together by pretty much the same code structure as the current revision machinery. You need 2 x N bits (for the above example, 2 x 2 bits) per commit.

I am not saying it is easy or we should start working on it, though ;-).

Junio C Hamano wrote:

I will try to drive the point home one more time. What do you really want to rebase? B ^A or B ^$(git merge-base A B)? They're two entirely different things as I've repeated countless times. And the latter is what I always really mean.