Two crazy proposals for changing git's diff commands

So, here I am as a newly converted index-embracer---no more index denying from me.

However, I'm still trying to wrap my brain around the various diff commands that git provides and how they would fit into my workflow, Junio and I have touched on this already in a previous thread, but I'm starting here with more fresh and complete analysis of the UI around diff.

The motivation for the *long* (my apologies) message below is largely the fact that I realized my workflow would use the following command very regularly:

	git diff-index --cached HEAD

I would be using this before every "got commit" to get a preview, and it seems like a painfully long name for such a common operation. Even its shortcut form:

	git diff --cached HEAD

is among the longest of the "git diff" shortcuts.

My other motivation is that in spite of having what I think is a fairly good grasp of the structure of git, (the object database, the index, etc.), I still have one heck of a time trying to remember which diff commands are which.

So here we go...

Background ========== At a conceptual level, there are 4 diff operations each of which acts on an ordered pair (from->to) of trees. One operation takes two explicit tree objects, while the other three act on 0 or 1 explicit trees and 2 or 1 implicit trees (based on either the index or files in the working directory). Specifically, these are tree->index, index->files, and the composite of the two, tree->files.

To get at these 4 different operations, git provides 4 commands named diff-tree, diff-index --cached, diff-files, and diff-index. On top of these, git also provides some syntactic sugar in the form of "diff" shortcuts for a total of 8 different diff commands.

This is all summarized in the following table:

Operation (from -> to) Core command Shortcut command ----------------------- ------------ --------------- diff <treeA> -> <treeB> diff-tree <treeA> <treeB> diff <treeA> <treeB> diff <tree> -> index diff-index --cached <tree> diff --cached <tree> diff index -> files diff-files diff diff <tree> -> files diff-index <tree> diff <tree>

I think this background is fairly complete, at least as far as the functionality exposed by git-diff goes---I am ignoring git-diff-stages for now, and throughout the remainder of this message.

Use cases ========= After understanding things that far, I asked myself what each of the four operations are useful for. The tree->tree case is easiest as it simply shows the difference between two trees that exist in the object store.

The remaining three cases are more interesting because they provide mechanisms for querying trees that don't yet (or may never) exist in the object store. Here are the questions I have been able to come up with so far that the operations can help in answering:

<treeA> -> <treeB> (diff-tree <treeA> <treeB>, or diff <treeA> <treeB>)
  What changed between two trees?

<tree> -> index (diff-index --cached <tree>, diff --cached <tree>)
  When <tree> is HEAD: What will "git commit" do?

index -> files (diff-files, diff)
  What work have I done that I haven't updated into the index yet?
  Or, if not manually updating the index: What will "git commit -a" do?

<tree> -> files (diff-index <tree>, diff <tree>)
  When <tree> is HEAD: What will "git commit -a" do?

Can anyone else think of common use cases for these various operations that I've missed?

Subjective comments/proposals ============================= Everything above should be pretty much objective descriptions of how things exist. From here on out, I'll start in with my opinions and hopefully some useful proposals, (ordered by increasing likelihood of being controversial).

It's interesting to me that both variants of diff-index (with and without --cached) require a tree argument, while at the same time, both variants seem most useful when used with HEAD. So it looks like there's a reasonable default value for that option that is missing. This should be pretty painless to fix, (no user retraining required):

	Proposal 1: Make diff-index use HEAD if no <tree> is specified

	With this proposal, my most-common command now shrinks to:

		git diff-index --cached

	(And I was quite surprised to just learn that the shortcut
	version of "git diff --cached" already does default to HEAD
	rather than calling git-diff-files and erroring out on the
	unknown --cached argument. That's handy, even if a bit
	unexpected from the documentation of "git diff".)

So, my common command is a bit shorter, but I'd like to shrink it more, and I still haven't addressed my which-diff-command-do-I-want confusion.

First, I see a potential problem in the use cases table above. The "git diff" command is taught in the tutorial as a way to preview what will be committed by "commit -a". But I think this lays a trap for git newcomers.

If "git diff" is learned as a commit preview, (during a larval index-unaware stage), then this behavior will have to be unlearned when the user starts using the index. At that point "git diff" becomes a way to examine what will *not* be committed by "git commit" rather than what *will* be committed by "git commit -a".

This seems an unkind thing to do to new users. Instead, users of "commit -a" should be provided with a HEAD->files diff operation for previewing commits. That's what they really want to see, (and not the index->files diff that only happens to match in the case they haven't manually dirtied the index).

So the 'correct' preview command for "commit -a" is currently one of "diff HEAD" or "diff-index HEAD", and under Proposal 1 it would be "diff-index".

So, without using the shortcut version, the tutorial's preview command is down to a single name "diff-index", but it is rather awkward to have a "-index" command in the index-avoidance stage of the tutorials.

And I think this goes toward my which-diff confusion. Consider the three diff operations that allow for investigating un-committed trees:

	<tree>  -> index	diff-index --cached <tree>
	index   -> files	diff-files
	<tree>  -> files	diff-index <tree>

Here, "diff-index" is the only one of the three commands that does not operate on the index. One can notice a similar thing in the ASCII diagram from the core tutorial where diff-index is the only one of these three operations that completely bypasses the index (!).

All that just to say that there's some inconsistent naming in the core commands. This is papered over somewhat by the "git diff" shortcuts but at the same time that also adds even _more_ diff commands for people like me to have to learn.

So, here, finally is a proposal to change the names of some diff commands. I expect this to be more controversial than proposal #1 as it is sure to run up against ingrained muscle memory in some cases. But I've tried to minimize that as much as possible, and I hope it will be workable.

	Proposal #2: Provide the following 4 diff commands:

Operation (from -> to) Proposal ----------------------- -------- diff <treeA> -> <treeB> diff <treeA> <treeA> diff <tree> -> index diff-index <tree> (default to HEAD) diff index -> files diff diff <tree> -> files diff-files <tree> (default to HEAD)

The goal here is that using diff, diff-index, and diff-files without any tree argument and without any options (such as --cached) should cover the most common cases. So there's less typing in general.

These 3 diff commands can be presented as fundamental, and usable without needing a layer of sugar above. So there is already less to learn.n

Also, diff-index and diff-files have parallel structure and naming, each performing a diff from HEAD or the given tree to either the current index or files, respectively. So the names should also be easier to learn.

Under this proposal, my "git commit" preview becomes:

	git diff-index

and the tutorial's "git commit -a" preview becomes:

	git diff-files

which looks pretty nice to me [*].

Let's examine the impact this proposal would have on the existing core and shortcut diff commands. Here's an explanation of the Notes column entries below:

"no change":	The existing command is provided in an identical form
		under the proposal.

"compatible":	The existing command will continue to function
		identically for backwards compatibility with muscle
		memory. But some things (such as --cached options)
		will simply be ignored and "unadvertised" under the
		new proposal

incompatible[*]:The proposal is not compatible with existing usage, so
		some amount of retraining will be needed. In each
		case, I've made notes on ways that might make this
		bearable.

Current core command After proposal Notes -------------------- -------------- ----- diff-tree <treeA> <treeB> diff <treeA> <treeB> compatible diff-index --cached <tree> diff-index <tree> compatible diff-files diff incompatible[1] diff-index <tree> diff-files <tree> incompatible[2]

[1] Existing "diff-files" is index->files while the proposed "diff-files" would be HEAD->files. Fortunately the retraining here is to a simpler command ("diff") which already exists. Hopefully, current users already prefer the simpler command anyway.

[2] Existing "diff-index <tree>" is <tree>->files while the proposed "diff-index <tree>" would be <tree>->index. Fortunately the existing "diff-index <tree>" has an existing shortcut as "diff <tree>" which can be maintained compatibly. So hopefully, current users already prefer the simpler command anyway. Otherwise, retraining for this command will involves an index->files substitution, but hopefully the consistent naming under the new proposal will help here.

Current shortcut command After proposal Notes ------------------------ -------------- ----- diff <treeA> <treeB> diff <treeA> <treeB> no change diff --cached <tree> diff-index <tree> compatible [3] diff diff no change diff <tree> diff-files <tree> compatible [3]

[3] The proposal doesn't recommend any version of "git diff" with a single <tree> argument. Fortunately we can continue to provide compatible support for both such existing uses since they differ based on the presence or absence of the --cached option.

Anyway, that's a proposal for some diff commands if we had the opportunity to do it from scratch. I'm not a trained user of git that would be impacted by this change, so I can't make any fair comment on whether the change would be worth making or not. But I would definitely be interested in hearing what existing users of git think of the idea.

And of course, I am glad to fix up the implementation and all the documentation as necessary to implement this proposal if people think it's a good idea.

-Carl

[*] It's not the original topic of this post, but now that I've finished this, I realize that if the diff proposal were implemented then "commit-files" would make a dandy replacement for "commit -a". That could lead to finally providing the parallel preview commands I originally wanted:

	git diff-index	# as preview for
	git commit-index
and:
	git diff-files	# as preview for
	git commit-files

Then "git commit" would just be a shortcut for git commit-index.

(Oh, and that would also lead to a natural "git ci" abbreviation too, if desired. This would parallel the "ci == checkin" abbreviation that some other systems provide.)

I think the separate notions of commit-index and commit-files would do a good job of allowing for simple tutorials, (eliminates the "what the heck is -a all about?" questions), that also don't contribute to general index-unawareness lead to later index-confusion as the current "git diff; git commit -a" does.

This might even lead to a natural distinction between "git status-index" and "git status files" too.

On Wed, 8 Feb 2006, Carl Worth wrote:

[ snip snip ]

Yeah, I agree. The "git diff" semantics are incomplete, and the choices may not be (aka "almost certainly aren't") the best. The recent "git show" actually came from a very common use (well, for me, at least) for "git-diff-tree" that the "git diff" command simply didn't support.

I don't think it's really worth changing the low-level command naming: nopefully few enough people use them that it doesn't matter, but that's also what scripts out there are likely to have, so the pain factor when something breaks is high enough.

But making "git diff" itself more often "Do The Right Thing(tm)" would probably be a good idea.

In particular, I think the real culprit is the plain "git diff" with no arguments at all. Right now it ends up showing just a piece of the picture, and the piece it shows is incomplete enough to be irritating.

So I would prefer a slightly different fix from yours: instead of changing the core commands (which hopefully people mainly use in scripts where stability is more important than "intuitive and easy defaults"), how about _just_ changing the meaning of "git diff"?

Right now a plain "git diff" only shows the differences in the current tree against the index. I think that was just the wrong choice. I think almost everybody would actually prefer the default to be to show the difference against HEAD.

The largest reason for why "git diff" defaults to diffing against the index (and not head) is _literally_ that it's faster. Yes, I'm ashamed, but "git diff HEAD" takes 0.35 seconds for me, and "git diff" takes 0.07 seconds, and largely based entirely on that (and not "what would users want"), I made the bad choice to default to git-diff-files for the "no arguments" case.

I hang my head in shame. I just like the _instantness_ of "nothing changed". That's a bad reason for choice of interfaces, though.

One option is to realize that "git status" is actually a much better thing to use (because it shows _both_ the "HEAD to index" _and_ "index to current tree" format).

Teaching "git status" to take a "-p" flag (for "patch" - or -v for "verbose") might actually be a good thing. Then, instead of "git diff", you'd use "git status -p" and it would show you what the differences are in the index, and what they are in the tree, so you'd _really_ know what "git commit" in all its glory would do.

		Linus

On Wed, 8 Feb 2006, Linus Torvalds wrote:

Btw, it was kind of luck (but it's definitely true) that the current "git diff" semantics happen to also be what you do want during merges when you try to resolve a conflict (while "git diff HEAD" is much less useful).

So making "git diff" default to diffing against HEAD is actually the wrong thing to do during merging, where the current behaviour is exactly what you'd want.

So I dunno. 

		Linus

On Wed, 8 Feb 2006 17:21:49 -0800 (PST), Linus Torvalds wrote:

That's certainly surprising history to learn.

I discovered this during the analysis and guess (incorrectly) that it was the motivation for the behavior of "git diff". Initially I think I would have made a proposal that had "git diff" creating a diff from HEAD except that I discovered two different targets that would be useful when diffing from HEAD:

	HEAD -> index
and:	HEAD -> files

so I chose "index" and "files" as the naming distinction there. In light of those, I couldn't think of any good naming for the remaining:

	index -> files

operation, so it seemed just as well to leave it with a bare "diff" as before.

But, as you've pointed out, my trying to create new high-level diff-index and diff-files operations fails in that it clashes with the existing low-level uses of those names, (which scripts may already be depending on).

I totally neglected to consider the pain of migrating scripts.

I still think the three above operations are the most commonly needed, but I don't yet have good suggestions for names for them.

I think I'll be back with some new proposals after considering your suggestion to expand git status.

-Carl

Carl Worth <cworth@cworth.org> writes:

By "three" do you mean these three?

	HEAD -> index
        HEAD -> files
        index -> files

The reason I am asking is because you already have them.

I agree with Linus that "git status -v" with commit options to give a commit preview is a very good change. That is one thing I pointed out lacking from the current set of the tools in one of the earlier message. I am not sure if you read what I wrote in that message, but one approach I suggested for this thread to take is to come up with a list of "useful comparisons" that the current set of tools, especially "git diff" wrapper, does not give you.

The three you listed above are not. They are useful, the low-level natively support them, and "git diff" wrapper gives easy shortcut for them. And one known "useful but not supported one" will be tackled soon with "git status -v" change. Are there other useful comparisons you wish you had that we do not currently give you easily?

Linus Torvalds <torvalds@osdl.org> writes:

Not necessarily. Your "during the merge" example is a good one, and "so far it looks good and I do not want to see these diffs while modifying things further -- update-index it!" workflow benefits immensely from it.

I somehow suspect this is welding training wheels to your bicycle.

I think this may not be a bad idea.

What we could do is this:

	$ git status -v [--only | --include] [paths...]

When -v is given, it takes the same parameter as "git commit", and changes its output format from the usual N x "# useful info lines" to something like:

	---
        diff ...
        --- a/path
        +++ b/path
        @@ -N,M +L,K @@

that shows the commit preview. At the same time we change "git commit" commit log message reader to stop reading the input at the first '---' line, just like we do for e-mailed patches.

Then:

 - If you want a commit preview _before_ initiating a commit,
   you can say:

	$ git status -v [whatever you planned to give git commit]

 - If you want a commit preview _while_ writing the commit log, you can
   say:

	$ git commit -v [whatever your parameters are]

   which internally would pass the -v and "$@" to git status
   that seeds the log message

 - If you want a commit preview after you made a commit, it is
   too late ;-).

Carl Worth <cworth@cworth.org> writes:

If there is none, then that means we have covered everything. But a more interesting question to ask is "are there useful comparisons that the existing diff lowlevel does not give me with a single command". And the answer is of course yes.

For example, you cannot preview "git commit --include paths..." with a single low-level command. You would need to run "git-diff-index --cached HEAD" for paths that are _not_ specified, and "git-diff-index HEAD" output for paths that you will give to "--include" when you make that commit.

No need. "git diff --cached" does it, as you discovered. I am open to update the lowlevel if there is a need, but this does not justify it.

Of course, learning various flags to give "git diff" is part of understanding the index, so if the user understands index, "git diff" without arguments will _not_ be the only diff command the user uses for a preview. And I suspect that depending on what development style you would use, the definition of "useful comparisons" would be different.

You are assuming that the user graduated from "larval unaware" in this part of the discussion, no?

By the way, diff-index without --cached *does* use index. Unindexed paths are not shown. All this means that there is still inconsistent understanding on your part.

Forget about renaming the core-level commands. This is UNNEGOTIABLE, at least with something like the reasoning presented in your message. You need to have something stronger than that.

I'm VERY open to give friendlier interface to 'git-diff' wrapper, though. And I appreciate that part of your discussion very much. The end users, once the improvement proposal like yours is done right, should NOT need to use the lowlevel diff command directly AT ALL, so renaming low-level serves no useful purpose other than confusing Porcelain people and me.

No, the goal should be for normal workflows there should be *no* need to use lowlevel commands by end users day-to-day. Myself, I have not typed lowlevel commands from the command line for quite a long time, except when I am debugging, and except when I wanted to see "diff-tree --cc" output for a single commit, only because there was no suitable wrapper. The last reason is now gone thanks to "git-show" Linus added. So as far as *MY* workflow is concerned, that goal has already been achieved.

I am not saying there is something wrong with your workflow -- the tool may still not give an easier way to get a diff you would want in your workflow. I've given one example already about the --include commit preview. So let's concentrate on this question:

	What are the useful workflows, in which during various
	phases of that workflow, existing "git diff" wrapper
	does not give us a useful comparison?

Then we can fill what is lacking. During the course of this exercise, it _might_ turn out that the low-level may need to be updated to make generation of such an output easier and/or more efficient, but first we need know what we want to add to the end result. Mucking with the machinery to generate output must come after that.

While I suspect what people would want most would be commit previews, I purposely said "useful comparison" instead of "commit preview" in the above. Reviewing a conflicted merge with "git diff" falls into "useful comparison but not commit preview" category.

Commit preview alone is rather easy. No matter how your worklow is (one extreme being no update-index until you are ready to commit, and say "git commit -a" to commit everything --- the other extreme being frequent update-index between commit, even multiple update-index on the same paths, and do "git commit --include paths..."), there are only four ways to make commits:

	$ git commit
        => git diff --cached

	$ git commit -a
        => git diff HEAD

        $ git commit --only paths...
	=> git diff HEAD paths...

	$ git commit --include paths...
	... there is no support in "git diff"; it cannot be done
        ... with a single low-level.

So we could even implement that with

	$ git commit --preview [other flags] [paths...]

and three quarters of necessary things are already there. There are obvious two other alternatives:

	$ git commit-preview [commit flags and paths...]
	$ git diff --preview [commit flags and paths...]

On Wed, 8 Feb 2006, Junio C Hamano wrote:

My argument for "git status" was that it very much _is_ about "what would I commit". So I'd much rather extend on "git status" than add a "--preview" flag to "git commit".

At least to me, "git status" is very much a "what is pending" kind of command. The fact that you can't con it into giving a diff is actually a downside: I would at times have preferred to have the "git commit" message contain an extended status that contained the diffs too.

Another way of saying that: I think it would make sense to have a verbose status report, and maybe also have that verbose flag passed into "git commit" so that you can see the verbose status when you edit the commit message.

Under that logic, "git status -v" would show all the diffs (not just filenames) and "git commit -v .." would be the same as "git commit .." but the "-v" flag would have been passed down to the "git status" call, so the commit message file would be pre-populated with the diff.

For small commits, it's actually nice to see the diff itself as you write the commit message.

		Linus

Linus Torvalds <torvalds@osdl.org> writes:

Our messages crossed. I'd agree about the reasoning completely, as I did in my previous message.

Now we need to find a volunteer to do all that work ;-).

Linus Torvalds <torvalds@osdl.org> writes:

Now that is done and sitting at the tip of "next".

Yes, that is my experience, and even with fairly big one, it is pleasant as long as your editor allows you to C-x 2 C-x o around the file you are editing, like "em" ;-).

commit cf7bb589af739563c90dc32b4901bea73aaaa9d0
Author: Junio C Hamano <junkio@cox.net>
Date:   Fri Feb 10 00:45:59 2006 -0800

    git-status -v
    
    This revamps the git-status command to take the same set of
    parameters as git commit.  It gives a preview of what is being
    committed with that command.  With -v flag, it shows the diff
    output between the HEAD commit and the index that would be
    committed if these flags were given to git-commit command.
    
    git-commit also acquires -v flag (it used to mean "verify" but
    that is the default anyway and there is --no-verify to turn it
    off, so not much is lost), which uses the updated git-status -v
    to seed the commit log buffer.  This is handy for writing a log
    message while reviewing the changes one last time.
    
    Now, git-commit and git-status internally share the same
    implementation.
    
    Unlike previous git-commit change, this uses a temporary index
    to prepare the index file that would become the real index file
    after a successful commit, and moves it to the real index file
    once the commit is actually made.  This makes it safer than the
    previous scheme, which stashed away the original index file and
    restored it after an aborted commit.
    
    Signed-off-by: Junio C Hamano <junkio@cox.net>

On Fri, 10 Feb 2006 01:05:28 -0800, Junio C Hamano wrote:

Thanks. This does look quite useful.

A couple of comments, though:

1) I think the patch should come after the traditional status summary,
   not before. If something is obviously "wrong" (non-updated file,
   etc.) that will be more obvious in the summary, so it's good to
   present that up front, and not bury it after the patch, (which
   might make it initially invisible without scrolling).

2) Using the "^---$" separator to separate the the edited contents
   into a commit message and ignored content seems risky to me.

   One risk is that the user might put that magic string in the commit
   message. Perhaps the risk is low, but it would be a silent
   destruction of historical data, which seems like a bad trap.

   A second risk is that with the current patch-then-summary order it
   is actually quite easy to inadvertently clobber the separator while
   editing the commit message. Particularly because the separator is
   placed exactly where the user must begin editing. So it would be
   easy to leave the separator trailing on a line of
   commit message comments or else introducing some initial
   whitespace that would break it.

   Moving the patch after the summary (as discussed above) would help
   greatly in avoiding the clobbered separator, but wouldn't address
   the separator-appears-in-commit-message problem.

-Carl

Carl Worth <cworth@cworth.org> writes:

Maybe.  A time for a quick poll.

That is a good argument for (1). Since the diff output never has '^---$' in itself, instead of discarding everything after the first '^---$', we could change the log cleanser to discard after the _last_ '^---$'. But coding this sanely is much trickier with sed ;-).

On Fri, 10 Feb 2006, Junio C Hamano wrote:

I think I agree. Especially if doing "git commit -v", the _top_ of the status message is what you'd normally be most aware of. I think. Especially if the patch is large, you'd grow bored looking at it long before you saw what followed.

			Linus

Linus Torvalds <torvalds@osdl.org> writes:

Well the exactly same reasoning on my part, who keeps tons of untracked files in his repository, left the patch output there. Otherwise I have to scroll off a long "untracked files" list to get to the patch ;-).

But even with that gripe, I'd agree that patch at the end would be a better choice. The user asked for a patch so she wouldn't fail to see it even if many '#' lines precede the patch. With the current ordering, '#' lines will most likely be ignored.

Dear diary, on Fri, Feb 10, 2006 at 10:09:41PM CET, I got a letter where Junio C Hamano <junkio@cox.net> said that...

It makes more sense to me. (Not that I'd be ever going to use the command, though. ;)

We do it for the mails (and Cogito for cg-mkpatch output) and it seems to work out just fine.

$ echo -- >a; >b; diff -u a b

Petr Baudis <pasky@suse.cz> writes:

Thanks.  Good point.  Forget what I said.

On Wed, 08 Feb 2006 17:21:12 -0800, Junio C Hamano wrote:

Yes, that's a more useful way to ask the question.

Personally, I see that as an argument against adding the --include semantics. It's a single command that is committing a mix of the index and the current files, (which neither "commit" nor "commit -a" do). Granted, Linus has said that he has relied on these existing semantics, (but he also conceded willingness to retrain to "update-index paths; commit" instead).

So there's some food for though there.

Yes, and I think it would be nice if the different workflows could be captured by different subsets of commands, (rather than a mishmash of different options to the same commands).

For example, I *think* we have consensus that two different, equally reasonable, workflows might currently use the following sets of commands (this is for basic work-preview-commit):

Index-lover:

	update-index
	diff --cached
	commit

Index-ignorer:

	diff HEAD
	commit -a

So, what I'm trying to work towards is a situtation where those workflows are captured through obviously-correlated command subsets. This is what I was getting at in the PS. of my previous post. For example, the above could be, (ignoring the clash with existing core commands), something like:

Index-lover:

	update-index
	diff-index
	commit-index

Index-ignorer:

	diff-files
	commit-files

[Note that due to the name clash problems, I'm not making a serious proposal here---merely showing again the consistent naming I'm interested in.]

The point of an exercise like that is that people could then settle on appropriate workflows by reading a small subset of man pages that link to each other, in their entirety. Compare that with the previous command examples where the workflow is documented in little subsets of man pages scattered about.

And please pardon me for my current fixation on the learnability of git, but I am in the situation of "selling" git to my fellow maintainers, (some of whom are saying "mercurial looks easier to learn").

Sure. In that light, try to conceive as my proposed "diff-files" and "diff-index" commands as two new highlevel commands. They would be useful in day-to-day operation, and most commonly without any need for command-line arguments. Those two operations exist today within the git-diff wrapper already as "diff HEAD" and "diff --cached HEAD" (or "diff --cached). But as described above, I think it would be better to put these operations on separate command names, and get them down to not requiring any command-line arguments in their common usage.

If you compare my above two chunks of example commands, there's nothing like a low-level vs. high-level distinction between them. It's more about separate command names being uses for different use cases and consistent naming used to group related commands within a use case.

I confess that prior to reading the replies from you and Linus I had never heard of git-show. I'll go look at that now, as well as consider the suggestion to extend git-status to give me what I'm looking for.

Anyway, thanks for all the patience. I'm really enjoying the conversation, and I hope I'm not being a bore.

-Carl

Carl Worth <cworth@cworth.org> writes:

I see where you are coming from and I personally kind of like this consistency. But I am hesitant to declare these two workflows as the _only_ ones officially supported by the tool at this moment. The collective use pattern of git is still young and leaving the door open would help us to evolve more useful workflows around the core in the future. One of our first goals would be to have good set of introductory documentations for best current practices -- if the project you work on fits this workflow, here is a way to do it. With that workflow, there is this way. By that time, hopefully many useless workflows (my "constantly rewinding pu branch" pattern _could_ fall into that category) would be withered away and it would be a good time to streamline the tool around officially supported workflows.

My feeling is that it is a bit premature to do that right now; I do not think we are there yet.

Sorry, I am not convinced about separate command names, but in the meantime you could have small wrappers around "git diff": "cw-diff-files" and "cw-diff-index" would be one liner each, wouldn't they?

And that is a _good_ thing about git. If the sample set of barebone Porcelains do not fit your needs, you can mix and match using lowlevel commands to quickly script your customized ones. If that is useful in general, that would become one of the best current practices.

I think that is where we differ. I think even for index-lover "diff HEAD" is useful in certain cases (obviously index-ignorer would not see much useful output from "diff --cached", though). In fact, I fall into "index-lover" camp but I use both depending on occasion. And as I said, I do not think "index-lover" and "index-ignorer" distinction above would be the only two valid workflows anyway, so I feel partitioning the command set along those lines is at least premature if not wrong.

On Thu, 09 Feb 2006 16:13:42 -0800, Junio C Hamano wrote:

Thanks. I think that means I've described my motivation now. Though I still have to come up with a valid proposal.

I wouldn't want to do that either. Fortunately, I don't think the above command set would do anything like that. The workflows above with the silly names are just examples, but the commands would still be available in general, and I would expect people to mix and match for whatever fits best.

The more significant aspect of the above grouping is not "what workflow is supported" but whether the commands operate primarily on the index or on the files in the working directory. The distinction is significant to understanding and taking full advantage of git. And this aspect of git is something that is particularly unique to git (and hence some source of confusion to newcomers).

Take a look at how the index vs. files distinction currently plays out with existing commands:

commit:	Acts on index. Can be made to act on files with -a.

diff:	Acts between index and files (which is a sort of special case).
	Can be made to act on files by passing a tree argument.
	Can be made to act on index by passing --cached.

update-index:	Acts on index, (as obviously encoded in the name).

I agree. I think the current 'tutorial introduction to git' does a good job at demonstrating one work flow. I'm picky though, so the two changes I'd like to have in it are: 1) No command-line options necessary (rename "commit -a" somehow). 2) Provide an honest HEAD->files diff rather than the current "git diff" that just 'accidentally' happens to give the same answer in this case.

For a taking-advantage-of-the-index workflow, I think the 'short git tutorial' does a poor job, (it makes the index notion just seem painful and in the way). But Linus' responses to me in the recent threads have helped me understand it better, so there's material for an improved tutorial there I think.

Yes, I can do that personally, and I do agree it's a nice benefit of git. (Though some might see it as a defect if such scripting is perceived as being necessary before getting comfortable use out of the tool.)

Yeah, that's a false partition. People will certainly use commands from both sets. I think the index-focused vs. files-focused orientation I've presented here is more valid partition.

And I think if there were a clean files-focused subset of commands (that also didn't "deny the index" in any fundamental way), that it would make a good introduction to git, (it would certainly provide a closer fit to what many people will conceive of as they come from other systems).

-Carl

Carl Worth <cworth@cworth.org> writes:

I think that is a wrong place to start from. There is no such thing as "file focused" operation in git. Git _fundamentally_ works with index. The reason "diff-index" without --cached is there is to tell you what _would_ happen if you were to run update-index on the path. The same thing for "diff-files". It tells you what damage you would be inflicting on the index if you were to update-index it with what you have in the working tree. They are simulations of what _would_ happen after you place them in the index, and until you place them in the index you obviously cannot write things out to a tree.

However much you want to make git pretend to be "file focused", you cannot deny the fact that the interesting things happen only after you place things in the index.

Now, from purely traditional SCM point of view, index may be secondary. People coming from index-less SCM background would just want two entities: the last checkin and what you have in the working tree. If your goal is you would want to have an alternative UI that completely hides index, that is a worthy goal (I think Cogito attempts to do so and does a reasonably good job), but that is something different from what git barebone Porcelains would want to force on its users.

But I do not think that is what you are aiming for in this round of your proposal.

Again, (2) is "git status -v". Simulation of what would happen if I were to commit right now. For (1), "git commit" to mean "git commit -a" has already been rejected, but you could introduce git-commit-all if you want. I however do not think that would fly well with general public; after already seeing 100+ commands adding one more would not be too much but I think that would be an additional confusion, not simplification.

Thanks. I take it to mean I can expect some updates to the section from you ;-).

As I already said, if enough people share the opinon that that particular wrapper is a good thing, it would become established BCP and eventually make it into a new command or a new option, even replacing the existing one if the benefit outweighs the transition cost.

I do not, unfortunately, see much merit in what you have said so far, such as renaming "git diff --cached" and "git diff HEAD". Running --cached and non --cached against something other than HEAD _is_ useful [*1*], so you cannot remove the existing "git diff --cached <commit-id>" or "git diff <commit-id>"; which means you would need two extra command just for the common cases to run "git diff --cached HEAD" and "git diff HEAD". Certainly that would be less typing for that particular case, but does that justify two extra commands? Along with the git-commit-all that would be three extra commands. I am not convinced.

I really feel I should slow down and stop responding for now, and instead start spending some time on doing the git-status updates.

Just so that I do not get misunderstood by the above statement. I do not mean I will _ignore_ your arguments; what I mean is I would stop talking, give your thought process time to go all the way, and then revisit your proposal after you finished thinking.

[Footnote]

*1* Often I find myself doing that while rewriting messy development history. For example, I start doing some work without knowing exactly where it leads, and end up with a history like this:

            "master"
        o---o
             \                    "topic" 
              o---o---o---o---o---o

At this point, "topic" contains something I know I want, but it contains two concepts that turned out to be completely independent. And often, one topic component is larger than the other. It may contain more than two topics.

In order to rewrite this mess to be more manageable, I would first do "diff master topic", to extract the changes into a single patch, start picking pieces from it to get logically self-contained units, and start building on top of "master":

        $ git diff master..topic >P.diff
        $ git checkout -b topicA master
        ... pick and apply pieces from P.diff to build
        ... commits on topicA branch.
                      
              o---o---o
             /        "topicA"
        o---o"master"
             \                    "topic" 
              o---o---o---o---o---o

Before doing each commit on "topicA" HEAD, I run "diff HEAD" before update-index the affected paths, or "diff --cached HEAD" after. Also I would run "diff --cached master" to make sure that the changes are only the ones related to "topicA". Usually I do this for smaller topics first.

After that, I'd do the remainder of the original "topic", but for that, I do not start from the patchfile I extracted by comparing "master" and "topic" I used initially. Still on "topicA", I extract "diff topic", and use it to rebuild the other topic:

        $ git diff -R topic >P.diff ;# --cached also would work fine
        $ git checkout -b topicB master
        ... pick and apply pieces from P.diff to build
        ... commits on topicB branch.

                                "topicB"
               o---o---o---o---o
              /
             /o---o---o
            |/        "topicA"
        o---o"master"
             \                    "topic" 
              o---o---o---o---o---o

After I am done, I'd try a pretend-merge between "topicA" and "topicB" in order to make sure I have not missed anything:

        $ git pull . topicA ;# merge it into current "topicB"
        $ git diff topic
                                "topicB"
               o---o---o---o---o---* (pretend merge)
              /                   /
             /o---o---o----------'
            |/        "topicA"
        o---o"master"
             \                    "topic" 
              o---o---o---o---o---o

The last diff better not to show anything other than cleanups for crufts. Then I can finally clean things up:

        $ git branch -D topic
        $ git reset --hard HEAD^ ;# nuke pretend merge

                                "topicB"
               o---o---o---o---o
              / 
             /o---o---o
            |/        "topicA"
        o---o"master"

On Thu, 09 Feb 2006 18:24:06 -0800, Junio C Hamano wrote:

Yes. Of course. My proposed commands still accept a revision argument, (they just defaulted to HEAD when one was not provided).

Yeah, me too. I've made enough noise for now. I'll spend more time using things before proposing anything else, and come back with something more concrete (== patches) next time instead of just lots of talk.

Anyway, thanks again for the time. And sorry for the distraction.

No reply necessary from this one. :-)

-Carl

PS. Thanks for the several examples. That's more good training for me, (the operations here provide a lot of help for things I've always had to do much more manually with things like CVS). And that's also more content for tutorials, (and yes, I did mean to sound like I would like to help generate some of those---I've got some people I'm trying to convince to switch to git right now, and they'll need some training).

Junio C Hamano <junkio@cox.net> wrote:

[etc.]

Oh, good.  It's not just me, then! ;-)

This is more or less exactly how I built the various patches I submitted here. The main difference is that I used StGIT because I'm less competent than Junio and didn't always manage to get all my changes in the right places to begin with.

(I really look forward to a StGIT which can withstand git-prune.)

-- [mdw]

Mark Wooding <mdw@distorted.org.uk> wrote:

Almost sure in the next release, probably sometime in March.

Mark Wooding wrote:

Since there were a few requests for this feature, I pushed it to the public repository (and also included it in the latest snapshot).

As it was discussed a few months ago, the top id of each patch (including the popped ones) is added to the .git/refs/patches/<branch>/<patch> file. The git-prune command will preserve these objects and their referents, making even the unapplied patches safe. There is another advantage with this approach - gitk shows the patch name for every corresponding commit.

Running any command (apart from help) on the current branch would create the initial references. You would need to do the same with all the other branches. Since this is a new feature, make sure you have backups before pruning (though it worked OK for me).

-- Catalin

Junio C Hamano <junkio@cox.net> writes:

Perhaps this discussion could be added under Documentation/howto?

/ Kent Engström, Lysator

Kent Engstrom <kent@lysator.liu.se> writes:

Maybe.

Traditionally, person who feels that way proofreads and copyedits the original posting and submits a patch ;-).

This howto consists of a footnote from an email by JC to the git mailing list (<7vfyms0x4p.fsf@assigned-by-dhcp.cox.net>).

Signed-off-by: Kent Engstrom <kent@lysator.liu.se>

---

 Documentation/howto/separating-topic-branches.txt |   91 +++++++++++++++++++++
 1 files changed, 91 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/howto/separating-topic-branches.txt

39f152ae224f45a3d977aa8966a477dbc1df676d

On Wed, Feb 08, 2006 at 05:21:12PM -0800, Junio C Hamano wrote:

Well, there's understanding the index, and then there's memorizing the flags. I would've thought it'd be a lot easier to remember something like

git diff HEAD INDEX git diff INDEX WORK git diff HEAD WORK

than, respectively,

git diff --cached git diff git diff HEAD

But maybe that's just me. (And maybe the namespace in question is already to crowded to allow for INDEX and WORK.)

--b.

"J. Bruce Fields" <bfields@fieldses.org> writes:

I do not think it is just you. The real problem, honestly speaking, is that "git diff" wrapper cheats and avoids doing its own set of flags.

The low-level is just a mechanism UI is built upon, and as a mechanism, except perhaps maybe --cached might be now better spelled as --index, has set of options and semantics that are consistent with its world model (index centric way of thinking).

Because "git diff" wrapper cheats, it ends up exposing the low-level flags and arguments to the end user, and to use that effectively, obviously you need to understand the world model the low-level is built upon.

It was OK (it could be argued that it was even better than sugar coating to make it *inconsistent* with the underlying world model) so far, as long as people who use it are aware of the index centric world model, but that "consistency with the underlying world model" makes it harder to approach and causes confusion.

That is why I these days often mention "welding training wheels". Doing half-baked sugarcoating of the UI layer would break mental model of people who understand the world model low-level builds and tries to make effective use of low-level through the UI.

Reading bits an pieces of your email gives an idea: [...given at the end of this email] On Wed, Feb 08, 2006 at 04:29:44PM -0800, Carl Worth wrote:

...

...and now the idea:
What if we leave the normal long commands alone, so scripts and
muscle memory are not disturbed, but create some short commands
that give intuitive true parallel previews:
  "git di" and "git ci" (diff or commit index)
  "git df" and "git cf" (diff or commit files)
  "git dm" and "git cm" (diff or commit merge)
and of course any other little bits as needed like:
  "git si", "git sf", etc. (status index, status file, etc.)
This would give a chance to "start over" with clear commands,
without causing a ruckus among existing loyal users.  WDYT?

--Tim Larson