patch, 2 partsRe: [PATCH 0/2] Making "git commit" to mean "git commit -a".

Carl Worth <cworth@cworth.org> writes:

This clearly shows that I did not express myself well. You are correct that there are commands that ignore the index by default (a notable example "git apply" has been given by both of us), and you do have to know about what the commands you use do to the index.

What I meant by "do not have to worry about" is not about the index operations each command invocation involves. Of course you need to know (unless you will do a "commit -a" at the end) that git apply without --index will leave the index out of sync relative to your working tree, for example.

What you do _not_ have to worry about all the time is the local changes you do not want to go in your next commit but still want to keep in your working tree. Although it probably is not kosher from the purist point of view, it is very convenient to be able to keep truly local changes (say, my GIT-VERSION-GEN, everybody's change to Makefile to set "prefix=/usr/local", or "#define DEBUG 1" in one of the C files you are currently mucking with) that you have no intention of committing, while you want to record the changes to the paths you worked on so far with patch application, merging and edit + update-index in your next commit. You record the latter in the index using git tools to build what you want to have in your next commit in the index in each step (again, each step you may have to be aware what you are doing). After you update the index, you can forget about them -- because the index remembers them for you. They are in the state you tentatively decided is good for the next commit. You do not have to worry about the local changes you still have that you do not want to have in the commit because you do not run update-index on them, and you can trust that git does not automatically do so either, so they stay local.

The above paragraph is not the important part of my message. What was much more important is what immediately followed it, which you did not quote:

    And at that point, I trust "git commit" to do the right thing --
    the damn thing I just checked with "git diff --cached" _is_ what
    will be committed.

Like it or not, git was designed by and for people who use the index to work in a dirty worktree. The "-a" option to "git commit" is politely explained as the "--all" option, but its true pronunciation is "screw the index -- I rightfully haven't been paying attention to the index (my workflow did not require me to) because I know all the changes in my worktree are what I want in the next commit" option.

The "screw the index" attitude is not a wrong thing per-se. It is perfectly a good habit to always work in a worktree that exactly matches HEAD after each commit, and the only index manipulation you would (unfortunately) need to do between your own two commits are "git add" (for this use pattern, "git rm" is an unnecessary thing to do -- just saying "rm" is enough). Then "git diff" (without --cached but perhaps with paths) would serve as a preview for your next commit because you are going to do the "screw the index" commit (except that unfortunate "git add" thing, which we _could_ fix with "intent to add" entries in the index) and you would be happy with the similarity to CVS.

Once you use "I care about the index" workflow, however, you will see more areas where git's index shine. For example, "git diff" starts to take a more useful role. "I care about the index" attitude means you let the index be the incremental staging area for your next commit, and when you reach a good "snapshot" point you update the index with various means provided by git. "git diff" will show "what could further be added to the next commit" without talking about what you already decided are good earlier and updated the index with. As we already discussed, "git merge" will update the index for cleanly merged paths to let you concentrate on more interesting cases (i.e. merge conflicts). To people who care about the index, the next commit preview is "git diff --cached", not "git diff HEAD". What git promises to them is not to update-index the local changes they have without being told and without their knowing.

This is where "git commit" that does "-a" by default goes quite against the underlying mental model of git. You staged what should appear in the next commit in the index because you did not want to worry about the local changes you still want to keep in your working tree. Doing the "screw the index" commit by default to these people is slap in the face. You do not want to get your index suddenly screwed at the final moment of making the commit, which happened to me when I did "commit --amend" with the version with those two patches applied.

Don't get me wrong. I know there are cases that it is useful to always commit with "-a", but that really has to be opt-in. When I work in my alternate "trivial fixes only" repository, I use the "screw the index" workflow myself. When I run git apply and the patch does not apply, I use "git apply --reject" and fix the mess by hand, and at that point I do not care if that operation updates the index for the paths involved or not (although I do check if the patch tries to add new paths -- they need to be told to git even whey you take the "screw the index" attitude), and I do not bother running update-index on them either. But that is possible only because I know I am going to commit the final result with "screw the index" option.

"grokking the index" is not about knowing how the index could be used in your workflow. It is about actually using the index to stage your next commit. Somebody a bit smarter than me once said that if you deny the index you are denying git. Although I would not say it that strongly, because "screw the index" is also a valid workflow to use (arguably part of) git, "screw the index" at the commit time _has_ _to_ be a conscious act.

On Tue, 28 Nov 2006, Junio C Hamano wrote:

This argument has its converse. What you should _not_ have to worry about all the time is whether your index really includes all the changes you want included in your next commit.

And whether wanting to leave local changes in the working directory without commiting them actually happen more often than wanting to commit every changes is arguable.

What should be pretty consensual though, is the fact that having experienced GIT users add an alias for "commit" actually becoming "comit -i" to preserve the current behavior is much easier than asking new GIT users do the same but with "commit -a".

So in that context I think having commit without arguments meaning commit -a is a pretty sensible default. And I don't think it has any influence on the "learning about the index" issue.

Nicolas Pitre <nico@cam.org> writes:

That's what we have "git diff" with various output options for; I often do "git diff --stat" or "git diff --name-status" when I know I am about to commit in a dirty working tree. I suspect that I am not getting your point.

I do not think anybody is talking about which happens more often. "screw the index" people do not have to worry about the index during the course of their changes in the working tree toward the next commit, and the only time they need to tell git (which _IS_ a system based on the index, dammit) about what they want to do with the index is at the commit time, and they tell git to "screw the index" by passing "-a" to "git commit". In other words, "-a" at commit time is a magic incantation to allow them to be casual about index manipulation before reaching the point to commit. They do not have to worry about differences between "git rm --force" vs "/bin/rm" nor "git apply" vs "git apply --index").

It might make sense to have a configuration in .git/config that says "user.workingtreeistheking = true". This should obviously affect what "git commit" does by default, but it also should change the behaviour of other commands to suit the "screw the index" workflow better.

For example, the configuration should probably make "git diff" (without an explicit --cached nor HEAD) pretend it was asked to show diff between HEAD and the working tree, because the user chose not to care about the index. Not caring about the index is different from consciously keeping the index clean; for example, running "git apply --index" by mistake when he meant to say "git apply" should be tolerated, and Porcelain-ish that is working under workingtreeistheking mode should behave as if the index does not exist. In other words, the index is _not_ a staging area towards the next commit for him; the working tree is.

I thought Cogito largely follows that model, so it certainly is possible to do things that way. And I would not mind if the changes are cleanly done and maintainable. I am NOT going to say that I will refuse to maintain the code that implements the workingtreeistheking half of the system, although it is very unlikely that I would ever enable that configuration in my repositories.

Would that make people happy? I do not think so. I think it will lead to more confusion to have two majorly different semantics in the same set of tools.

On Tue, 28 Nov 2006, Junio C Hamano wrote:

I'm afraid this conversation is getting nowhere then.

But I do.

No one talked about "screw the index" people. Those are happily using Cogito.

We're talking about flattening the GIT learning curve. And as futile it may seem to you that newbies should just use "commit -a" without thinking, they still get bothered by that -a there. And probably they'll forget about it once in a while and then GIT will _appear_ as malfunctioning to them.

Of course amongst those newbies that didn't went away at this point there will be those who decide to study further and come to the index concept. And I hope that we all agree that the index is a powerful but still advanced concept that should not be presented up front.

But my point is: why not making a very little change to the default commit behavior. Really little change involving -a being the default. The impact on newbies will be significant as they won't have to grok everything at once to make sense of this -a we are telling them to use blindly. And it will sort of match known expectations to commit everything dirty.

And actually my point above is that in many cases, maybe the majoryty of those case but this is arguable, what one is doing is not keeping dirty and uncommited state around but rather committing every changes all the time. In _that_ case, which might not be all the time but often enough, then using -a is annoying[1].

So having -a the default makes GIT much more friendly to new users. You "add" files, you "commit", you edit some files, you "commit" again, and everything works fine, and you are happy and starts feeling good about GIT.

Now for those who've seen the light and want to use the index it is not much of a bother to add a -i to their commit invokation. At this point if you understand the index you know what you're doing, and using -i won't bother you as much it bothered you to use -a without knowing why when you was a newbie.

But still, if you are a GIT old fart and have difficulties switching habits, or if you simply are the kind with dirty not-to-commit state in your tree and adding -i all the time bothers you just like [1] above, then there is a way out! You are a GIT expert at this point of course and certainly know how to add an alias for the -i to be implicit with your "commit".

Therefore I think this is much more logical to ask the experts to add an alias for "commit -i" than asking such tricks from less experienced users. This is all my point is about.

Nicolas Pitre wrote:

But for different reasons this alias cannot be named "commit". So you cannot with alias make "git commit" (with -a by default) work with index.

If "git commit -a" by default heresy ;-) was accepted, I'd rather it be via configuration option.

Jakub Narebski wrote:

So the newbie-friendly behavior should require learning how to edit a configuration file, and the expert-friendly behavior should be the one you get on your first out-of-the-box exposure to git?

Steven Grimm wrote:

Well, newbie friendly could be the default, while hacker friendly (expert friendly) would require learning how to configure git (there is actually no need to hand-edit configuration file, by the way).

Nicolas Pitre <nico@cam.org> writes:

I do not necessarily agree with the last sentence.

While I understand that is what you are suggesting, my point is that it will not stop at "commit -a". I already gave an example that "diff" needs to behave differently, if you want to match the behaviour of git consistently with "newbie" expectations.

If you go down that path, you would end up with two systems named git, that base their operations on two quite different mental models and behave differently. You do not want to go there; well at least I don't.

So don't tell them to use "-a" blindly. Teach them what it means using the terms they understand; we assume they haven't learned the "index" yet, so we would explain that "it is a way to commit all changes in the working tree". It is a short-hand so that they do not have to list all modified paths (or their git is recent enough, ".").

After they learn what it means and get used to using it, they will start wondering why we do not default to "-a". By that time, they would already have learned the config (because the first thing the tutorial teaches is user.name/user.email), and can use the alias mechanismk there to alias -a away.

Junio C Hamano wrote:

More likely they will start wondering the instant you tell them to use it the first time. Or at least they'll ask, "If '-a' means commit all files, what's the default?" And then you either have to blow off the question or start telling them about the index.

-Steve

Hi,

On Wed, 29 Nov 2006, Steven Grimm wrote:

So what? Tell them that there is a staging area, which makes many operations of git very powerful and fast. And this staging area is called "the index" in git. And to put some files into it, specify those files. If you want _all_ modified files there, use "-a". That's it.

Ciao, Dscho

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

Well said.

I think I have stated my preference and reasoning clearly enough on this topic, so I won't waste my time repeating them. My time is better spent on _listening_ to people who might want to make convincing arguments to influence what I will end up deciding (the final decision will be mine anyway).

By the way, I've been having fun with the xdl_merge() stuff; thanks for a job well done. I will push some of it out in 'pu' shortly.

Hi,

On Wed, 29 Nov 2006, Junio C Hamano wrote:

Oh *blush*, I fixed a few things since I sent that out... Will submit the remaining things as soon as the beast shows up in pu.

Ciao, Dscho

Junio C Hamano <junkio@cox.net> writes:

So I will add my $0.02:

  Summary: leave git-commit as-is

  I think the intimidation factor of the index is overstated.  A
  scratch working area is a natural concept -- that's what people do
  who don't even _know_ what an scm is.  I agree with Junio's
  description of commit -a as default breaking the mental model.

  I've played with three distributed scm: tla, hg, and git.  I've
  stuck with git because, in part, it helped me to learn its more
  advanced features -- not because it was easiest to use intially out
  of the box.  And if I'm not going to get something more than cvs or
  svn gives me, what's the point of switching in the first place?

  + seth
  
  

On Tue, 28 Nov 2006 23:44:57 -0800, Junio C Hamano wrote:

I think you just backed up the point you didn't understand. You said, "when I know I am about to commit in a dirty working tree". That's an exceptional thing to do, (from the point of view of your "pure discipline"), so you will always be conscious of doing it when you do.

That's exactly what I'm trying to talk about here.

I've been using your "commit -i and -a as default" patches since you sent them out, (thanks again for sending them). I will readily admit that the very first commit I wanted to make was a "partial, dirty tree" commit.

What happened was that I had made a couple of independent changes in concert and now I wanted to commit them separately. I think that in my usage, this is the most common case for me using update-index and commit rather than "commit -a". It also happened that the independent changes modified disjoint sets of files.

Now, the index works perfectly for this kind of situation, and I like it. Some of the people that I know, (people who are currently refusing to touch git, and people for whom I've been trying to be a proponent in this discussion), would just use "git commit file1 file2 ...." in this situation. I don't do that since I really like being able to identify the files in one command, (update-index), and then preview the commit I'm going to make before I do so, (with "diff --cached). Sure, I could just do the commit, do an after-the-fact review with "show" and then reset if I screwed up, but that just feels like the wrong way to go about it.

So, I'm an index lover here. I see how it's useful and I use it on a regular basis. But I do believe I make "clean index" commits more frequently. And regardless, I still think we should change the default for git-commit to make it easier for users to learn git.

[As an aside, the situation of independent changes being mixed in a working tree is not always so lucky as to be cleanly separated into disjoint file sets. When it's not, I have to disentangle them. Now, the index could really help during this operation too, but we would need better tools than update-index which only works on a per-file basis. Something that let me easily select chunks of the patch would be really nice.]

I don't think "screw the index" is an accurate way to characterize my position at least. [In fact is "_the_ index" even a good way to describe what git has? Doesn't a commit operation that lists explicit files build up an alternate index which it will throw away if there's a failure at some point?]

So some git operations work by creating a tree object by first putting some state into an index file. That's fine. And users can even take advantage of doing that themselves if they want to. That's fine too.

All I'd like is that new users didn't have to learn those concepts as early as they have to do with current git.

Now, one time git really does have "_the_ index" and when the user really needs to know about it is when there is a conflicted merge. As Linus has been pointing out recently, there are some important benefits that the index provides at this point. And I think this is the right time to teach users about the index. I think they can learn it and take advantage of it at that point, (and I wouldn't even expect them to necessarily want to change the configuration of what "git commit" means).

In general, the process of resolving a conflicted merge is poorly presented to the user by git. The commands that leave the conflicted index, (git pull, git rebase, etc.) and that examine it, (git status), don't do a good job of telling the user what to do (git update-index) and how to examine the situation (git diff --merge). I think this could be improved a lot, and one piece of that is the "git resolve" thing I've been proposing recently.

I'd love to see a configuration value added, but I don't think we gain anything at all if we add a configuration value and leave the default value the same. It's not easier to tell new users to configure git to work this way then it is to tell them to use "commit -a".

Actually, I strongly disagree on this point. Months ago, before I understood the index as well as I do now, I did argue for a change like that, since I thought the index was just confusing. But I think it would be a mistake to make two fundamentally different models creep through all the tools. (Which is to say, I'm agreeing with what I think your motivation is for pushing back against that kind of thing.)

Plus, this model would be just broken anyway. The problem is that "git diff" meaning "git diff HEAD" works just fine when you're in a "normal" situation. But as soon as you're looking at a conflicted merge, then "git diff HEAD" isn't useful at all, but the difference between the index and the working tree is very useful. In that situation, the behavior of "git diff" suddenly makes a lot of sense, and the index has its chance to shine. This can lead to a nice epiphany for users I think.

                                    Not caring about the index

Yes, those are different. And I agree with you that a "pretend the index doesn't exist" mode would not be an improvement.

What I would like to see instead is that all commands keep the index clean by default, (with the notable exception of failed merge). And the only way to get it dirty would be with an explicit option such as "apply --index".

See? If users only get a dirty index by asking for it, then the likelihood of confusion goes way down, as users who haven't heard of "the index" certainly won't have any reason to pass a "--index" option around.

Agreed. Let's not go there.

I think what I'm asking for is a much more mild change. The "keep the index clean" behavior exists almost everywhere already, (the few exceptions are things like "git cherry-pick -n", and the notable exception of a conflicted merge). So I don't think supporting "commit -a by default" means we have to introduce a large conceptual change.

Also, for the case of the conflicted merge, the index really is a key part of what lets the user work through things. But there's really not an _essential_ need for the user to fully grok the index to take advantage of that. It's not hard to describe the behavior of "git diff" and "git diff --merge" in terms of things that the user wants to know, without mentioning the index at all. (Now, if a user asks, "how is git able to tell me all this amazing stuff", then would be a great time to explain the index, I think).

Did I succeed in getting you to consider anything new here? I hope so. I don't want you to feel like we're both just saying the same things back and forth over and over with no progress.

-Carl

Hi,

On Wed, 29 Nov 2006, Carl Worth wrote:

I regularly do something like

	$ git diff [file] > a1.patch
	$ vi a1.patch
	  [edit out the chunks I want to commit]
	$ git apply -R < a1.patch
	$ git commit [file]
	$ git apply < a1.patch

Seems a little bit convoluted, but works...

Hth, Dscho

Carl Worth <cworth@cworth.org> writes:

We seem to be agreeing (and Linus seems to, too, in a thread next door) that it is a good thing that git keeps index clean unless you explicitly ask it to.

We also seem to be agreeing that people with more involved needs can deliberately make index different from HEAD, way before issuing "git commit", and that they can commit even when "git diff" gives nonempty differences, and these are good things.

Are we on the same page?

Now what does it mean to make "commit" silently update the index with all the changes in the working tree without being told?

Unless you are introducing "working tree is the king" mode to git to make everything ignore the index's "contents" part (in other words, the index is used as the CVS/Entries file, nothing more, under that mode of operation), I think you just introduced an inconsistency at the place where the difference matters most.

I do not agree what you are asking is a "mild change" at all, and I said it already that it goes against the mental model of how git tools work.

Earlier, the world model was "you build it in the index and you make a commit of what is in the index; there is a last-minute index operation you can do by passing paths to commit and as a short-hand there is -a as well [*1*]". Now you made the world model "it does not matter what you have in the index before issuing git-commit; if you want to preserve what you built in the index, you have to do something non-default". That WOULD solicit more newbie confusion. "If it does not matter at the end unless you do something special, why bother doing it at all?" would be the question you would face.

Earlier on the "UI warts" thread, people said that the users do not form the mental model of how the toolset works by reading the tool's documentation but by trying things out, and I think that is a valid observation. We should not be sending a wrong message by introducing inconsistencies like that.

The tool's UI should naturally reflect what world model it is based on, and like it or not, the world model of git includes the index. The way to explain "-a" to new users should not be "you can _ignore_ index as long as you use -a". I do not think denying the index buys the new users anything. Rather, "building your next commit incrementally in the index is the workflow git is designed to support, but you are not required to do that _incrementally_. Until you encounter a complex situation such as resolving a large conflicting merge, doing that incrementally does not buy you anything as long as you work in a clean working tree. Instead, you can tell git what you want to commit when you run 'git-commit' by giving paths or directory names, or if you want to commit everything in the working tree, then you can also say '-a'".

I am all for rewording the cryptic "use update-index to update" message with "use 'commit -a' to commit all of them" or somesuch. That does NOT break the mental model.

Another thing that we need to be aware of is that new users won't be "newbies" forever, and the tool should not be optimized for the first few pages of the tutorial. You and Nico say "experienced people can always alias UI warts away", but I think that is a wrong attitude. Users with experience, long after this discussion is forgotten, would complain "other tools help us build the next commit in the index, but git-commit by default discards the distinction between what were marked for commit and what were not, unless explicitly told not to. Why does it do -a by default, and why should I forced to alias that stupid default away?"

[Footnote]

*1* In retrospect, making "commit -o" the default was a very bad change; I got tired of repeating myself in that discussion and applied that change, but it was probably a mistake.

Junio C Hamano <junkio@cox.net> writes:

Side note. I think the above "Until..." is an overstatement, and maybe the readers of the tutorial can be taught a lot earlier how the index can help them. Maybe the following sequence can be added to an early part of the tutorial sequence?

 $ edit hello.c
 $ make test
 $ git diff
 $ git update-index hello.c; # ok, that is good so far.
 $ edit hello.c; # hack more
 $ make test; # oops, does not work
 $ git diff; # ah, that overeager edit broken what was good
 $ git checkout hello.c; # get the last good one back

Junio C Hamano wrote:

That actually points out one of the things I think isn't so hot about using update-index for checkpointing your work. Here's a longer development session:

$ edit hello.c $ make test $ git update-index hello.c; # so far so good $ edit hello.c $ make test $ git update-index hello.c; # looks good too $ edit hello.c $ make test $ git update-index hello.c; # sure, seems okay $ edit hello.c $ make test; # oops! design flaw in the second edit uncovered! $ ???; # how do I back out the last three edits but not the first?

If you know for certain that you will only ever want to back out the most recent edit during your development, or back out all the way to HEAD, then update-index is fine, but if (like me) you want to checkpoint your work frequently so you can step back in a very fine-grained fashion, then it's less than ideal to have only one checkpoint that keeps getting overwritten.

For frequent checkpointing, as far as I can tell I pretty much need to commit (to a development branch, of course) every time. I do that because I never know beforehand whether I'll need to go back by more than one step later on; 99% of the time I don't have to, of course, but the remaining 1% is pretty painful if I can't.

Am I missing some magic index command that would support multi-level backing out? Obviously StGIT is an option as well, but that seems like overkill when all I want is to checkpoint my work. The above is why, even though (I think) I know enough about the index to use it as you describe, I often don't bother and just run "commit -a" during development instead. When I merge, I usually fold all my checkpoint commits together and merge the change as a logical unit.

But I'm still a relative n00b and would appreciate knowing if I'm just missing some big obvious technique.

Steven Grimm wrote:

Try `git commit --amend'.

Steven Grimm <koreth@midwinter.com> writes:

You are doing just fine. Using commits to do snapshot is another workflow people often use. These people tend to do so in a work branch and cherry-pick the resulting series of commits onto more permanent branch while reorganizing and tidying up the development history to be published to the outside world.

Hi,

On Wed, 29 Nov 2006, Junio C Hamano wrote:

I like it. Sort of a "temporary commit" to check against.

Ciao, Dscho

On Thu, 30 Nov 2006, Johannes Schindelin wrote:

I (very) occasionally do this for patches I get.

You can do

	git-apply --index patch

and it will apply the patch and update the index for you. That's great for committing the patch (because it means that it adds and removes your files automatically for you), but most of the time when I get an email that I want to apply, I just use "git-applymbox".

So where doing the "git apply --index" thing is great is when you see a patch that has some obvious deficiency that makes you not want to commit it directly, but add some fixup of your own.

That's when it's useful to use the index to your advantage - you can do "git diff" (to see just the fixups you did on top of the patch), or you can do "git diff HEAD" (to see the combined effect of both the patch _and_ your fixups).

That said, I have to admit that I usually (a) don't do this very often (ie this is not part of my daily routine) and (b) I tend to do "git reset" fairly soon afterwards (or alternatively, just "git commit -a") to get back to the situation where the index will match the current HEAD 100% again. So the "index doesn't match HEAD" situation is always just a _temporary_ thing for me.

Hi,

On Wed, 29 Nov 2006, Linus Torvalds wrote:

An obvious deficiency would also be the presence of hundreds of debug quirks I had to introduce to find the bug which I finally fixed. But I do not want to commit, because it is such a mess. So: into the index, ye files.

Now I can clean up everything I introduced to find the bug. If the result does not work as expected? "git diff"!

But now that I cleaned up the mess, I find that there is a more elegant way to solve the problem. Into the index, ye files! Clicketyclick, if I mess up, I always have the state in the index!

Ciao, Dscho

Linus Torvalds wrote:

A staging area is per definition meant to keep temporary things before they are committed to their designated place so there's nothing odd about that.

On Thu, 30 Nov 2006, Andreas Ericsson wrote:

Sure. It's just that some people seem to expect the index to be different from HEAD, and are afraid of being "confused" by it.

The fear seems to be about "git diff" getting different results from "git diff HEAD", and always having to _check_ the two.

So I wanted to make it clear that I never have that situation, because I never leave the index "dirty". I agree that there is nothing odd about it, but I think that people who don't actively use the index (or don't use git at all, and just worry about it) see it as a kind of separate entity with a life all its own.

I can see that if you think the index is likely to be out of kilter with HEAD, you'd always worry about "ok, so maybe the diff I get from 'git diff' isn't the _true_ diff, so now I have to do _both_ 'git diff' and 'git diff HEAD' to make sure I know what's up".

I just wanted to clarify that that is never the case for me, and I doubt anybody else really does it either. For a very complicated merge, I could possibly see somebody having a dirty index for a day or two and taking a break with the index dirty, but

 (a) I've certainly never seen that myself (and it would have to be 
     something very messy indeed - I remember multi-day merges with CVS, 
     but that was because CVS is so bad at merging, not because the merges 
     per se would have been all that messy)

 (b) if you have a merge _that_ messy, I don't think you're likely to 
     forget about it, and rather than be confused about the difference 
     between 'git diff' and 'git diff HEAD', you'll be really really happy 
     that you have some way of seeing just the _remaining_ pieces, rather 
     than all the crud you already fixed up.

In other words, the fact that the index _normally_ matches the HEAD may be obvious, but it's also important - it's important to allay fears from non-index users about it being somehow scary and confusing. It's not.

On Thu, Nov 30, 2006 at 07:58:09AM -0800, Linus Torvalds wrote:

Well, sure, because the documentation *talks* about it as a separate entity all its own. Just look at the man page for git-diff as a great example of this, or the ascii art diagram of the index. It is all technically _correct_, but it is scary as all heck.

But then why is the default for "git commit" to commit the index, if the index is almost == HEAD? And why is git-update-index given such prominence in the documentation?

If everyone agrees with this, I think it would be easier to make changes to the documentation and maybe some UI tweaks about what the default might be.

One suggestion is that perhaps a mode where warns users when index != HEAD for certain critical commands might not be a bad thing. That might give users that are just graduating beyond novice git usage, and just starting to become aware of the index, reassurance because if they *don't* see the warning message, they can rest assured that they don't have to do both "git diff" and "git diff HEAD", for example.

On Thu, 30 Nov 2006, Theodore Tso wrote:

The default is: commit everything that you ask for to be committed.

If you haven't marked anything to be committed (which you can do with "git add" too, or with simply being in the middle of a merge, or by having done something like "git pull -n" or similar that does everything _but_ commit), then git commit will say "nothing to do".

It has NOTHING to do with the index per se. 

I still don't understand why people are so hung up about the index.

So ignore the index entirely, and follow along with me:

	"git commit" with no parameters simply DOES NOT DO ANYTHING YOU 
	HAVEN'T ALREADY ASKED YOU TO DO.

It's that simple. It's that logical. Ignore the index. Ignore everything else. Just read that simple, straightforward, and logical sentence on its own. It all makes sense.

Then, the trivial follow-up is:

	If you want to commit _all_ dirty files, use "git commit -a". 
	Otherwise, name the files or subdirectories you want to commit 
	explicitly.

Again: THIS JUST MAKES SENSE.

Asking for "-a" to be the default behaviour is BAD.

For example, in "git commit --amend", it's _important_ that "-a" not be the default, because you may well want to just amend the commit _message_. No files updated AT ALL. You may have other state that is still dirty (because you didn't ask it to be committed last time), and they should NOT be committed, because the simple rule is:

	"git commit" with no parameters simply DOES NOT DO ANYTHING YOU 
	HAVEN'T ALREADY ASKED YOU TO DO.

Repeat the above sentence again. IT JUST MAKES SENSE.

So maybe the documentation shouldn't mention the "index" at all, because it apparently scares and confuses people. But the fact is, the documentation started out as _technical_ documentation, that explains the _technical_ side of git. We don't have lots of "end-user" docs.

But the lack of such end-user documentation should not cause idiotic threads like this, where people blame "the index".

Yeah, so the docs are too scary. But none of this has anything to do with "the index". It's all logical on its own, and the default behaviour to not commit anything you haven't asked to be committed is the right one.

Make "git commit" just say "You didn't say what you wanted to commit. Maybe you meant 'git commit -a'" if there's nothing to commit. How hard can that be? But don't change semantics now, and please DO NOT change them to something _worse_ than what we have now (and automatically adding the "-a" only in _certain_ circumstances is definitely much worse imnsho)

On Thu, 30 Nov 2006, Linus Torvalds wrote:

Might it be a good idea to have "git-add" do the same as "git-update-index" on already tracked files? That could be easily taught as "you must explicitly _add_ files to your next commit" and whether the file is already tracked or not wouldn't matter. This would help newbies actually getting used the index without mentioning the dreaded word "index" at all.

Right now git-add on an already tracked file does nothing, not even a message to say it did nothing.

On Thu, 30 Nov 2006 12:37:49 -0500 (EST), Nicolas Pitre wrote:

I think this is worth doing, but it doesn't solve the problem.

It would be nice if this worked, because then it would be natural to teach users to "git add" their changes, and then when the obvious complaint about that being annoying to type all the time, then respond by teaching "git commit -a" as a shortcut. That would be very natural.

But this doesn't quite work. Here's a major "index is confusing" scenario that I first hit on my first exposure to git, (it was so long ago that I completely forgot about it when when Linus asked what the big deal is about the index).

It's all about the fine details of what "git add" does that can be _extremely_ surprising to new users. Here's a simple scenario:

	$ git init-db
	defaulting to local storage area
	$ echo "hello wurld" > hello
	$ git add hello

	# Oops! Look at that typo.

	$ echo "hello world" > hello
	$ git commit -m "add hello"
	Committing initial tree 0267c1bf2956b3df47851e0163f2ea86c002379d
	$ git diff
	diff --git a/hello b/hello
	index b1df5e6..3b18e51 100644
	--- a/hello
	+++ b/hello
	@@ -1 +1 @@
	-hello wurld
	+hello world

And here the new user reaction is "What?! I fixed that before I made the commit! What kind of broken system is this."

The above example is not at all surprising to someone who understands the index. And it's that "why should that behavior be confusing" disconnect that I'm trying to bridge here. Can you see why the above confuses new users?

In most other systems I've used, 'add' means "I want the system to 'know' about this file" while 'commit' means "Please commit the current state of all files you 'know' about (or the ones I mention here on the command line)".

The current semantics do nothing to avoid this "interaction bug" and there is no way to explain it to the user without going through an "explanation of the index" and the user is left to decide that the index is just there to help make broken commits.

So, I'd love for "git add" to be a shorter way to type "update-index", (I have been campaigning for eliminating hyphens after all), but without different semantics _somewhere_ the default behavior still is potentially very confusing. The "ignore the index" stance Linus has been proposing recently just doesn't work here.

And all of this contributes to make git harder to learn than it should be.

-Carl

PS. It was actually "hard" for me to create that example above. The first time I ran through the commands I ended up with an empty diff at the end. "Huh? I _know_ that git does surprising things here." That was because I was using Junio's "commit -a" patch which did right thing rather than demonstrating the old wrong behavior I was trying to demonstrate.

Carl Worth wrote:

while in git "git add" means "I want to add this file" (in the state it is now) and not "I want the system to 'know' about this file". And "commit" mean "Please commit the current 'known' state of all files (or/and the current state of files I mention here on the comand line)".

Yes, this is different than what other SCM do...

On Thu, 30 Nov 2006 19:47:16 +0100, Jakub Narebski wrote:

Yes. There is a logical explanation for what git does, and it is self-consistent.

It just means that the user is _forced_ to pass file state across the:

	"working tree" -> git

boundary at two different times with two different commands for the very first commit the user makes. And the user _must_ understand that this is a two-step process, (even though, without the "typo" in my example above it would be natural to conclude the transition occurred only during "commit").

See? Git _is_ harder to learn, and a user really cannot learn it without being careful about the index right from the very beginning.

-Carl

Carl Worth wrote:

I'm not so sure about that. I came from CVS / SVN, although I've fiddled quite a bit with other scm's as well. The two-step commit process of git didn't terrify me at all, and I had used git at least a month before I joined the mailing-list and found out that there's this thing called an "index". I knew about it before, since back then (June or July 2005) there was only git-update-index to mark things to commit. I just didn't worry about it but expected the scm to tell me if I was about to break something horribly (which it often but not always did).

I think the main thing people are having difficulties with when it comes to git is that it doesn't do things like other SCM's do it. Imo this is a good thing, because it allows git to be more powerful than other SCM's. Otoh it forces users migrating from darcs/hg/monotone/perforce/whatever to git actually read the documentation (and quite a lot of it), while hg -> bzr migrators use pretty much the same commands for pretty much the same actions. This makes users accustomed to not reading docs / trying things out before attempting Real Work(tm), which breaks down horribly when user expectations doesn't match reality. The simplest and usually most effective solution is to meet the users half-way, and tell them early on that this power comes at the cost of having to read the documentation and do the tutorials.

Andreas Ericsson escreveu:

I still don't know exactly how to operate adds and commits from the command line. I regularly get bitten by not supplying the -a and -i options.

I'm coming from darcs, where you can select which each diff hunk to put in a commit separately.

However, I almost never do that. I operate git like darcs, from the emacs support mode. I almost never do -a commits anyway, because with emacs (M-x git-status) it's more natural to make functionally distinct commits, at the risk of introducing non-tested tree states in the repository.

On Thu, 30 Nov 2006 10:38:25 -0800, Carl Worth wrote:

By the way, I think I've said all I can in this thread.

If the "create file; git add; edit file; git commit" confusion isn't blisteringly obvious to the git maintainers then I think I have to give up here.

And this isn't just CVS-induced brain damage. It's the user being required to mentally juggle 3 states for the file, (the last "committed" state, the current "working tree" state, and this "something else" state). The sequence above, (which is very natural), exposes this "something else" state that to a new user.

If we imagine a new user as coming, not from cvs, but coming from no revision control system, then it's less confusing to add one single new state, (the "last committed" state), in addition to the "working tree" state the user is familiar with.

Forcing the user to learn two instead of one is just plain harder, (which is completely separate from git _allowing_ this extra state once you learn it).

So if git is determined to just be harder to learn this way, then I don't know what more I can do to help here.

I love git, and I think everyone should use it. I would just like to help make it a bit easier for people to do that.

-Carl

On Thu, 30 Nov 2006, Carl Worth wrote:

I'm sorry, but you are wrong.

It really _is_ CVS-induced brain damage, and I'm trying to teach you. You can give up, but that's really "refuse to see the damage that systems like RCS and CVS has done to the world"

The fundamental brain damage that CVS (and RCS, and SVN, and just about anything else) has had is thinking that "filenames" (and sometimes this is "fixed" to be "file ID's") are somehow special, and a totally separate thing from "file contents".

Really. It's a BUG. It's a deficiency in CVS and friends. And it's a deficiency that you have gotten so used to that you don't even see that it's simply obviously NOT TRUE.

You _cannot_ have a filename without the contents of that filename. That whole concept doesn't make sense, except in the twisted AND WRONG mental model of "files have identities even without content".

The whole point of git is that it is about "project state" and the history that binds those states together. People have kind of come to accept that, and a lot of people realize what it means, but I don't think you've really accepted what it means for something as simple as a "git add" command.

Again, totally ignore the index. Imagine that it doesn't exist. Imagine that you never actually learnt about it, and that none of the documentation ever mentions it, and just ask yourself:

	"What does 'adding a file' really mean?"

I mean _really_. It cannot be about the "filename", because a filename simply doesn't have any meaning alone. Remember what git is all about.

No, when you do a "git add", YOU DO NOT TALK ABOUT FILENAMES AT ALL.

	NOT EVEN CLOSE!

No. Git is, and has always been, all about tracking project content. The fact that CVS is crap, and thinks that "filenames" are special (and this causes major problems when you do renames), and the fact that SVN is crap, and things that "file identities" are special (and this causes major problems when you split a file or when two files join) is very much about THEIR F*CKING IDIOTIC FUNDAMENTAL BRAINDAMAGE!

So take five minutes to really think about that. Take an hour. Take a week. Ponder it.

What does it mean to "add" something to a project? It has _nothing_ to do with "filenames". Yeah, the filename obviously exists, but it's not something that exists on its own. You add the ONLY thing that git tracks.

You add CONTENT.

When you do "git add file.c" you aren't adding a filename to the list of files that git knows about. Not even CLOSE. No. You are really adding _content_ to the project you are tracking. You haven't bound it to a commit yet, but it's there. It's there both conceptually, and very much in a real technical sense too (you've literally added the git object that that file describes to the object database - the "commit" and "tree" objects to tie it all together is just waiting to be added, but they really just expose it - the actual file object has already been created when you do "git add".)

So yes, you very much ARE talking about CVS braindamage. The reason why

	git add file.c
	echo New line >> file.c
	git commit

commits the _old_ content, is very much because git is ALL ABOUT THE CONTENT. It has _never_ been about filenames. And it _shouldn't_ be about filenames, because that would be BUGGY AND BROKEN.

Sorry for shouting, but as long as you think "git add" adds a filename, you're just not getting it. And I think it's really sad that you don't even seem to understand that yes, this _is_ braindamage that has been forced upon you by decades of mental rape done by bad source control systems.

Please. File identities are _bad_ in the SVN kind of setting. The CVS kind of "filename == file identity" is even _worse_, but it's still exactly the same disease. It's the disease of thinking that metadata is somehow "different" from real data, and that "files" have identities that are somehow separate from the data they contain.

Face it, git is consistent, and if it acted the way you seem to expect it to act, it would actually be a BUG. Exactly because you cannot and MUST NOT think that "filename" is something that has meaning without "file content" (or "file type" and "file permissions" - they all go together).

And notice? NONE OF THIS HAS ANYTHING AT ALL TO DO WITH 'INDEX'!! The explanation above is not "this is how the index works". It's a much more fundamnetal issue of getting the right mental model, where the only thing that matters is contents.

So even without an index, "git add" should work the way it works, once you can just let go of the broken model that is CVS.

Please. Join me, Luke. The power of the git side is stronger. I am your father.

On Thu, 30 Nov 2006, Linus Torvalds wrote:

Great.  But let me repeat my last question:

Would it make sense for "git add" to do the same as "git update-index" on already tracked files? Given the explanation above this would make 100% sense to me.

Even for newbies this might help them understand the power of the index with only one command. "You _add_ your changes together before you may commit." That's simple to understand even for newbies. And then they'll start using the power of the index even without realizing it.

But right now, doing "git add" on an already tracked file simply does nothing. This is even worse than erroring out.

On Thu, 30 Nov 2006, Nicolas Pitre wrote:

Yeah, I think it would probably make sense. I also think it would make sense to rename "update-index" entirely, or at least offer other names for it (ie the "git resolved" suggestion).

In short - I agree that it's all just facets of the same thing: telling git that some part of the working tree is now in a state ready to be committed. Whether it's because we want to "add" content, or just mark it as no longer having conflicts, or any other reason.

Yeah, that's arguably a stupid thing to do ("you already added it, what do you want me to do?") but the choice we use is probably the worst of the three straightforward possibilities (ignore, update or error).

The _original_ "git add" was literally just this one-liner:

	#!/bin/sh
	git-update-index --add -- "$@"

which actually was better in this respect (it updated the content), but that didn't do sub-directories, so this is arguable a bug introduced by commit 37539fbd:

    [PATCH] Improved "git add"

    This fixes everybodys favourite complaint about "git add", namely that it
    doesn't take directories.

which started using 

	git-ls-files --others -z -- "$@"

together with the exclude files to generate the list of files to add. At that point, we lost files that already existed (since "--others" specifies just files we don't know about).

Linus Torvalds wrote:

So should we use then

        git-ls-files --cached --others -z -- "$@"

in git-add?

I'm very much for having git-add, -rm, -mv and -resolved as porcelain wrappers around git update-index, so there would be even less events when you have to use this plumbish command directly.

On Thu, 30 Nov 2006 Linus Torvalds wrote:

On Thu, 30 Nov 2006 22:16:29 +0100, Jakub Narebski wrote:

I'm happy with the direction of having several commands that take the place of update-index, each with its own name oriented toward what the user wants to do.

Obviously, "add", "mv", and "rm" have obvious places where the user wants to use them.

There's the merge case where "resolve" and "resolved" have both been floated as possible names.

It might even make sense to invent one more name for the case where the user wants to inform git that a file has been edited and that git should accept the new contents. It's the sort of "note that file is edited" operation that could be recommended to the user with "add; fix typo; commit" confusion.

Sure, "add" could be used again, and "update-index" clearly _works_ but it's a rather ugly name, (and already has "plumbing" functionality like --add and --remove that we don't want here).

If "resolved" is the name for the new command, then "edited" might work, but I think these adjectives don't work well next to the more active verbs that git normally accepts, (and yes, "mv" and "rm" are verbs even if horribly mangled spellings).

So I'd vote for "resolve" along with something else for the mark-as-edited case. Maybe "refresh"? That's the best I've thought of so far. Anyone else have a better suggestion? It does clash with the separate notion of "git update-index --refresh" which is a bit annoying. Any other suggestions for this?

-Carl

On 11/30/06, Carl Worth <cworth@cworth.org> wrote:

git mark

and git add becomes just a synonym for git mark.

Cheers,

On Thu, 30 Nov 2006 13:41:37 -0800, "Michael K. Edwards" wrote:

I actually thought of that. But compared to "refresh" it sounds more like something that suggests marking a file path rather than copying contents, so our dark lord might not approve of the wrong ideas it might allow to persist in brain-damaged heads.

It is nice and short, which is a bonus though.

-Carl

Dnia czwartek 30. listopada 2006 22:50, Carl Worth napisał:

What about "git update"? "git add" would also work, I think.

Hi,

On Thu, 30 Nov 2006, Jakub Narebski wrote:

... and suffer another thread a la "pull/merge"? Please no.

Ciao, Dscho

On Thursday 30 November 2006 22:37, Carl Worth wrote:

git stage

Hi,

On Thu, 30 Nov 2006, Carl Worth wrote:

I suggest "commit". How about this: after editing the file, you tell git that you finished editing it by doing

	git commit the-edited-file.txt

Hmmm?

Ciao, Dscho

On Thu, 30 Nov 2006, Johannes Schindelin wrote:

Sure.  ;-)

On Thu, 30 Nov 2006, Carl Worth wrote:

I disagree. "add" is beautiful. It is short, easy to remember, and transcend pretty much what the index is all about. And just because "add" and "edited" can be made into the same command is a pretty damn good reason not to create a separate command.

   You "add" changes to the changeset then you commit that changeset.

No need to care whether or not this is a new file, an edited file, etc.

Carl Worth <cworth@cworth.org> writes:

checkin.

You check things into index with "git checkin" and later commit the index with "git commit".

On Thursday 2006 November 30 21:03, Linus Torvalds wrote:

How about this:

git-update-index becomes plumbing only - never expect a user to run it.

Hence,

git-add becomes git-prepare and does
 a) update-index --add
    when the file being "prepared" is not tracked
 b) update-index
    when the file is already tracked

git-rm takes on "git-update-index --remove" (and --force-remove) with appropriate switches.

git-mv does what it does - it's already pretty perfect

git-cp gets added; even though it's simply "cp a b; git-prepare b"

Obviously with all the details that I've left out filled in.

Andy

On Friday 01 December 2006 08:34, Andy Parkins wrote:

...

Why can't it stay as git-add

It means "add the current state of the content to the index"

It has the useful property that git-commit -a can be seen as a short cut for add all the files in the working try and commit.

Nicolas Pitre <nico@cam.org> writes:

I think this makes sense and what we did in the original "git add".

On Thu, 30 Nov 2006, Junio C Hamano wrote:

Wonderful! We might be converging to something then.

Because, conceptually, it then becomes much easier to tell newbies about the index as follows (this could be pasted in a tutorial somewhere):

  Contrary to other SCMs, with GIT you have to explicitly "add" all
  the changes you want to commit together.  This can be done in a few
  different ways:

  1) By using "git add <file_spec...>"

     This can be performed multiple times before a commit.  Note that 
     this is not only for adding new files.  Even modified files must be
     added to the set of changes about to be committed.  The "git 
     status" command gives you a summary of what is included so far for 
     the commit.  When done you should use the "git commit" command to
     make it real.

     Note: don't forget to "add" a file again if you modified it after 
     the first "add" and before "commit". Otherwise only the previous 
     "added" state of that file will be committed.

  2) By using "git commit -a" directly

     This is a quick way to automatically "add" all modified files to 
     the set of changes and perform the actual commit without having to 
     separately "add" them.  This will not "add" new files -- those 
     files still have to be added explicitly before performing a commit.

  Here's a twist.  If you do "git commit <file1> <file2> ..." then 
  only the  changes belonging to those explicitly specified files will 
  be committed, entirely bypassing the current "added" changes.  Those
  "added" changes will still remain available for a subsequent commit.

  There is a twist about that twist: if you do "git commit -i <file>..." 
  then the commit will consider changes to those specified files 
  _including_ all "added" changes so far.

  But for instance it is best to only remember "git add" + "git 
  commit" and/or "git commit -a".

Doesn't it sounds nice? The index is being introduced up front without even mentioning it, and I think the above should be fairly palatable to newbies as well. Would only lack some enhancements to the commit template and the "nothing to commit" message so the user is cued about the fact that "current changeset is empty -- don't forget to 'git add' modified files, or use 'git commit -a'".

What do you think?

Nicolas Pitre <nico@cam.org> writes:

Yup. And it appears that we even agree that "intent to add" is a bad idea ;-).

I sense that you are inviting me to argue for reverting the other "git commit" braindead which is spelled "--only" (and worse yet, it is the default). I am very tempted.

Other than these "twists", I think it makes sense, and that is what I think.

But making sense to me does not necessarily validate that a tutorial document is great for its intended audience, since I lost git virginity long time ago. I can only endorse that the description is technically accurate.

On Thu, 30 Nov 2006, Junio C Hamano wrote:

I actually really like the current defaults for "git commit".

And I say that despite the fact that the defaults are _different_ from the original ones.

I like the fact that when you do "git commit filename", it really will commit _only_ that file, not the other files you added. It's logical.

So you can do

	echo a > file-a
	echo b > file-b
	git add file-a file-b

	git commit file-a

and it will only commit one of them. Now, admittedly, the status message that you get is slightly misleading (it says that "file-b" is untracked, which is not strictly true or at least somewhat confusing - it's untracked only within the context of that one commit), but this is all logically consistent.

And it's wonderful for noticing "Oh, sh*t, I forgot to commit that previous change, let's commit that other file first!". You can do things like this _despite_ the fact that you've added other files to the list of files to be committed.

The current semantics for "git commit" are really powerful, and really flexible. Yes, there are different "cases", but they all have their place:

 - "git commit" on its own:

    This is for things where you prepared things earlier: notably a merge, 
    for example. But it might be the "--amend" case too, where you simply 
    don't want to change any of the working tree, just the commit message.

   The fact that it has a nice fallback for the "nothing to commit" case 
   which basically just prints out the same thing as "git status" is just 
   gravy.

 - "git commit explicit/file anotherfile"

   This is great (and I use it) exactly for when you have random other 
   additions in your tree that are brewing, and you just want to commit a 
   particular fix that takes precedence. You may be in the middle of 
   something else, but you found a critical bug that needs to be fixed, 
   and while you _could_ do it on another branch, it's just a lot more 
   convenient to specify exactly what you want to commit.

 - "git commit -a"

   Sure, this is the common case, but just how painful is it to type those 
   extra few characters? And not defaulting to this is just sensible, 
   considering that "-a" really is not _that_ special, and the above two 
   cases simply aren't that unusual either. And it's just good to make 
   people have to _think_ about the fact that it commits everything that 
   have dirty.

At least for me, it turns out that the only mode I _never_ use personally is the "git commit -i" thing, which was actually the original behaviour, and which you'd think that I would encourage for that reason. But no. Of all the modes of "git commit", that's the one I think is the least important, and least interesting.

Of course, during a merge, you do need "-i" if you list files, but I think "-a" subsumes almost all cases (you _can_ use "-i file-list" or totally manually decide to have some extra edits you did that you don't want to commit together with the merge, but that's such a special case that I doubt anybody does it, so I don't think it's a big deal).

Anyway, we have "-i", and we don't force anybody to use it, so the fact that it's a bit odd and not that useful doesn't really matter. It certainly "fits" in the git commit family as another case, it's just not one of the important cases.

On Thu, 30 Nov 2006 15:40:58 -0800 (PST), Linus Torvalds wrote:

But, wait a second. What if I do my typo fix to file-a in between that "git add" and the "git commit". Why should "git commit" insist so vehemently on committing the _old_ state of file-a while "git commit file-a" so happily commits the _new_ state?

I'm really not seeing conceptual consistency to what "commit <files>" does compared to "commit".

The "commit <files>" case is a special-case of "commit -a" while "commit" is just plain different. See?

I do like the direction Nico is going with "git add" and his little tutorial. I think the first part of that, (without the twists), could make a very nice introduction to git.

The "twists" part though is quite a nightmare, and "twists on twists" is something that should be left to B-grade screenwriters, not technical writers, (nothing against Nico there---the stuff is really confusing).

But dropping back to the old default for "git commit explicit/file" really would only make things worse.

For "notably a merge", how often is this not equivalent to "commit -a" in your view?

Personally, I think I use --amend to change the tree of the last commit as often as I change just the message. Regardless of the relative frequency, I definitely use it often for both of those modes, so both should be supported.

As I mention above, this usage is exactly "commit -a" but restricted to just a few paths. Why isn't it just as brain-damaged to take contents from the working tree instead of the index here as in the case of my "add; fix typo; commit" example?

If we all agree it's the most common, then why isn't it the default?! It's the right thing for after resolving a conflicted merge, (since the working tree is where the user _first_ resolves things, before the changes make it to the index), the command is consistent with the behavior of "git commit explicit/file", and the existing "git commit" behavior would still be available in the rare cases it's desired, (and then, how painful would it be to type the few extra characters to ask for that?).

I know, I know. I said I'd give up and yet I keep harping on this stuff. I'm hopeless.

-Carl

On Thu, 30 Nov 2006, Carl Worth wrote:

Because that's what "git commit filename" means.

It means "commit the changes to this file".

Exactly what the command says. "git commit" says "commit everything I've told you to commit". While "git commit filename" says "commit the changes that I've made to this file".

Yes, they are two totally different cases, but nobody sane can claim that it is strange. Exactly _because_ you explicitly list the filename, that also means that you want the file content AT THAT TIME. If you don't list the filename, you obviously must be talking about committing something you did earlier.

On Thu, 30 Nov 2006 16:21:56 -0800 (PST), Linus Torvalds wrote:

Call me insane.

I think it is very strange that the following command sequences commit different content for myfile:

	echo old > myfile
	git add myfile
	echo new > myfile
	git commit

compared to:

	echo old > myfile
	git add myfile
	echo new > myfile
	git commit myfile

The difference there is very subtle and really does prevent a reasonable, "just ignore the index and you can use git comfortably".

Now, the behavior above can be explained. You and I both understand it just fine. And Nico's proposed tutorial document even explains it, (twists and all).

But it is strange, and I cannot see how the first case is at all useful.

The point is that by-and-large I don't ever want to commit anything that is not present in my working tree at the time I type "commit". That's precisely why I'm typing "commit" at that time.

If I do ever want to commit content that is not currently in my working tree, then I should have to do something exceptional. Otherwise, git will always have user-interface traps that will trip people up.

You might point at my first example and say that the timing of my "git add" command is the exceptional thing I did to trigger the old commit. But the second command above destroys that explanation. In the second, the timing of "git add" doesn't trigger any exceptional behavior at all. The new content of the file is still committed.

You're explaining the behavior, but not providing any justification for this being the right thing based on what a user actually wants to do.

-Carl

On Thu, 30 Nov 2006, Carl Worth wrote:

If you want to ignore the index, you'd always use "git commit -a".

So what's your point?

The old argument by incredulity. The fact that you cannot see it doesnt' change the fact that I use it all the time. Usually not for the same file, but committing something that is different in the index and in the working tree is my normal behaviour - it's how any number of things work (like merging while you have dirty state, or applying patches while you have changes).

So, let me condense the argument:

 - your argument is that cannot understand how anybody would ever want to 
   use this.

That's really what it all boils down to. That's your ONLY argument. You didn't actually answer any of me explaining _why_ it's how it works, and why it _has_ to be why it works. Your argument just boils down to "I can't believe it's useful to be consistent".

My argument is:

 - the current behaviour is actually very powerful, and I've given 
   examples of when I actually do use it (and whether you know it or not, 
   they are also things you've probably done - the "pull from somebody 
   else with dirty files" is actually exactly the same thing, you just 
   never realized it just because it happened to be the same thing on two 
   different files)

 - the "git add file ; change file ; git commit" behaviour is absolutely 
   REQUIRED once you get the whole "git tracks content" logic. Doing 
   anything but committing the old version (the one you added) would be 
   illogical and wrong, because it's strictly against the whole POINT of 
   tracking content.

So. That's what it boils down to. Your personal incredulity against fundamental concepts and real usage.

The reason I like UNIX is that it has "fundamental concepts". And surprise, surprise, a lot of the same issues are at play in "git" too. Pretty much _all_ the behaviour (apart from actual _naming_ issues) really come from fundamental concepts, and _not_ doing special cases.

I guarantee you, in the end you're better off building a world-view on a coherent guiding logic, than on "personal incredulity" or "I can't believe that anybody would actually ever want to do that".

As an X developer, don't you get tired of hearing people saying "I can't believe anybody would ever want to use a network transparent protocol and do graphics from another machine"? The non-X people (and every single "let's replace X with something more efficient" discussion) always tend to take that approach.

Same thing. It really doesn't matter what you think is "normal". What matters a lot more in the end is that you keep a coherent set of concepts, so that once you really learn the concepts, it all makes sense.

And in git, the over-arching concept for just about _anything_ is "git tracks contents, and filenames don't matter". The behaviour of "git add" and "git commit" is just a small detail in that whole picture.

On Thu, 30 Nov 2006 17:32:22 -0800 (PST), Linus Torvalds wrote:

I'm sorry, Linus. I'm really failing to communicate with you somehow here.

I _do_ use this all the time too. I don't need you to explain it to me again. I get it, I like it, I use it. That's not at all what I'm trying to discuss here.

Nope. I get it and I use it. Most commonly I do things like "update-index;commit" to separate independent changes, (I could use "git commit explicit/files" instead but I *gasp* actually enjoy being able to manually stage things in the index first).

What I'm trying to say is that the _defaults_ are not well geared toward helping new users do what they want to with git.

I do understand how git works. I understand your points about filenames and content. I understand that.

One thing you never really answered was my conversation with a new user that left the user with the impression of "git is bizarre". How can I fix that conversation?

The idea I have for improving this is to do nothing more than change the default semantics for one command to the semantics of that same command with a single-letter option. You keep saying that my idea is brain-damaged, (that carrying the filename from "git add" to "git commit" without the old contents would violate something fundamental). But "commit -a" already exists and does _exactly_ what I'm asking for here. People use that without destroying git's model. Why would it necessarily be so evil to do that by default?

By the way, none of the "confusion" arguments are coming from me directly. They're coming through me by proxy, (on behalf of people I've seen deciding against using git). Personally, I love git and plan to continue using it forever. I'm very happy with it. I'm even comfortable with all of the aspects of its interface that I'm arguing against changing here, (which really are not big fundamental things). I just think its harder for new people to get to that point than it should be.

Here's another question based on that example. You say above that committing the old version is the only logical thing to do. Separately, you say that after adding file-a and file-b it's often convenient to be able to commit file-a without file-b.

So, given the following:

	echo a > file-a
	echo b > file-b
	git add file-a file-b
	echo a-not-ready-yet > file-a

How could I do both of these at once? That is, how can I commit the _old_ version of file-a, (the only logical choice), but not commit file-b? There's no commit command that does that is there? (I don't think that's a fault in git since I think this would be a fairly exceptional thing to do).

[I'll snip the rest, because I really do think I understand, agree with, and accept the fundamental concepts of git.]

But as for "real usage", by far, the thing I want to do most often is to commit the state of my files as the exist in my working tree at the time of commit. Any time I do anything else, I'm well aware of it, and I'm quite happy to do "special" index-manipulating commands, (taking advantage of the fundamental, content-tracking nature of git to do it).

I think the defaults for git-commit should reflect that aspect of real usage. That's all.

-Carl

On Thu, 30 Nov 2006, Carl Worth wrote:

I really think we're better off just telling people how things work (with practical examples, and _not_ by trying to explain things at too high a conceptual level).

I don't think people generally are all that stupid, and I think it's actually counter-productive to try to basically lie about how things work. It will just make it harder for people later.

I really _think_ that a lot of that is in the documentation being overly technically oriented and talking often about the technical side of how things work in the index, rather than the purely user side of what that _results_ in.

I really believe that people can understand the concept of "git add" squirrelling away the whole state of the file at add-time, and suddenly it's not all that complicated. Also, it's not even something that people really need to worry about, and I think we should make that more clear.

In other words, the documentation could _literally_ give the example of

	git add file.c
	.. change file.c ..
	git commit
	git diff file.c

and talk about this issue up-front, but then just say outright to people that "if you don't want to know about these details, you can always just use 'git commit -a', and you'll never really even notice".

There really isn't all that much to "hide". I think we've sometimes done a horrible job on _presentation_, but I also think that it's better to give examples of thigns like this, and then tell people not to worry, because they'll never need to care unless they actually start doing something fancy.

So all your examples of "badness" aren't really all that bad. Newbies should be told:

 - use "git commit -a" normally (with pointers on fancier usage)

 - and yes, we obviously should change the message to say "git commit -a"
   instead of "git update-index"

 - do NOT use the "-m" flag, and look at what git tells you in the
   commit message!

   This is actually important, because even for non-newbie users, the git 
   commit message for a conflicting  merge contains useful information, 
   and people should read it. It lists the conflicting filenames for a 
   reason, namely so that you can talk about what the conflicts in 
   question _were_.

[ Btw, I can't stress that last point enough. Using the "-m" flag should 
  really really REALLY be discouraged. It is almost always a horrible 
  mistake, not just because it means that you miss what git will tell you 
  about which files are getting checked in, but because it invariably 
  leads to bad single-line commit messages.

  In my personal opinion, the "-m" flag is really only good for scripts 
  and sending example git sequences to each other, ie it's there for 
  automated "do this" kind of scripting, and using it for real work should 
  be a castration offence, so that you don't perpetuate your genes.

  (I've seen _way_ too many projects where there's a lot of "update 
  version" one-liner commit messages. Damn, I can understand that in CVS, 
  where the logs are useless _anyway_, but in git it shouldn't be 
  allowed!). ]

Ok, with that rant out of the way, my _point_ is that we're actually much better off educating users about _why_ git is different, than trying to lie to them and say "it's just like CVS by default, but when you're a real man, we'll show you how you can rock your world".

On Thu, 30 Nov 2006 18:44:41 -0800 (PST), Linus Torvalds wrote:

Sure. There's no need to lie to about how things work. And I agree that would cause later problems. As far as getting the message out, we have several different ways to do that:

  1. In-person presentations, demos, tutorials

  2. Written tutorials

  3. Reference documentation

  4. Output from git commands

  5. The default behavior of git commands

And that list is roughly sorted from most to least "information bandwidth". The limited bandwidth at the bottom-most levels in my list, (together with the fact that people often start trying the tool at only those levels), means we have to be even that much more careful about the messaging there.

In my experience, I think we succeed quite well at level 1. In person, we get immediate feedback from the user and it's easy to catch and cut-off any user confusion up-front. "Oh, you don't even need to run that command...use this instead", etc.

As for the other levels, you hit on most of them in your comments:

Yes, this is a big problem at level 3. Things like the "man git-diff" scare factor that Ted pointed out. So let's work to fix all of that.

One trick with saying "we just need to document this better" to avoid the confusion is that approach assumes that the users are actually _reading_ the right documentation. Now, we're currently making this harder than it should be at level 4 by doing things like sending users to the documentation for update-index. But still, if we can make things less surprising in some cases _without_ needing the documentation, then we make it that much easier.

Yes, adding lots of good examples to the written documentation will help, (anyone that reads it at least).

...

...

So here you're arguing for documenting the heck out of "commit -a" at all of levels 1-4. If we're going to do that, why not just go the next tiny step and make it work as "git commit" by default, (which people _will_ try). If we can say, "come from hg or bzr and things will just work", people can try that, be satisfied that git isn't bizarre, and then we can teach where git's actually superior.

Interesting. I do use -m almost exclusively. I do that for speed I think, (but I do do multi-line commit messages). The only drawback I was aware of was that I'm doing manual word wrapping, but I might start trying this to see information in the commit message, (instead I've been checking first with "git diff --cached").

I still don't see any lie here. If we all agree that "git commit -a" is the most commonly desired form, it's what users expect by default (based on _any_ other system they might be coming from), and we agree we need to mention it a lot more at every level of the documentation---given all that, why do we insist on having something else be the default?

Because it gives us an opportunity to teach about the power of the index to anyone that gets confused and complains? That strategy ignores everyone that gets confused and just leaves without talking to us.

I'm talking about changing the default of what "git commit" does, yes, but it can still be documented honestly as to what it really does and why. It would fit in just fine with Nico's new documentation for "git add" for example, and "git diff" doesn't need to be changed at all, (but it's documentation should be made much less scary).

-Carl

I think Linus is right that the semantics of "git commit" and "git add" are not broken and do not need fixing. But Carl and others are right that they are not self-explanatory to most people, whether or not they have been tainted by CVS and its ilk. Maybe this is a job for a little contextual documentation (aka hand-holding) to accompany the tutorial and reference docs.

How about we add a set of "expert" flags in the config, gating access
to non-intuitive behavior and to idioms that should be discouraged in
casual use.  For instance, with an empty config, "git commit -m" fails
with a message to the effect of:
    "As a general rule, you shouldn't use the -m flag unless you're
scripting git for automation purposes.  If this is what you are doing,
or if you insist on committing without feedback about the state of
your tree, you need to set the 'expert-commit-message' flag".

Likewise, when your working copy does not match HEAD, "git commit"
with neither -a nor an explicit list of files fails, saying:
    "'git commit' on a dirty working copy does the Right Thing!  But
some people find the Right Thing counter-intuitive at first.  Either
stick to 'git commit -a' or read the docs and set the
'expert-commit-dirty' flag."

And "git add" still does the right thing, but warns:
    "Remember, git is not CVS.  'git add' has taken a snapshot of the
current _contents_ of the newly added files, not just their names.
From now on, you will need to 'git mark' edits to them if you want
them to be part of the next commit, just like edits to files that have
already been committed.  This warning can be suppressed by setting the
'expert-add-content' flag."

Note that 'expert-commit', etc. should NOT change the semantics of any command that doesn't error out. They should just enable idioms that a novice user is likely to try and get unexpected results. They should be overridable from the environment, of course, either one by one (export GIT_EXPERT_COMMIT_MESSAGE=y) or wholesale (export GIT_I_GROK_IN_FULLNESS_UP_TO=1.4.4).

Cheers,

Linus Torvalds <torvalds@osdl.org> writes:

Hmmm. I did not make my judgement based on what the command did in the ancient _original_ version. The --only behaviour being totally different from others (from "index is the only thing that matters" point of view) was what bothered me. It did not feel logical.

However, if you put it this way:

that makes tons of sense.

What's "logical" largely depends on how things ought to behave in the mental model, and the mental model you would form largely depends on how things are explained to you.

Linus Torvalds wrote:

So, in short, -i is easier to explain, but is also least used. Perhaps also because one can simply update index with <files> before git commit instead of doing "git commit -i <files>".

On Thu, 30 Nov 2006, Junio C Hamano wrote:

No no no !

I argued for that at the time and I still stands behind that change !

Nicolas Pitre wrote:

"This is because git tracks content, so what you're really 'add'ing to the commit is the *content* of the file in the state it is in when you 'add' it."

.. "add" all tracked and modified files to ...

me likes

On Friday 01 December 2006 08:59, Andreas Ericsson wrote:

add all tracked and modified content

...

me too

Carl Worth wrote:

Exactly. We had a tutorial for the project I contribute to (admittedly the initial users were all used to how CVS worked) and while a number of people got the concept of the index and were fairly happy with it, it did add to the confusion of the tutorial, so now it doesn't mention the index at all.

The tutorial introduced it as a staging area for commits, but the trouble is that once you work like this you have to remember that "git-diff" won't show you what will be committed, so you have to use "git-diff-index" as well. If you get them mixed up then you end up committing the wrong thing.

Here's a selected list of the commands introduced in the tutorial, without mentioning the index: git diff git commit -a git commit <changed-files> git reset HEAD^ git cherry-pick

Here would be the same entries, but introducing the index too: git-update-index git diff git diff-index git commit git commit -a git commit <changed-files> git reset HEAD^ git reset --soft HEAD^ git cherry-pick git cherry-pick -n

The tutorial then goes from having ~12 common commands to learn up to ~17.

Having the index exposed for even simple operations means that the user has to initially learn three states instead of two. The worst thing about the index is that it is a limbo state. The committed content is in the history and can be viewed by gitk (and other tools that the user will be introduced to later) and the working tree is exactly what the user sees in their editor. Having a hidden state isn't very good from an HCI point of view.

Once you understand the concept of the index, it is very useful. However, new users should be shielded from it if at all possible.

I'm not advocating making "git-commit" equal to "git-commit -a" as I've been frustrated by command's semantics changing in git before. I can understand long-time git users would automatically try to use "git-commit" to just commit their index and get annoyed if it did something unexpected. Therefore, I would advocate there being no default behaviour for "git-commit" except for displaying a help message, and making previous "git-commit" users now use "git-commit -i".

Robert Shearman wrote:

Index is accessible, just like committed contents. The fact that gitk, qgit, git-gui doesn't display state of index is their limitation.

Jakub Narebski <jnareb@gmail.com> wrote:

Actually git-gui shows the index, but not quite as well as diff and friends would.

But based on this thread I had a major realization: git-gui is totally wrong in how it displays files (and therefore gitool is too!). I'm going to rewrite that part of git-gui's UI, hopefully early next week.

Linus is right: To deny the index is to deny git itself. Trying to hide part of the index in git-gui is just wrong and makes things like merge conflict resolutions harder, not easier.

Actually qgit let's you see state index for each file when committing with qgit commit dialog.

Indeed you can also choose to update only the index state of a file without committing, ie without create a new tree object.

Internally, the index state of each file is known and tracked. No other interface is provided, as example a diff, just because I found it confusing and of little concrete help.

Of course if knowning the index state became important to perform some concrete and quite common operation I could add whatever GUI would suite.

Suggestions are welcomed ;-)

On Thu, 30 Nov 2006 09:13:52 -0800 (PST), Linus Torvalds wrote:

...

OK. I'll try to not mention the "index" any more in any postings here.

And can we agree that any time git spits out a message directing the user to use update-index that that's a (minor) user-interface wart?

I think the response that would come from the people that are confused is:

	"But I told git I wanted to track this file when I said 'git
	add' long ago. Why is it making me tell it again."

As you mentioned, all systems have _some_ mechanism for keeping track of the files to be committed, (and git's is only unique in having a name and providing more functionality for direct manipulation and lots of extra information in the case of a merge conflict).

But I think most every system out there _except_ git default to a state of committing every file it "knows" about as it exists in the working tree, and then allowing the user to restrict that behavior to some subset of the files.

Git allows the same subsetting, and has behavior that is very similar to these other systems when the user provides a list of files.

Git also provides a unique mode in that users can "stage" file state to be committed later in spite of subsequent different changes being made to the same files in the working tree that won't be committed. Some git users love this functionality. But mentioning it to new users does scare them off to some extent, ("Why would I _want_ to do that?", "What if that happens accidentally?").

And I think one thing that happens is that the current defaults naturally lead users to hear about this "scary" functionality, even if the presenter, (whether a human or printed documentation), isn't trying to go that direction:

Presenter:	So use "git commit -a" here.

New user:	Why -a?

Presenter:	To tell git that you want to commit all files rather
		than having to list them all on the command line.

New user:	Why not just "git commit" for that then?

Presenter:	Because that's something else.

New user:	What's that?

Presenter: 	It lets you stage things---stuff you think is ready to
		commit, but when you want to delay that commit until
		after making other changes to the files that you don't
		want to commit.

New user:	What? Really? That's bizarre.

Presenter:	It can be useful in some situations. But for now,
		just use "commit -a" and it will do what you want.

And at this point the user either trusts me, accepts it, gives git a try and falls in love, or the user gives up and uses something else.

I think the above accurately captures the essence of actual conversations I've had with new new users. And I'd be glad to take suggestions on how to improve what I say here. But it's that feeling of "git is bizarre" that I'd like to reduce, and I'd like to improve the success rate of the conversation, (though I think I've done pretty well for people that trust me).

And note that the same kind of conversation happens when using git directly with tutorials and man pages, but without a human presenter. Only, there the conversation is much worse. First, it's harder to pull off "just trust me and give it a try" in technical documentation. Second, the documentation does not do a good job of letting the user know when they're getting more technical information than they need.

For example, there's "git status", (used by "git commit"), that directs the user to "git update-index". Then there's the documentation of git-commit that says "Updates the index file...and makes a commit object."

And so far my best response to those problems is to short-cut them by improving the defaults of git-commit, (the documentation should be improved too, and I did submit a patch to get "update-index" out of git-status output for example).

Anyway, I'm repeating myself on some of these details, but only because some people still haven't seemed to grasp the real, new-user confusion that arises here.

That's really what I'm trying to reduce with all the talk about "commit all known files by default".

Yes. And it makes sense for the user to be able to say "unless I tell you differently, I want to always commit the working-tree state of <files> with every commit".

-Carl

On Thu, 30 Nov 2006, Carl Worth wrote:

If so, you should make it a special case.

I refuse to believe in the "people who know what the hell they are doing should work more at it" philosophy.

The "git commit -a" behaviour as it is now is better than the alternatives, exactly because it's more flexible. It _allows_ you to not commit anything at all (and as already mentioned, there are cases where that is exactly what you want).

If you want to have a "-a by default", then that you require _you_ to do more work, and no, it's NOT an excuse to say "I'm a clueless newbie, and I don't know how to set a config option, so I think it's the smart and beautiful people who should suffer for my shortcomings".

You can even do it by doing an alias like

	[alias]
		ci = commit -a

and then you can revel in your CVS-induced mudpit all you want. Just don't try to convince people who have gotten over that braindamage to live in the same muck with you.

Problem solved. For all I care, we can make that alias a default one, so people who just can't get their mind out of the gutter that is CVS can continue with their evil ways.

On Tue, 28 Nov 2006, Junio C Hamano wrote:

Perhaps you'd be happier if the command to commit what "git diff --cached" shows were "git commit --cached" rather than "git commit -i"? (Or if they were both --index; how did we miss that last September?)

It seems logical to me that "git commit" would commit the changes shown by "git diff" (in addition to changes in the index, of course, which are so obvious as to need no mention). I personally check with "git diff" and commit if everything there looks good; otherwise I tweak stuff until it does. And if there are a lot of changes, and all of those in some files look good, but those in other files need work, I can "git update-index" the ones I know I like so I don't have to go through them each time I'm checking on other stuff next time.

That is not so clear to me. Maybe you're putting changes into the index to reduce the noise in "git diff", by updating everything that's unquestionable while you examine the other stuff. I think that everything in the index is clearly in the next commit, but it's obviously not true that everything in the next commit is in the index (because you might not be done updating things yet).

I personally think that --amend should default to retaining the same tree, with options available for using the index or -a or paths. Using the index by default is just as wrong as -a; you're just more careful about it by experience. The index holds stuff to go in the *next* commit, but --amend generates a new version of the *previous* commit, so the logical basis for the new previous commit is the old previous commit's tree, leaving the index alone.

	-Daniel

On Thu, 30 Nov 2006, Junio C Hamano wrote:

And personally I think this bastardizes the index.

I think so far the index is not a problem. the problem is in the UI and in the doc. With a "git add" as I described it I don't think the user will ask for such thing.

And let's have a "git diff --commit" be a much meaningful alias for the appropriate diff argument (which argument I always forget about myself) needed to show what is going to be committed.

Hi,

On Thu, 30 Nov 2006, Junio C Hamano wrote:

I think this is so important, that I vote for including this email as Documentation/howto/explain-why-git-is-better-than-cvs.txt

Ciao, Dscho