threads / discuss / 9002

Volume of commits

Subject: Volume of commits

## tl;dr

14 messages between Jul 12, 2007 and Jul 21, 2007.

replies: 13people: 9as markdown or json

Fredrik Tolf· Jul 12, 2007, 13:16 UTC · lore
Hi List,

I was wondering -- whenever I see Git patches committed to projects like the Linux kernel or Git itself, the commits always seems to be committing rather large changes and be rather well-defined in terms of what they change.

When I develop for myself, I usually commit incrementally quite a bit, if for no other reason because Git won't let me switch between branches if I don't commit first. I usually try to keep my commits well-defined, but I don't manage to get anywhere close to what I see when I look at the history of Linux or Git.

So what I'm wondering is how you people manage to do this? Do you actually always commit changes this way (and, in that case, how do you switch between branches)? Or do you somehow aggregate the smaller commits into larger patches and recommit them? Or is there some third possibility that I'm missing?

Fredrik Tolf
VMiklos· Jul 12, 2007, 13:29 UTC · re: Fredrik Tolf · lore

Re: Volume of commits

Hello,
Na Thu, Jul 12, 2007 at 03:16:47PM +0200, Fredrik Tolf <fredrik@dolda2000.com> pisal(a):
Show 5 quoted lines
> So what I'm wondering is how you people manage to do this? Do you
> actually always commit changes this way (and, in that case, how do you
> switch between branches)? Or do you somehow aggregate the smaller
> commits into larger patches and recommit them? Or is there some third
> possibility that I'm missing?

you can cherry-pick the relevan patches to a separate branch and commit then at once (cherry-pick -n), or can merge --squash to archive something similar

- VMiklos
Johannes Schindelin· Jul 12, 2007, 13:59 UTC · re: VMiklos · lore

Re: Volume of commits

Hi,
On Thu, 12 Jul 2007, VMiklos wrote:
Show 10 quoted lines
> Na Thu, Jul 12, 2007 at 03:16:47PM +0200, Fredrik Tolf <fredrik@dolda2000.com> pisal(a):
> > So what I'm wondering is how you people manage to do this? Do you 
> > actually always commit changes this way (and, in that case, how do you 
> > switch between branches)? Or do you somehow aggregate the smaller 
> > commits into larger patches and recommit them? Or is there some third 
> > possibility that I'm missing?
> 
> you can cherry-pick the relevan patches to a separate branch and commit 
> then at once (cherry-pick -n), or can merge --squash to archive 
> something similar

What I do these days is committing early, and often, and then rearrange with "git rebase -i". Beware: this is a 1.5.3 feature! But IMHO it is really something you can look forward to.

A little diagram hopefully explains what you can do with it:
- upstream
 \
   A - B - C - D - E - F - G - HEAD with a messy history

In this case, "messy history" means that there are tiny patches which are often in the wrong order, or should be squashed into one commit. "git rebase -i upstream" presents you with the list of A - HEAD, and you can reorder the patches. If you want to, you can combine ("squash") some into one commit, or you can skip it, by removing the corresponding line.

The result can look like this:
- upstream
 \
   B+C+F - A - G+E

Note that D is missing, which can be desirable, for example when you made a commit only introducing lots and lots of debug output. See, nobody has to know what you did. The end result will look elegant.

This demonstration of why distributed SCM is good ("it lowers the 
embarrasment factor") was brought to you by Git.
 
Ciao,
Dscho
Sven Verdoolaege· Jul 13, 2007, 10:30 UTC · re: Johannes Schindelin · lore

Re: Volume of commits

On Thu, Jul 12, 2007 at 02:59:53PM +0100, Johannes Schindelin wrote:
Show 5 quoted lines
> In this case, "messy history" means that there are tiny patches which are 
> often in the wrong order, or should be squashed into one commit.  "git 
> rebase -i upstream" presents you with the list of A - HEAD, and you can 
> reorder the patches.  If you want to, you can combine ("squash") some 
> into one commit, or you can skip it, by removing the corresponding line.

If I squash a whole series of commits, how do I prevent git-rebase -i from firing up an editor after every single commit in the series?

Also, if I do the following:
bash-3.00$ git init
Initialized empty Git repository in .git/
bash-3.00$ for i in a b c; do touch $i; git add $i; git commit -m $i -a; done
Created initial commit 19a8485: a
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 a
Created commit 4a00f85: b
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 b
Created commit defe3b5: c
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 c
bash-3.00$ git rebase -i  HEAD~2

and then replace the second "pick" by "squash", then I get presented a commit message that contains the commit message of "c" twice and after the rebase there are still three commits in the history. This is with git version 1.5.3.rc1.10.gae1ae (on top of v1.5.3-rc1-4-gaf83bed).

skimo
Alex Riesen· Jul 13, 2007, 12:46 UTC · re: Sven Verdoolaege · lore

Re: Volume of commits

On 7/13/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> If I squash a whole series of commits, how do I prevent git-rebase -i
> from firing up an editor after every single commit in the series?
It is started only for squashed commits. But you can set VISUAL to true or ":".
> a commit message that contains the commit message of "c" twice and
> after the rebase there are still three commits in the history.

Known, fixed (but not yet approved) in [PATCH] Fix git-rebase -i to allow squashing of fast-forwardable commits

Johannes Schindelin· Jul 21, 2007, 17:09 UTC · re: Sven Verdoolaege · lore

[PATCH] rebase -i: call editor just once for a multi-squash

Sometimes you want to squash more than two commits. Before this patch, the editor was fired up for each squash command. Now the editor is started only with the last squash command.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
	On Fri, 13 Jul 2007, Sven Verdoolaege wrote:
	> If I squash a whole series of commits, how do I prevent 
	> git-rebase -i from firing up an editor after every single commit 
	> in the series?
	By applying this patch ;-)
 git-rebase--interactive.sh    |   56 +++++++++++++++++++++++++++++++++-------
 t/t3404-rebase-interactive.sh |    9 ++++++
 2 files changed, 55 insertions(+), 10 deletions(-)
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index a2d4d09..579a45e 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -19,6 +19,8 @@ require_work_tree
 DOTEST="$GIT_DIR/.dotest-merge"
 TODO="$DOTEST"/todo
 DONE="$DOTEST"/done
+MSG="$DOTEST"/message
+SQUASH_MSG="$DOTEST"/message-squash
 REWRITTEN="$DOTEST"/rewritten
 PRESERVE_MERGES=
 STRATEGY=
@@ -158,6 +160,38 @@ pick_one_preserving_merges () {
 	esac
 }
 
+nth_string () {
+	case "$1" in
+	*1[0-9]|*[04-9]) echo "$1"th;;
+	*1) echo "$1"st;;
+	*2) echo "$1"nd;;
+	*3) echo "$1"rd;;
+	esac
+}
+
+make_squash_message () {
+	if [ -f "$SQUASH_MSG" ]; then
+		COUNT=$(($(sed -n "s/^# This is [^0-9]*\([0-9]\+\).*/\1/p" \
+			< "$SQUASH_MSG" | tail -n 1)+1))
+		echo "# This is a combination of $COUNT commits."
+		sed -n "2,\$p" < "$SQUASH_MSG"
+	else
+		COUNT=2
+		echo "# This is a combination of two commits."
+		echo "# The first commit's message is:"
+		echo
+		git cat-file commit HEAD | sed -e '1,/^$/d'
+		echo
+	fi
+	echo "# This is the $(nth_string $COUNT) commit message:"
+	echo
+	git cat-file commit $1 | sed -e '1,/^$/d'
+}
+
+peek_next_command () {
+	sed -n "1s/ .*$//p" < "$TODO"
+}
+
 do_next () {
 	test -f "$DOTEST"/message && rm "$DOTEST"/message
 	test -f "$DOTEST"/author-script && rm "$DOTEST"/author-script
@@ -194,17 +228,19 @@ do_next () {
 			die "Cannot 'squash' without a previous commit"
 
 		mark_action_done
-		MSG="$DOTEST"/message
-		echo "# This is a combination of two commits." > "$MSG"
-		echo "# The first commit's message is:" >> "$MSG"
-		echo >> "$MSG"
-		git cat-file commit HEAD | sed -e '1,/^$/d' >> "$MSG"
-		echo >> "$MSG"
+		make_squash_message $sha1 > "$MSG"
+		case "$(peek_next_command)" in
+		squash)
+			EDIT_COMMIT=
+			cp "$MSG" "$SQUASH_MSG"
+		;;
+		*)
+			EDIT_COMMIT=-e
+			test -f "$SQUASH_MSG" && rm "$SQUASH_MSG"
+		esac
+
 		failed=f
 		pick_one -n $sha1 || failed=t
-		echo "# And this is the 2nd commit message:" >> "$MSG"
-		echo >> "$MSG"
-		git cat-file commit $sha1 | sed -e '1,/^$/d' >> "$MSG"
 		git reset --soft HEAD^
 		author_script=$(get_author_ident_from_commit $sha1)
 		echo "$author_script" > "$DOTEST"/author-script
@@ -213,7 +249,7 @@ do_next () {
 			# This is like --amend, but with a different message
 			eval "$author_script"
 			export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_AUTHOR_DATE
-			git commit -F "$MSG" -e
+			git commit -F "$MSG" $EDIT_COMMIT
 			;;
 		t)
 			cp "$MSG" "$GIT_DIR"/MERGE_MSG
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 43a6675..8206436 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -65,6 +65,7 @@ cat > fake-editor.sh << EOF
 #!/bin/sh
 test "\$1" = .git/COMMIT_EDITMSG && {
 	test -z "\$FAKE_COMMIT_MESSAGE" || echo "\$FAKE_COMMIT_MESSAGE" > "\$1"
+	test -z "\$FAKE_COMMIT_AMEND" || echo "\$FAKE_COMMIT_AMEND" >> "\$1"
 	exit
 }
 test -z "\$FAKE_LINES" && exit
@@ -212,4 +213,12 @@ test_expect_success 'verbose flag is heeded, even after --continue' '
 	grep "^ file1 |    2 +-$" output
 '
 
+test_expect_success 'multi-squash only fires up editor once' '
+	base=$(git rev-parse HEAD~4) &&
+	FAKE_COMMIT_AMEND="ONCE" FAKE_LINES="1 squash 2 squash 3 squash 4" \
+		git rebase -i $base &&
+	test $base = $(git rev-parse HEAD^) &&
+	test 1 = $(git show | grep ONCE | wc -l)
+'
+
 test_done
-- 
1.5.3.rc1.16.g9d6f-dirty
Johannes Schindelin· Jul 21, 2007, 17:12 UTC · re: Sven Verdoolaege · lore

Re: Volume of commits

Hi,
On Fri, 13 Jul 2007, Sven Verdoolaege wrote:
Show 21 quoted lines
> [...] if I do the following:
> 
> bash-3.00$ git init
> Initialized empty Git repository in .git/
> bash-3.00$ for i in a b c; do touch $i; git add $i; git commit -m $i -a; done
> Created initial commit 19a8485: a
>  0 files changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 a
> Created commit 4a00f85: b
>  0 files changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 b
> Created commit defe3b5: c
>  0 files changed, 0 insertions(+), 0 deletions(-)
>  create mode 100644 c
> bash-3.00$ git rebase -i  HEAD~2
> 
> and then replace the second "pick" by "squash", then I get presented
> a commit message that contains the commit message of "c" twice and
> after the rebase there are still three commits in the history.
> This is with git version 1.5.3.rc1.10.gae1ae
> (on top of v1.5.3-rc1-4-gaf83bed).

It no longer reproduces, which probably means that I inadvertently fixed the bug ;-)

Ciao, Dscho

Karl Hasselström· Jul 12, 2007, 13:49 UTC · re: Fredrik Tolf · lore

Re: Volume of commits

On 2007-07-12 15:16:47 +0200, Fredrik Tolf wrote:
> When I develop for myself, I usually commit incrementally quite a
> bit, if for no other reason because Git won't let me switch between
> branches if I don't commit first.

You can switch branches even if you have local modifications if you use the -m flag to git-checkout.

Show 9 quoted lines
> I usually try to keep my commits well-defined, but I don't manage to
> get anywhere close to what I see when I look at the history of Linux
> or Git.
>
> So what I'm wondering is how you people manage to do this? Do you
> actually always commit changes this way (and, in that case, how do
> you switch between branches)? Or do you somehow aggregate the
> smaller commits into larger patches and recommit them? Or is there
> some third possibility that I'm missing?

When it's time to post patches for review on the mailing list, people clean up the history so that it'll be easier for the reviewers to read (and to make the permanent history easier to read in case the patches are accepted). Of course, with practice one can write clean patches to begin with, at least for simpler changes, but the conceptual workflow is along the lines of:

  1. Do the changes, and make sure everything works.
  2. Rewrite your changes as a series of easy-to-read and mostly
     independent patches, and make sure that everything compiles and
     just generally makes sense at all intermediate steps. The
     maintainer might very well accept some of your patches but not
     all!
  3. Repeat until you're satisfied with the result.

The rationale for spending time making history legible is the same as for making the end result code legible: stuff is written once by one person, and read many times by many people.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Karl Hasselström· Jul 12, 2007, 14:03 UTC · re: Fredrik Tolf · lore

Re: Volume of commits

On 2007-07-12 15:16:47 +0200, Fredrik Tolf wrote:
> Or do you somehow aggregate the smaller commits into larger patches
> and recommit them? Or is there some third possibility that I'm
> missing?

Aggregating commits and recommitting is easy, so that's a common tool, I'd say. But it's equally possible to take a large commit and pick it apart, for example by editing the patch by hand and reapplying it, or by using the hunk selection feature of git-gui.

For example, if you have just committed several changes as one big commit, you can do

  $ git reset HEAD^

to undo the commit but retain the changes in your working tree, and then use git-gui to select a subset of the changes and commit them, then select another subset and commit that, and so on.

If you need to edit a commit that isn't HEAD, you can use git-reset to go back to the commit you want to edit, do the editing, and then use git-rebase to reapply the other commits on top of the changed commit.

In general, there are a thousand ways to use git to rewrite history, either "by hand" or by using tools such as StGIT. (StGIT is what I personally use most of the time. I find it a good tool for producing readable patch series.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Fredrik Tolf· Jul 12, 2007, 14:51 UTC · re: Karl Hasselström · lore

Re: Volume of commits

Karl Hasselström <kha@treskal.com> writes:
Show 10 quoted lines
> On 2007-07-12 15:16:47 +0200, Fredrik Tolf wrote:
>
>> Or do you somehow aggregate the smaller commits into larger patches
>> and recommit them? Or is there some third possibility that I'm
>> missing?
>
> Aggregating commits and recommitting is easy, so that's a common tool,
> I'd say. But it's equally possible to take a large commit and pick it
> apart, for example by editing the patch by hand and reapplying it, or
> by using the hunk selection feature of git-gui.

I see. Initially, it sounds like a lot of work, but the more I think about it, the more I realize that it probably isn't that bad.

> [...]
> If you need to edit a commit that isn't HEAD, you can use git-reset to
> go back to the commit you want to edit, do the editing, and then use
> git-rebase to reapply the other commits on top of the changed commit.

git-rebase is one of those tools I haven't been looking at so far (I'm still rather new to Git), so I should probably read through its manpage.

> In general, there are a thousand ways to use git to rewrite history,
> either "by hand" or by using tools such as StGIT. (StGIT is what I
> personally use most of the time. I find it a good tool for producing
> readable patch series.)
I hadn't heard of StGIT, but it looks interesting.

Thanks for all the suggestions! I'll be needing some time to look them through. :)

Fredrik Tolf
Joshua N Pritikin· Jul 12, 2007, 16:09 UTC · re: Fredrik Tolf · lore

Re: Volume of commits

On Thu, Jul 12, 2007 at 04:51:44PM +0200, Fredrik Tolf wrote:
> git-rebase is one of those tools I haven't been looking at so far (I'm
> still rather new to Git), so I should probably read through its
> manpage.

I was in your shoes but I recently learned rebase. If you want to edit a commit then find the SHA-1 in git log. Then assuming you are on master:

1. git checkout -b tmp SHA-1
2. git commit --amend
3. git checkout master
4. git rebase --onto tmp SHA-1
Both times, use the same SHA-1.
Karl Hasselström· Jul 12, 2007, 16:21 UTC · re: Fredrik Tolf · lore

Re: Volume of commits

On 2007-07-12 16:51:44 +0200, Fredrik Tolf wrote:
> I see. Initially, it sounds like a lot of work, but the more I think
> about it, the more I realize that it probably isn't that bad.

You're right that it isn't that much work, but it still is work. The point is that just like investing time in making code nice and readable, investing time in making the history nice and readable is a net win. Others can follow what you do, which makes reviewing the changes much cheaper, and you can look back six months and actually understand what you were thinking back then.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Linus Torvalds· Jul 12, 2007, 16:46 UTC · re: Fredrik Tolf · lore

Re: Volume of commits

On Thu, 12 Jul 2007, Fredrik Tolf wrote:
Show 5 quoted lines
> 
> I was wondering -- whenever I see Git patches committed to projects
> like the Linux kernel or Git itself, the commits always seems to be
> committing rather large changes and be rather well-defined in terms of
> what they change.

Well, people already talked about how to do it using git, but I'd like to point out that one of the reasons you see this kind of pattern is that when you push patches around by email for commentary (which is what both git and the kernel do a _lot_), that very much inherently encourages a setup where each patch makes sense on its own, and I think it causes *much* better code cleanliness behaviour.

When people make changes that they know will be shown as patches, they just tend to make more sure that the changes make logical sense. Part of it is cleanups after-the-fact, but part of it is that when you get used to doing it, after a while you start _thinking_ in those terms when you make the changes, and that's also a good thing!

When I do any bigger changes, I usually tend to commit at points where it starts working, but then before I actually would send it to Junio, I'd go back and clean up the series (by creating a new branch, and cherry-picking, and doing diffs between the branch and applying the parts I want to).

But I do that only for stuff where I can't see the end result as a clean series of steps from the beginning. If I know exactly what I'm doing, I'll just do it the clean way from the get-go, and I don't need to clean up the series after-the-fact.

(Most of the time I actually try to get it right the first time. It's actually become a challenge to me to notice when some change needs a cleanup first in order to make the later changes much easier, so I really *like* trying to actually do the actual development in a logical order: first re-organize the code, and verify that the re-organized code works identically to the old one, then commit that, then start actually working on the new feature with the now cleaner code-base).

And no, I didn't start out programming that way. But when you get used to looking at changes as a nice series of independent commits in emails, you really start _working_ that way yourself. And I'm 100% convinced that it actually makes you a better programmer too.

Show 5 quoted lines
> When I develop for myself, I usually commit incrementally quite a
> bit, if for no other reason because Git won't let me switch between
> branches if I don't commit first. I usually try to keep my commits
> well-defined, but I don't manage to get anywhere close to what I see
> when I look at the history of Linux or Git.

The stuff you see in git or the kernel has mostly been discussed as emails, or at least been sent out that way (and if it didn't cause any discussion, it was probably "obviously clean and correct"). And that whole flow really *does* end up causing people to write cleaner patches.

It's absolutely worth emulating it, but in some respect, if it's just your own project, I suspect you'll just never have the incentive to have your history be quite as clean as the kernel/git development itself has.

		Linus
Jakub Narebski· Jul 13, 2007, 00:40 UTC · re: Fredrik Tolf · lore

Re: Volume of commits

Fredrik Tolf wrote:
Show 5 quoted lines
> When I develop for myself, I usually commit incrementally quite a
> bit, if for no other reason because Git won't let me switch between
> branches if I don't commit first. I usually try to keep my commits
> well-defined, but I don't manage to get anywhere close to what I see
> when I look at the history of Linux or Git.

First, if you commit only to switch branches you can always instead of adding commit on top of whis WIP commit, just --amend it.

Second, there is git-stash just created for saving state to go back to it.

Third, I guess that the neat patch series are result of reworking existing series using tools like StGIT (which I use and find very nice to work with, going and correcting back and forth between patches in series), or guilt (similar to StGIT), or git-gui, or new interactive mode of git-rebase, or git-cherry-pick...

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

← back to recent threads