threads / patch / 43966

patch, 34 partssequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Subject: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

## tl;dr

212 messages between Aug 31, 2016 and Jan 17, 2017. Diffs are folded; open one to read it.

replies: 211people: 10as markdown or json

Johannes Schindelin· Aug 31, 2016, 08:53 UTC · lore

[PATCH 00/34] Teach the sequencer to act as rebase -i's backend

This marks the count down to '3': two more patch series after this (really tiny ones) and we have a faster rebase -i.

The idea of this patch series is to teach the sequencer to understand all of the commands in `git-rebase-todo` scripts, to execute them and to behave pretty much very the same as `git rebase -i --continue` when called with the newly-introduced REPLAY_INTERACTIVE_REBASE setting.

Most of these patches should be pretty much straight-forward. When not, I tried to make a point of describing enough background in the commit message. Please feel free to point out where my explanations fall short.

Note that even after this patch series is applied, rebase -i is still unaffected. It will require the next patch series which introduces the rebase--helper that essentially implements `git rebase -i --continue` by calling the sequencer with the appropriate options.

The final patch series will move a couple of pre- and post-processing steps into the rebase--helper/sequencer (such as expanding/shrinking the SHA-1s, reordering the fixup!/squash! lines, etc). This might sound like a mere add-on, but it is essential for the speed improvements: those stupid little processing steps really dominated the execution time in my tests.

Apart from mostly cosmetic patches (and the occasional odd bug that I fixed promptly), I used these patches since mid May to perform all of my interactive rebases. In mid June, I had the idea to teach rebase -i to run *both* scripted rebase and rebase--helper and to cross-validate the results. This slowed down all my interactive rebases since, but helped me catch three rather obscure bugs (e.g. that git commit --fixup unfolds long onelines and rebase -i still finds the correct original commit).

This is all only to say that I am rather confident that the current code does the job.

As stated earlier, my plan is to integrate all of these patch series
into Git for Windows v2.10.0. And then be mostly offline during the
second half of September ;-) There are multiple reasons why I want to
pull through with this: I have been working on these patches since mid
February and it is high time that they benefit users other than me.
Also: I got a bit tired of postponing the integration for a couple of
Git for Windows versions already. Originally, I had planned it for
v2.9.0, but then I wanted to give the patches some review, regretting
that idea when it took one of the 14 patch series over a month to
stabilize.

This means that I will not perform any major refactoring at this stage, but postpone any such work, if necessary at all.

Just to reiterate why I do all this: it speeds up the interactive rebase substantially. Even with a not yet fully builtin rebase -i, but just the part after the user edited the `git-rebase-todo` script.

The performance test I introduced to demonstrate this (p3404) shows a speed-up of +380% here (i.e. roughly 5x), from ~8.8 seconds to ~1.8 seconds. This is on Windows, where the performance impact of avoiding shell scripting is most noticable.

On MacOSX and on Linux, the speed-up is less pronounced, but still noticable, at least if you trust Travis CI, which I abused to perform that test for me. Check for yourself (searching for "3404.2") here: https://travis-ci.org/git/git/builds/156295227. According to those logs, p3404 is speeded up from ~0.45 seconds to ~0.12 seconds on Linux (read: about 3.5x) and from ~1.7 seconds to ~0.5 seconds on MacOSX (read: almost 4x).

Johannes Schindelin (34):
  sequencer: support a new action: 'interactive rebase'
  sequencer (rebase -i): implement the 'noop' command
  sequencer (rebase -i): implement the 'edit' command
  sequencer (rebase -i): implement the 'exec' command
  sequencer (rebase -i): learn about the 'verbose' mode
  sequencer (rebase -i): write the 'done' file
  sequencer (rebase -i): add support for the 'fixup' and 'squash'
    commands
  sequencer (rebase -i): implement the short commands
  sequencer (rebase -i): write an author-script file
  sequencer (rebase -i): allow continuing with staged changes
  sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed
  sequencer (rebase -i): skip some revert/cherry-pick specific code path
  sequencer (rebase -i): the todo can be empty when continuing
  sequencer (rebase -i): update refs after a successful rebase
  sequencer (rebase -i): leave a patch upon error
  sequencer (rebase -i): implement the 'reword' command
  sequencer (rebase -i): allow fast-forwarding for edit/reword
  sequencer (rebase -i): refactor setting the reflog message
  sequencer (rebase -i): set the reflog message consistently
  sequencer (rebase -i): copy commit notes at end
  sequencer (rebase -i): record interrupted commits in rewritten, too
  sequencer (rebase -i): run the post-rewrite hook, if needed
  sequencer (rebase -i): respect the rebase.autostash setting
  sequencer (rebase -i): respect strategy/strategy_opts settings
  sequencer (rebase -i): allow rescheduling commands
  sequencer (rebase -i): implement the 'drop' command
  sequencer (rebase -i): differentiate between comments and 'noop'
  run_command_opt(): optionally hide stderr when the command succeeds
  sequencer (rebase -i): show only failed `git commit`'s output
  sequencer (rebase -i): show only failed cherry-picks' output
  sequencer (rebase -i): suggest --edit-todo upon unknown command
  sequencer (rebase -i): show the progress
  sequencer (rebase -i): write the progress into files
  sequencer (rebase -i): write out the final message
 run-command.c |  23 ++
 run-command.h |   1 +
 sequencer.c   | 990 +++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 sequencer.h   |   4 +-
 4 files changed, 971 insertions(+), 47 deletions(-)
Based-On: prepare-sequencer-and-wt_status at https://github.com/dscho/git
Fetch-Base-Via: git fetch https://github.com/dscho/git prepare-sequencer-and-wt_status
Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v1
Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v1
-- 
2.10.0.rc2.102.g5c102ec

base-commit: 45bfeb0c4044dfcf090ad97e839d250d62308fac
Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

This is a huge patch, and at the same time a huge step forward to execute the performance-critical parts of the interactive rebase in a builtin command.

Since 'fixup' and 'squash' are not only similar, but also need to know about each other (we want to reduce a series of fixups/squashes into a single, final commit message edit, from the user's point of view), we really have to implement them both at the same time.

Most of the actual work is done by the existing code path that already handles the "pick" and the "edit" commands; We added support for other features (e.g. to amend the commit message) in the patches leading up to this one, yet there are still quite a few bits in this patch that simply would not make sense as individual patches (such as: determining whether there was anything to "fix up" in the "todo" script, etc).

In theory, it would be possible to reuse the fast-forward code path also for the fixup and the squash code paths, but in practice this would make the code less readable. The end result cannot be fast-forwarded anyway, therefore let's just extend the cherry-picking code path for now.

Since the sequencer parses the entire `git-rebase-todo` script in one go, fixup or squash commands without a preceding pick can be reported early (in git-rebase--interactive, we could only report such errors just before executing the fixup/squash).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 242 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 231 insertions(+), 11 deletions(-)
Show changes to sequencer.c +231 −11
diff --git a/sequencer.c b/sequencer.c
index 3bff3d9..3ae5978 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -45,6 +45,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The commit message that is planned to be used for any changes that
+ * need to be committed following a user interaction.
+ */
+static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
+/*
+ * The file into which is accumulated the suggested commit message for
+ * squash/fixup commands. When the first of a series of squash/fixups
+ * is seen, the file is created and the commit message from the
+ * previous commit and from the first squash/fixup commit are written
+ * to it. The commit message for each subsequent squash/fixup commit
+ * is appended to the file as it is processed.
+ *
+ * The first line of the file is of the form
+ *     # This is a combination of $count commits.
+ * where $count is the number of commits whose messages have been
+ * written to the file so far (including the initial "pick" commit).
+ * Each time that a commit message is processed, this line is read and
+ * updated. It is deleted just before the combined commit is made.
+ */
+static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
+/*
+ * If the current series of squash/fixups has not yet included a squash
+ * command, then this file exists and holds the commit message of the
+ * original "pick" commit.  (If the series ends without a "squash"
+ * command, then this can be used as the commit message of the combined
+ * commit without opening the editor.)
+ */
+static GIT_PATH_FUNC(rebase_path_fixup_msg, "rebase-merge/message-fixup")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -670,6 +699,8 @@ enum todo_command {
 	TODO_PICK,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_FIXUP,
+	TODO_SQUASH,
 	TODO_EXEC,
 	TODO_NOOP
 };
@@ -678,6 +709,8 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"fixup",
+	"squash",
 	"exec",
 	"noop"
 };
@@ -689,16 +722,129 @@ static const char *command_to_string(const enum todo_command command)
 	die("Unknown command: %d", command);
 }
 
+static int is_fixup(enum todo_command command)
+{
+	return command == TODO_FIXUP || command == TODO_SQUASH;
+}
+
+static const char *nth_for_number(int n)
+{
+	int n1 = n % 10, n10 = n % 100;
+
+	if (n1 == 1 && n10 != 11)
+		return "st";
+	if (n1 == 2 && n10 != 12)
+		return "nd";
+	if (n1 == 3 && n10 != 13)
+		return "rd";
+	return "th";
+}
+
+static int update_squash_messages(enum todo_command command,
+		struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int count;
+	const char *message, *body;
+
+	if (file_exists(rebase_path_squash_msg())) {
+		char *p, *p2;
+
+		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
+			return error("Could not read %s",
+				rebase_path_squash_msg());
+
+		if (buf.buf[0] == '\n' || !skip_prefix(buf.buf + 1,
+				" This is a combination of ",
+				(const char **)&p))
+			return error("Unexpected 1st line of squash message:\n"
+				"\n\t%.*s",
+				(int)(strchrnul(buf.buf, '\n') - buf.buf),
+				buf.buf);
+		count = strtol(p, &p2, 10);
+
+		if (count < 1 || *p2 != ' ')
+			return error("Invalid 1st line of squash message:\n"
+				"\n\t%.*s",
+				(int)(strchrnul(buf.buf, '\n') - buf.buf),
+				buf.buf);
+
+		sprintf((char *)p, "%d", ++count);
+		if (!*p2)
+			*p2 = ' ';
+		else {
+			*(++p2) = 'c';
+			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
+		}
+	}
+	else {
+		unsigned char head[20];
+		struct commit *head_commit;
+		const char *head_message, *body;
+
+		if (get_sha1("HEAD", head))
+			return error("Need a HEAD to fixup");
+		if (!(head_commit = lookup_commit_reference(head)))
+			return error("Could not read HEAD");
+		if (!(head_message = get_commit_buffer(head_commit, NULL)))
+			return error("Could not read HEAD's commit message");
+
+		body = strstr(head_message, "\n\n");
+		if (!body)
+			body = "";
+		else
+			body = skip_blank_lines(body + 2);
+		if (write_file_gently(rebase_path_fixup_msg(), body, 0))
+			return error("Cannot write %s",
+				rebase_path_fixup_msg());
+
+		count = 2;
+		strbuf_addf(&buf, "%c This is a combination of 2 commits.\n"
+			"%c The first commit's message is:\n\n%s",
+			comment_line_char, comment_line_char, body);
+
+		unuse_commit_buffer(head_commit, head_message);
+	}
+
+	if (!(message = get_commit_buffer(commit, NULL)))
+		return error("Could not read commit message of %s",
+			oid_to_hex(&commit->object.oid));
+	body = strstr(message, "\n\n");
+	if (!body)
+		body = "";
+	else
+		body = skip_blank_lines(body + 2);
+
+	if (command == TODO_SQUASH) {
+		unlink(rebase_path_fixup_msg());
+		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
+			comment_line_char,
+			count, nth_for_number(count), body);
+	}
+	else if (command == TODO_FIXUP) {
+		strbuf_addf(&buf,
+			"\n%c The %d%s commit message will be skipped:\n\n",
+			comment_line_char, count, nth_for_number(count));
+		strbuf_add_commented_lines(&buf, body, strlen(body));
+	}
+	else
+		return error("Unknown command: %d", command);
+	unuse_commit_buffer(commit, message);
+
+	return write_message(&buf, rebase_path_squash_msg());
+}
 
 static int do_pick_commit(enum todo_command command, struct commit *commit,
-		struct replay_opts *opts)
+		struct replay_opts *opts, int final_fixup)
 {
+	int edit = opts->edit, cleanup_commit_message = 0;
+	const char *msg_file = edit ? NULL : git_path_merge_msg();
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res = 0, unborn = 0, allow;
+	int res = 0, unborn = 0, amend = 0, allow;
 
 	if (opts->no_commit) {
 		/*
@@ -744,7 +890,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
-	if (opts->allow_ff &&
+	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
 	     (!parent && unborn)))
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
@@ -808,6 +954,28 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
+	if (is_fixup(command)) {
+		if (update_squash_messages(command, commit, opts))
+			return -1;
+		amend = 1;
+		if (!final_fixup)
+			msg_file = rebase_path_squash_msg();
+		else if (file_exists(rebase_path_fixup_msg())) {
+			cleanup_commit_message = 1;
+			msg_file = rebase_path_fixup_msg();
+		}
+		else {
+			const char *dest = git_path("SQUASH_MSG");
+			unlink(dest);
+			if (copy_file(dest, rebase_path_squash_msg(), 0666))
+				return error("Could not rename %s to "
+					"%s", rebase_path_squash_msg(), dest);
+			unlink(git_path("MERGE_MSG"));
+			msg_file = dest;
+			edit = 1;
+		}
+	}
+
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res |= do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
@@ -859,9 +1027,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res |= sequencer_commit(opts->edit ?
-				NULL : git_path_merge_msg(),
-			opts, allow, opts->edit, 0, 0);
+		res |= sequencer_commit(msg_file, opts, allow, edit, amend,
+			cleanup_commit_message);
+
+	if (!res && final_fixup) {
+		unlink(rebase_path_fixup_msg());
+		unlink(rebase_path_squash_msg());
+	}
 
 leave:
 	free_message(commit, &msg);
@@ -1001,7 +1173,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf;
-	int i, res = 0;
+	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
 	for (i = 1; *p; i++) {
 		char *eol = strchrnul(p, '\n');
@@ -1011,8 +1183,15 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		if (parse_insn_line(item, p, eol)) {
 			res |= error("Invalid line %d: %.*s",
 				i, (int)(eol - p), p);
-			item->command = -1;
+			item->command = TODO_NOOP;
 		}
+		if (fixup_okay)
+			; /* do nothing */
+		else if (is_fixup(item->command))
+			return error("Cannot '%s' without a previous commit",
+				command_to_string(item->command));
+		else if (item->command != TODO_NOOP)
+			fixup_okay = 1;
 		p = *eol ? eol + 1 : eol;
 	}
 	if (!todo_list->nr)
@@ -1406,6 +1585,20 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int error_failed_squash(struct commit *commit,
+	struct replay_opts *opts, int subject_len, const char *subject)
+{
+	if (rename(rebase_path_squash_msg(), rebase_path_message()))
+		return error("Could not rename %s to %s",
+			rebase_path_squash_msg(), rebase_path_message());
+	unlink(rebase_path_fixup_msg());
+	unlink(git_path("MERGE_MSG"));
+	if (copy_file(git_path("MERGE_MSG"), rebase_path_message(), 0666))
+		return error("Could not copy %s to %s", rebase_path_message(),
+			git_path("MERGE_MSG"));
+	return error_with_patch(commit, subject, subject_len, opts, 1, 0);
+}
+
 static int do_exec(const char *command_line)
 {
 	const char *child_argv[] = { NULL, NULL };
@@ -1447,6 +1640,21 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int is_final_fixup(struct todo_list *todo_list)
+{
+	int i = todo_list->current;
+
+	if (!is_fixup(todo_list->items[i].command))
+		return 0;
+
+	while (++i < todo_list->nr)
+		if (is_fixup(todo_list->items[i].command))
+			return 0;
+		else if (todo_list->items[i].command < TODO_NOOP)
+			break;
+	return 1;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1462,9 +1670,15 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_EDIT) {
+		if (is_rebase_i(opts)) {
+			unlink(rebase_path_message());
+			unlink(rebase_path_author_script());
+			unlink(rebase_path_stopped_sha());
+			unlink(rebase_path_amend());
+		}
+		if (item->command <= TODO_SQUASH) {
 			res = do_pick_commit(item->command, item->commit,
-					opts);
+					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
@@ -1475,6 +1689,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (res && is_fixup(item->command)) {
+				if (res == 1)
+					intend_to_amend();
+				return error_failed_squash(item->commit, opts,
+					item->arg_len, item->arg);
+			}
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
@@ -1565,7 +1785,7 @@ static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	return do_pick_commit(opts->action == REPLAY_PICK ?
-		TODO_PICK : TODO_REVERT, cmit, opts);
+		TODO_PICK : TODO_REVERT, cmit, opts, 0);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
-- 
2.10.0.rc2.102.g5c102ec
Dennis Kaarsemaker· Sep 1, 2016, 08:33 UTC · re: Johannes Schindelin · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> +static int is_fixup(enum todo_command command)
> +{
> +	return command == TODO_FIXUP || command == TODO_SQUASH;
> +}

It sounds wrong to have a function named is_fixup return true when the command isn't a fixup but a squash. Maybe name it changes_previous_commit or something?

Show 12 quoted lines
> +static const char *nth_for_number(int n)
> +{
> +	int n1 = n % 10, n10 = n % 100;
> +
> +	if (n1 == 1 && n10 != 11)
> +		return "st";
> +	if (n1 == 2 && n10 != 12)
> +		return "nd";
> +	if (n1 == 3 && n10 != 13)
> +		return "rd";
> +	return "th";
> +}
>8---
Show 12 quoted lines
> +	if (command == TODO_SQUASH) {
> +		unlink(rebase_path_fixup_msg());
> +		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
> +			comment_line_char,
> +			count, nth_for_number(count), body);
> +	}
> +	else if (command == TODO_FIXUP) {
> +		strbuf_addf(&buf,
> +			"\n%c The %d%s commit message will be skipped:\n\n",
> +			comment_line_char, count, nth_for_number(count));
> +		strbuf_add_commented_lines(&buf, body, strlen(body));
> +	}

This way of handling numbers is not translatable, and I really think we should mark these strings for translation, like they are in the .sh version.

D.
Johannes Schindelin· Sep 1, 2016, 15:17 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Hi Dennis,
On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
Show 10 quoted lines
> On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> 
> > +static int is_fixup(enum todo_command command)
> > +{
> > +	return command == TODO_FIXUP || command == TODO_SQUASH;
> > +}
> 
> It sounds wrong to have a function named is_fixup return true when the
> command isn't a fixup but a squash. Maybe name it
> changes_previous_commit or something?

I can see how that may sound confusing, unless you understand that a squash is a fixup that lets the user edit the commit message, too. So essentially squash = fixup + edit, if you will.

Maybe the name is more appropriate in that light?
Show 31 quoted lines
> > +static const char *nth_for_number(int n)
> > +{
> > +	int n1 = n % 10, n10 = n % 100;
> > +
> > +	if (n1 == 1 && n10 != 11)
> > +		return "st";
> > +	if (n1 == 2 && n10 != 12)
> > +		return "nd";
> > +	if (n1 == 3 && n10 != 13)
> > +		return "rd";
> > +	return "th";
> > +}
> 
> >8---
> 
> > +	if (command == TODO_SQUASH) {
> > +		unlink(rebase_path_fixup_msg());
> > +		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
> > +			comment_line_char,
> > +			count, nth_for_number(count), body);
> > +	}
> > +	else if (command == TODO_FIXUP) {
> > +		strbuf_addf(&buf,
> > +			"\n%c The %d%s commit message will be skipped:\n\n",
> > +			comment_line_char, count, nth_for_number(count));
> > +		strbuf_add_commented_lines(&buf, body, strlen(body));
> > +	}
> 
> This way of handling numbers is not translatable, and I really think we
> should mark these strings for translation, like they are in the .sh
> version.

Ah, this is the risk of working on something as big as rebase--helper. Back when I started with it, the relevant code in git-rebase--interactive read like this:

	nth_string () {
		case "$1" in
		*1[0-9]|*[04-9]) echo "$1"th;;
		*1) echo "$1"st;;
		*2) echo "$1"nd;;
		*3) echo "$1"rd;;
		esac
	}
I merely did a faithful translation of that...

Now, I see that git-rebase--interactive was switched to use eval_gettext, which in turn is handled in git-sh-i18n whose code is quite convoluted. In the absence of gettext, it uses git-sh-i18n--envsubst, which has no C API whatsoever.

And I see that the beautiful ordinal computation was given up in favor of a lousy "#1", "#2", "#3", etc (it used to be "1st", "2nd", "3rd" etc).

In any case, translation is not my main concern until v2.10.0, so I'll take care of this after that release.

Ciao, Dscho

Dennis Kaarsemaker· Sep 1, 2016, 16:19 UTC · re: Johannes Schindelin · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On do, 2016-09-01 at 17:17 +0200, Johannes Schindelin wrote:
Show 21 quoted lines
> Hi Dennis,
> 
> On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
> 
> > 
> > On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> > 
> > > 
> > > +static int is_fixup(enum todo_command command)
> > > +{
> > > +	return command == TODO_FIXUP || command == TODO_SQUASH;
> > > +}
> > It sounds wrong to have a function named is_fixup return true when
> > the
> > command isn't a fixup but a squash. Maybe name it
> > changes_previous_commit or something?
> I can see how that may sound confusing, unless you understand that a
> squash is a fixup that lets the user edit the commit message, too. So
> essentially squash = fixup + edit, if you will.
> 
> Maybe the name is more appropriate in that light?

Kinda makes sense. It's not how I use fixup/squash as a user of rebase -i though. But we can't go there, that's bikeshed country :)

Show 68 quoted lines
> > > +static const char *nth_for_number(int n)
> > > +{
> > > +	int n1 = n % 10, n10 = n % 100;
> > > +
> > > +	if (n1 == 1 && n10 != 11)
> > > +		return "st";
> > > +	if (n1 == 2 && n10 != 12)
> > > +		return "nd";
> > > +	if (n1 == 3 && n10 != 13)
> > > +		return "rd";
> > > +	return "th";
> > > +}
> > > 
> > > 8---
> > > 
> > > +	if (command == TODO_SQUASH) {
> > > +		unlink(rebase_path_fixup_msg());
> > > +		strbuf_addf(&buf, "\n%c This is the %d%s commit
> > > message:\n\n%s",
> > > +			comment_line_char,
> > > +			count, nth_for_number(count), body);
> > > +	}
> > > +	else if (command == TODO_FIXUP) {
> > > +		strbuf_addf(&buf,
> > > +			"\n%c The %d%s commit message will be
> > > skipped:\n\n",
> > > +			comment_line_char, count,
> > > nth_for_number(count));
> > > +		strbuf_add_commented_lines(&buf, body,
> > > strlen(body));
> > > +	}
> > This way of handling numbers is not translatable, and I really
> > think we
> > should mark these strings for translation, like they are in the .sh
> > version.
> Ah, this is the risk of working on something as big as rebase
> --helper.
> Back when I started with it, the relevant code in git-rebase
> --interactive
> read like this:
> 
> 	nth_string () {
> 		case "$1" in
> 		*1[0-9]|*[04-9]) echo "$1"th;;
> 		*1) echo "$1"st;;
> 		*2) echo "$1"nd;;
> 		*3) echo "$1"rd;;
> 		esac
> 	}
> 
> I merely did a faithful translation of that...
> 
> Now, I see that git-rebase--interactive was switched to use
> eval_gettext,
> which in turn is handled in git-sh-i18n whose code is quite
> convoluted. In
> the absence of gettext, it uses git-sh-i18n--envsubst, which has no C
> API
> whatsoever.
> 
> And I see that the beautiful ordinal computation was given up in
> favor of
> a lousy "#1", "#2", "#3", etc (it used to be "1st", "2nd", "3rd"
> etc).
> 
> In any case, translation is not my main concern until v2.10.0, so
> I'll
> take care of this after that release.

Hmm, not sure if I agree with that. I'd see it as a regression to lose the i18n there.

D.
Johannes Schindelin· Sep 2, 2016, 07:13 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Hi Dennis,
On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
Show 11 quoted lines
> On do, 2016-09-01 at 17:17 +0200, Johannes Schindelin wrote:
>
> > And I see that the beautiful ordinal computation was given up in favor
> > of a lousy "#1", "#2", "#3", etc (it used to be "1st", "2nd", "3rd"
> > etc).
> > 
> > In any case, translation is not my main concern until v2.10.0, so I'll
> > take care of this after that release.
> 
> Hmm, not sure if I agree with that. I'd see it as a regression to lose
> the i18n there.

As Git for Windows does not ship with translations (for multiple reasons), it would not be a regression.

Having said that, I see that having a different text than the current rebase -i can be seen as a regression, so I changed that.

Thanks for the review! Dscho

Dennis Kaarsemaker· Sep 2, 2016, 12:06 UTC · re: Johannes Schindelin · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On vr, 2016-09-02 at 09:13 +0200, Johannes Schindelin wrote:
> As Git for Windows does not ship with translations (for multiple
> reasons), it would not be a regression.

I'm confused, how does "git for windows does not ship with translations" translate to "this is not a regression"? Is this patch series only meant to be for git for windows and not go into git.git itself?

D.
Johannes Schindelin· Sep 2, 2016, 14:22 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Hi Dennis,
On Fri, 2 Sep 2016, Dennis Kaarsemaker wrote:
Show 9 quoted lines
> On vr, 2016-09-02 at 09:13 +0200, Johannes Schindelin wrote:
> 
> > As Git for Windows does not ship with translations (for multiple
> > reasons), it would not be a regression.
> 
> I'm confused, how does "git for windows does not ship with
> translations" translate to "this is not a regression"? Is this patch
> series only meant to be for git for windows and not go into git.git
> itself?
Oh, I thought I had clarified my plan... The timeline is:
- I submit the remaining rebase--helper patch series for review (last week
  and this one),
- I publish a preview of Git for Windows v2.10.0 that already uses these
  patches (done: https://github.com/git-for-windows/git/releases/tag/v2.9.3.windows.3)
- once upstream Git v2.10.0 is released (possibly today, after my work
  hours), I perform a final "Git garden shears" run (read: rebase Git for
  Windows' patches, retaining the branch structure) on top of v2.10.0 and
  release Git for Windows v2.10.0, tagged as v2.10.0.windows.1 in
  https://github.com/git-for-windows/git (due to time zone differences
  relative to Junio, the most likely time for this release would be
  some time around noon tomorrow, given that the release engineering takes
  roughly 2-4 hours, running unsupervised for the most part).
- as far as Git for Windows is concerned, l10n is not really an issue yet:
  the installer is released without any localizations.
- After releasing Git for Windows v2.10.0, I will pay a lot of attention
  to feedback. Not only to hear a lot of praise, but also to catch any
  possible regressions. Not that I expect anything dramatic to happen
  because I really tested this as thoroughly as I can: not a single one of
  my interactive rebases since mid May has been performed without
  involving the rebase--helper. In the three cases where I *did* find a
  regression, I solved it immediately, of course.
- After releasing Git for Windows v2.10.0, I will have a nice beer. Or
  three.
- Then I will leisurely try to address the l10n issues.
- Then, I will send out the current iterations of the patch series that
  are in flight.
- I have the entire week to address concerns with Git for Windows as well
  as with the patch series (where the former takes precedence, of course).
- The second half of September, I will relax from this marathon that
  started in early February. Meaning: I will be mostly offline.

I hope this clarifies why I am not so concerned about some issues such as translation, or commit messages, or grammar, and more so about others, such as incorrect code.

Ciao, Dscho

Dennis Kaarsemaker· Sep 2, 2016, 17:04 UTC · re: Johannes Schindelin · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On vr, 2016-09-02 at 16:22 +0200, Johannes Schindelin wrote:
> I hope this clarifies why I am not so concerned about some issues
> such as translation, or commit messages, or grammar, and more so
> about others, such as incorrect code.
It does, thanks!
D.
Junio C Hamano· Sep 1, 2016, 18:15 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Dennis Kaarsemaker <dennis@kaarsemaker.net> writes:
Show 31 quoted lines
>> +static const char *nth_for_number(int n)
>> +{
>> +	int n1 = n % 10, n10 = n % 100;
>> +
>> +	if (n1 == 1 && n10 != 11)
>> +		return "st";
>> +	if (n1 == 2 && n10 != 12)
>> +		return "nd";
>> +	if (n1 == 3 && n10 != 13)
>> +		return "rd";
>> +	return "th";
>> +}
>
>>8---
>
>> +	if (command == TODO_SQUASH) {
>> +		unlink(rebase_path_fixup_msg());
>> +		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
>> +			comment_line_char,
>> +			count, nth_for_number(count), body);
>> +	}
>> +	else if (command == TODO_FIXUP) {
>> +		strbuf_addf(&buf,
>> +			"\n%c The %d%s commit message will be skipped:\n\n",
>> +			comment_line_char, count, nth_for_number(count));
>> +		strbuf_add_commented_lines(&buf, body, strlen(body));
>> +	}
>
> This way of handling numbers is not translatable, and I really think we
> should mark these strings for translation, like they are in the .sh
> version.
Correct.

For those who were not paying attention on the 'master' front during this pre-release period [*1*], I have to point out that the scripted Porcelain has been updated to lose the Anglo-centric st/nd/rd/th and this series would want to get updated to match.

[Footnote]
*1* Why weren't you?  Repent! ;-)
Johannes Schindelin· Sep 2, 2016, 07:15 UTC · re: Junio C Hamano · lore

Re: [PATCH 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Hi Junio,
On Thu, 1 Sep 2016, Junio C Hamano wrote:
Show 9 quoted lines
> For those who were not paying attention on the 'master' front during
> this pre-release period [*1*], I have to point out that the scripted
> Porcelain has been updated to lose the Anglo-centric st/nd/rd/th and
> this series would want to get updated to match.
> 
> 
> [Footnote]
> 
> *1* Why weren't you?  Repent! ;-)
I tried to. But, you know, I was kinda busy with a couple of patch series.

In any case, I changed the code this morning. Can't say that I like those forced last-minute changes.

Ciao, Johannes

Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 05/34] sequencer (rebase -i): learn about the 'verbose' mode

When calling `git rebase -i -v`, the user wants to see some statistics after the commits were rebased. Let's show some.

The strbuf we use to perform that task will be used for other things in subsequent commits, hence it is declared and initialized in a wider scope than strictly needed here.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 ++++++++++++++++++++++
 sequencer.h |  1 +
 2 files changed, 23 insertions(+)
Show changes to 2 files +23 −0

sequencer.c, sequencer.h

diff --git a/sequencer.c b/sequencer.c
index a58bb91..95c31bb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -63,6 +63,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  * command-line (and are only consumed, not modified, by the sequencer).
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
+static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
+static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -1096,6 +1098,9 @@ static int read_populate_opts(struct replay_opts *opts)
 			}
 		}
 
+		if (file_exists(rebase_path_verbose()))
+			opts->verbose = 1;
+
 		return 0;
 	}
 
@@ -1468,9 +1473,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
+
+		if (opts->verbose) {
+			const char *argv[] = {
+				"diff-tree", "--stat", NULL, NULL
+			};
+
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
+				return error("Could not read %s",
+					rebase_path_orig_head());
+			strbuf_addstr(&buf, "..HEAD");
+			argv[2] = buf.buf;
+			run_command_v_opt(argv, RUN_GIT_CMD);
+			strbuf_reset(&buf);
+		}
+		strbuf_release(&buf);
 	}
 
 	/*
diff --git a/sequencer.h b/sequencer.h
index edd7d4a..fd2a719 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -24,6 +24,7 @@ struct replay_opts {
 	int allow_empty;
 	int allow_empty_message;
 	int keep_redundant_commits;
+	int verbose;
 
 	int mainline;
 
-- 
2.10.0.rc2.102.g5c102ec
Dennis Kaarsemaker· Aug 31, 2016, 18:39 UTC · re: Johannes Schindelin · lore

Re: [PATCH 05/34] sequencer (rebase -i): learn about the 'verbose' mode

On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> +               if (file_exists(rebase_path_verbose()))
> +                       opts->verbose = 1;

I don't see anything in this series that creates this file, will that be part of a later series?

D.
Johannes Schindelin· Aug 31, 2016, 20:36 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 05/34] sequencer (rebase -i): learn about the 'verbose' mode

Hi Dennis,
On Wed, 31 Aug 2016, Dennis Kaarsemaker wrote:
Show 7 quoted lines
> On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> 
> > +               if (file_exists(rebase_path_verbose()))
> > +                       opts->verbose = 1;
> 
> I don't see anything in this series that creates this file, will that
> be part of a later series?

No. The sequencer does not write that file, but just consumes it. `git rebase` writes it.

Ciao, Dscho

Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 04/34] sequencer (rebase -i): implement the 'exec' command

The 'exec' command is a little special among rebase -i's commands, as it does *not* have a SHA-1 as first parameter. Instead, everything after the `exec` command is treated as command-line to execute.

Let's reuse the arg/arg_len fields of the todo_item structure (which hold the oneline for pick/edit commands) to point to the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)
Show changes to sequencer.c +58 −0
diff --git a/sequencer.c b/sequencer.c
index fe2f6e7..a58bb91 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -17,6 +17,7 @@
 #include "argv-array.h"
 #include "quote.h"
 #include "log-tree.h"
+#include "wt-status.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -661,6 +662,7 @@ enum todo_command {
 	TODO_PICK,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_EXEC,
 	TODO_NOOP
 };
 
@@ -668,6 +670,7 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"exec",
 	"noop"
 };
 
@@ -964,6 +967,12 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return -1;
 	bol += padding;
 
+	if (item->command == TODO_EXEC) {
+		item->arg = bol;
+		item->arg_len = (int)(eol - bol);
+		return 0;
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -1372,6 +1381,47 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int do_exec(const char *command_line)
+{
+	const char *child_argv[] = { NULL, NULL };
+	int dirty, status;
+
+	fprintf(stderr, "Executing: %s\n", command_line);
+	child_argv[0] = command_line;
+	status = run_command_v_opt(child_argv, RUN_USING_SHELL);
+
+	/* force re-reading of the cache */
+	if (discard_cache() < 0 || read_cache() < 0)
+		return error(_("Could not read index"));
+
+	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
+
+	if (status) {
+		warning("Execution failed: %s\n%s"
+			"You can fix the problem, and then run\n"
+			"\n"
+			"  git rebase --continue\n"
+			"\n",
+			command_line,
+			dirty ? "and made changes to the index and/or the "
+				"working tree\n" : "");
+		if (status == 127)
+			/* command not found */
+			status = 1;
+	}
+	else if (dirty) {
+		warning("Execution succeeded: %s\nbut "
+			"left changes to the index and/or the working tree\n"
+			"Commit or stash your changes, and then run\n"
+			"\n"
+			"  git rebase --continue\n"
+			"\n", command_line);
+		status = 1;
+	}
+
+	return status;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1401,6 +1451,14 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					!res);
 			}
 		}
+		else if (item->command == TODO_EXEC) {
+			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			int saved = *end_of_arg;
+
+			*end_of_arg = '\0';
+			res = do_exec(item->arg);
+			*end_of_arg = saved;
+		}
 		else if (item->command != TODO_NOOP)
 			return error("Unknown command %d", item->command);
 
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 03/34] sequencer (rebase -i): implement the 'edit' command

This patch is a straight-forward reimplementation of the `edit` operation of the interactive rebase command.

Well, not *quite* straight-forward: when stopping, the `edit` command wants to write the `patch` file (which is not only the patch, but includes the commit message and author information). To that end, this patch requires the earlier work that taught the log-tree machinery to respect the `file` setting of rev_info->diffopt to write to a file stream different than stdout.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 112 insertions(+), 2 deletions(-)
Show changes to sequencer.c +112 −2
diff --git a/sequencer.c b/sequencer.c
index f85dc9c..fe2f6e7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -16,6 +16,7 @@
 #include "refs.h"
 #include "argv-array.h"
 #include "quote.h"
+#include "log-tree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -43,6 +44,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
 /*
+ * When an "edit" rebase command is being processed, the SHA1 of the
+ * commit to be edited is recorded in this file.  When "git rebase
+ * --continue" is executed, if there are any staged changes then they
+ * will be amended to the HEAD commit, but only provided the HEAD
+ * commit is still the commit to be edited.  When any other rebase
+ * command is processed, this file is deleted.
+ */
+static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
+/*
+ * When we stop at a given patch via the "edit" command, this file contains
+ * the long commit name of the corresponding patch.
+ */
+static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -645,12 +660,14 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 enum todo_command {
 	TODO_PICK,
 	TODO_REVERT,
+	TODO_EDIT,
 	TODO_NOOP
 };
 
 static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
+	"edit",
 	"noop"
 };
 
@@ -1279,9 +1296,85 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
+static int make_patch(struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct rev_info log_tree_opt;
+	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject;
+	int res = 0;
+
+	if (write_file_gently(rebase_path_stopped_sha(),
+			      short_commit_name(commit), 1) < 0)
+		return -1;
+
+	strbuf_addf(&buf, "%s/patch", get_dir(opts));
+	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
+	init_revisions(&log_tree_opt, NULL);
+	log_tree_opt.abbrev = 0;
+	log_tree_opt.diff = 1;
+	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
+	log_tree_opt.disable_stdin = 1;
+	log_tree_opt.no_commit_id = 1;
+	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
+	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
+	if (!log_tree_opt.diffopt.file)
+		res |= error_errno("could not open '%s'", buf.buf);
+	else {
+		res |= log_tree_commit(&log_tree_opt, commit);
+		fclose(log_tree_opt.diffopt.file);
+	}
+	strbuf_reset(&buf);
+
+	strbuf_addf(&buf, "%s/message", get_dir(opts));
+	if (!file_exists(buf.buf)) {
+		find_commit_subject(commit_buffer, &subject);
+		res |= write_file_gently(buf.buf, subject, 1);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
+	strbuf_release(&buf);
+
+	return res;
+}
+
+static int intend_to_amend(void)
+{
+	unsigned char head[20];
+
+	if (get_sha1("HEAD", head))
+		return error("Cannot read HEAD");
+
+	return write_file_gently(rebase_path_amend(), sha1_to_hex(head), 1);
+}
+
+static int error_with_patch(struct commit *commit,
+	const char *subject, int subject_len,
+	struct replay_opts *opts, int exit_code, int to_amend)
+{
+	if (make_patch(commit, opts))
+		return -1;
+
+	if (to_amend) {
+		if (intend_to_amend())
+			return -1;
+
+		fprintf(stderr, "You can amend the commit now, with\n"
+			"\n"
+			"  git commit --amend %s\n"
+			"\n"
+			"Once you are satisfied with your changes, run\n"
+			"\n"
+			"  git rebase --continue\n", gpg_sign_opt_quoted(opts));
+	}
+	else if (exit_code)
+		fprintf(stderr, "Could not apply %s... %.*s\n",
+			short_commit_name(commit), subject_len, subject);
+
+	return exit_code;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	int res;
+	int res = 0;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	if (opts->allow_ff)
@@ -1294,9 +1387,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_REVERT)
+		if (item->command <= TODO_EDIT) {
 			res = do_pick_commit(item->command, item->commit,
 					opts);
+			if (item->command == TODO_EDIT) {
+				struct commit *commit = item->commit;
+				if (!res)
+					warning("Stopped at %s... %.*s",
+						short_commit_name(commit),
+						item->arg_len, item->arg);
+				return error_with_patch(commit,
+					item->arg, item->arg_len, opts, res,
+					!res);
+			}
+		}
 		else if (item->command != TODO_NOOP)
 			return error("Unknown command %d", item->command);
 
@@ -1305,6 +1409,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			return res;
 	}
 
+	if (is_rebase_i(opts)) {
+		/* Stopped in the middle, as planned? */
+		if (todo_list->current < todo_list->nr)
+			return 0;
+	}
+
 	/*
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 02/34] sequencer (rebase -i): implement the 'noop' command

The 'noop' command is probably the most boring of all rebase -i commands to support in the sequencer.

Which makes it an excellent candidate for this first stab to add support for rebase -i's commands to the sequencer.

For the moment, let's also treat empty lines and commented-out lines as 'noop'; We will refine that handling later in this patch series.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)
Show changes to sequencer.c +25 −3
diff --git a/sequencer.c b/sequencer.c
index 5598534..f85dc9c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -644,12 +644,14 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 
 enum todo_command {
 	TODO_PICK,
-	TODO_REVERT
+	TODO_REVERT,
+	TODO_NOOP
 };
 
 static const char *todo_command_strings[] = {
 	"pick",
-	"revert"
+	"revert",
+	"noop"
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -916,6 +918,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
+	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
+		item->command = TODO_NOOP;
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
@@ -924,6 +934,13 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	if (i >= ARRAY_SIZE(todo_command_strings))
 		return -1;
 
+	if (item->command == TODO_NOOP) {
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
@@ -1277,7 +1294,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(item->command, item->commit, opts);
+		if (item->command <= TODO_REVERT)
+			res = do_pick_commit(item->command, item->commit,
+					opts);
+		else if (item->command != TODO_NOOP)
+			return error("Unknown command %d", item->command);
+
 		todo_list->current++;
 		if (res)
 			return res;
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 01/34] sequencer: support a new action: 'interactive rebase'

This patch introduces a new action for the sequencer. It really does not do a whole lot of its own right now, but lays the ground work for patches to come. The intention, of course, is to finally make the sequencer the work horse of the interactive rebase (the original idea behind the "sequencer" concept).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 +++++++++++++++++++++++++++++---
 sequencer.h |  3 ++-
 2 files changed, 31 insertions(+), 4 deletions(-)
Show changes to 2 files +31 −4

sequencer.c, sequencer.h

diff --git a/sequencer.c b/sequencer.c
index a89ef27..5598534 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -28,6 +28,14 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static GIT_PATH_FUNC(rebase_path, "rebase-merge")
+/*
+ * The file containing rebase commands, comments, and empty lines.
+ * This file is created by "git rebase -i" then edited by the user. As
+ * the lines are processed, they are removed from the front of this
+ * file and written to the tail of 'done'.
+ */
+static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
@@ -43,16 +51,20 @@ static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
-	return 0;
+	return opts->action == REPLAY_INTERACTIVE_REBASE;
 }
 
 static const char *get_dir(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path();
 	return git_path_seq_dir();
 }
 
 static const char *get_todo_path(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path_todo();
 	return git_path_todo_file();
 }
 
@@ -176,7 +188,15 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? "revert" : "cherry-pick";
+	switch (opts->action) {
+	case REPLAY_REVERT:
+		return "revert";
+	case REPLAY_PICK:
+		return "cherry-pick";
+	case REPLAY_INTERACTIVE_REBASE:
+		return "rebase -i";
+	}
+	die("Unknown action: %d", opts->action);
 }
 
 struct commit_message {
@@ -407,7 +427,10 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
-		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
+		/*
+		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
+		 * "rebase -i".
+		 */
 		return error(_("%s: Unable to write new index file"),
 			action_name(opts));
 	rollback_lock_file(&index_lock);
@@ -1186,6 +1209,9 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	const char *todo_path = get_todo_path(opts);
 	int next = todo_list->current, offset, fd;
 
+	if (is_rebase_i(opts))
+		next++;
+
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("Could not lock '%s'"),
diff --git a/sequencer.h b/sequencer.h
index 688fff1..edd7d4a 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -7,7 +7,8 @@ const char *git_path_seq_dir(void);
 
 enum replay_action {
 	REPLAY_REVERT,
-	REPLAY_PICK
+	REPLAY_PICK,
+	REPLAY_INTERACTIVE_REBASE
 };
 
 struct replay_opts {
-- 
2.10.0.rc2.102.g5c102ec
Kevin Daudt· Sep 2, 2016, 21:13 UTC · re: Johannes Schindelin · lore

Re: [PATCH 01/34] sequencer: support a new action: 'interactive rebase'

On Wed, Aug 31, 2016 at 10:54:02AM +0200, Johannes Schindelin wrote:
Show 21 quoted lines
> @@ -43,16 +51,20 @@ static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
>  /* We will introduce the 'interactive rebase' mode later */
>  static inline int is_rebase_i(const struct replay_opts *opts)
>  {
> -	return 0;
> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
>  }
>  
>  static const char *get_dir(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path();
>  	return git_path_seq_dir();
>  }
>  
>  static const char *get_todo_path(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path_todo();
>  	return git_path_todo_file();
>  }

This patch fails to apply for me because function is_rebase_i has never been introduced before (no record of it anywhere). Currently, only IS_REBASE_I macro is present.

Johannes Schindelin· Sep 3, 2016, 06:57 UTC · re: Kevin Daudt · lore

Re: [PATCH 01/34] sequencer: support a new action: 'interactive rebase'

Hi Kevin,
On Fri, 2 Sep 2016, Kevin Daudt wrote:
Show 26 quoted lines
> On Wed, Aug 31, 2016 at 10:54:02AM +0200, Johannes Schindelin wrote:
> > @@ -43,16 +51,20 @@ static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
> >  /* We will introduce the 'interactive rebase' mode later */
> >  static inline int is_rebase_i(const struct replay_opts *opts)
> >  {
> > -	return 0;
> > +	return opts->action == REPLAY_INTERACTIVE_REBASE;
> >  }
> >  
> >  static const char *get_dir(const struct replay_opts *opts)
> >  {
> > +	if (is_rebase_i(opts))
> > +		return rebase_path();
> >  	return git_path_seq_dir();
> >  }
> >  
> >  static const char *get_todo_path(const struct replay_opts *opts)
> >  {
> > +	if (is_rebase_i(opts))
> > +		return rebase_path_todo();
> >  	return git_path_todo_file();
> >  }
> 
> This patch fails to apply for me because function is_rebase_i has never
> been introduced before (no record of it anywhere). Currently, only
> IS_REBASE_I macro is present.

I did not send out a new iteration of the prepare-sequencer patch series (mostly because I wanted reviewers to look at the later patch series, rather than re-review a 2nd iteration of something they already saw).

But I did address the concerns mentioned in the review already, of course, because part of the reason to show those patch series was to get valuable feedback before including the work in Git for Windows v2.10.0.

You can find the current iteration of the prepare-sequencer here: https://github.com/dscho/git/compare/libify-sequencer...prepare-sequencer

Please note that I will most likely try to address some l10n concerns before sending out the next iteration of the patch series.

The patch introducing is_rebase_i() (and no longer IS_REBASE_I()) is: https://github.com/dscho/git/commit/76d272020bb72618957308f06083c807efe59aca

If you want to have the latest iteration of the entire patch thicket, just `git fetch https://github.com/dscho/git interactive-rebase`. I update that more frequently than I send out updates via mail.

Ciao, Johannes

Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 08/34] sequencer (rebase -i): implement the short commands

For users' convenience, most rebase commands can be abbreviated, e.g. 'p' instead of 'pick' and 'x' instead of 'exec'. Let's teach the sequencer to handle those abbreviated commands just fine.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)
Show changes to sequencer.c +21 −13
diff --git a/sequencer.c b/sequencer.c
index 3ae5978..919698b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -705,20 +705,23 @@ enum todo_command {
 	TODO_NOOP
 };
 
-static const char *todo_command_strings[] = {
-	"pick",
-	"revert",
-	"edit",
-	"fixup",
-	"squash",
-	"exec",
-	"noop"
+static struct {
+	char c;
+	const char *str;
+} todo_command_info[] = {
+	{ 'p', "pick" },
+	{ 0,   "revert" },
+	{ 'e', "edit" },
+	{ 'f', "fixup" },
+	{ 's', "squash" },
+	{ 'x', "exec" },
+	{ 0,   "noop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if (command < ARRAY_SIZE(todo_command_strings))
-		return todo_command_strings[command];
+	if (command < ARRAY_SIZE(todo_command_info))
+		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
 
@@ -1126,12 +1129,17 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
-		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_strings))
+		else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
+			bol++;
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_info))
 		return -1;
 
 	if (item->command == TODO_NOOP) {
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 09/34] sequencer (rebase -i): write an author-script file

When the interactive rebase aborts, it writes out an author-script file to record the author information for the current commit. As we are about to teach the sequencer how to perform the actions behind an interactive rebase, it needs to write those author-script files, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 48 insertions(+), 1 deletion(-)
Show changes to sequencer.c +48 −1
diff --git a/sequencer.c b/sequencer.c
index 919698b..ef818a2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -528,6 +528,51 @@ static int is_index_unchanged(void)
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
 
+static int write_author_script(const char *message)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *eol;
+
+	for (;;)
+		if (!*message || starts_with(message, "\n")) {
+missing_author:
+			/* Missing 'author' line? */
+			unlink(rebase_path_author_script());
+			return 0;
+		}
+		else if (skip_prefix(message, "author ", &message))
+			break;
+		else if ((eol = strchr(message, '\n')))
+			message = eol + 1;
+		else
+			goto missing_author;
+
+	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, " <", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, "> ", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_DATE='@");
+	while (*message && *message != '\n' && *message != '\r')
+		if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\n");
+	return write_message(&buf, rebase_path_author_script());
+}
+
 static char **read_author_script(void)
 {
 	struct strbuf script = STRBUF_INIT;
@@ -979,7 +1024,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
+	if (is_rebase_i(opts) && write_author_script(msg.message) < 0)
+		res |= -1;
+	else if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res |= do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 12/34] sequencer (rebase -i): skip some revert/cherry-pick specific code path

When a cherry-pick continues without a "todo script", the intention is simply to pick a single commit.

However, when an interactive rebase is continued without a "todo script", it means that the last command has been completed and that we now need to clean up.

This commit guards the revert/cherry-pick specific steps so that they are not executed in rebase -i mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 23 +++++++++++++----------
 1 file changed, 13 insertions(+), 10 deletions(-)
Show changes to sequencer.c +13 −10
diff --git a/sequencer.c b/sequencer.c
index 683b89f..7f017a8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1859,22 +1859,25 @@ int sequencer_continue(struct replay_opts *opts)
 		if (commit_staged_changes(opts))
 			return -1;
 	}
-	if (!file_exists(get_todo_path(opts)))
+	else if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
 		return -1;
 
-	/* Verify that the conflict has been resolved */
-	if (file_exists(git_path_cherry_pick_head()) ||
-	    file_exists(git_path_revert_head())) {
-		int ret = continue_single_pick();
-		if (ret)
-			return ret;
+	if (!is_rebase_i(opts)) {
+		/* Verify that the conflict has been resolved */
+		if (file_exists(git_path_cherry_pick_head()) ||
+		    file_exists(git_path_revert_head())) {
+			int ret = continue_single_pick();
+			if (ret)
+				return ret;
+		}
+		if (index_differs_from("HEAD", 0))
+			return error_dirty_index(opts);
+		todo_list.current++;
 	}
-	if (index_differs_from("HEAD", 0))
-		return error_dirty_index(opts);
-	todo_list.current++;
+
 	res = pick_commits(&todo_list, opts);
 	todo_list_release(&todo_list);
 	return res;
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 11/34] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

The scripted version of the interactive rebase already does that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
Show changes to sequencer.c +6 −1
diff --git a/sequencer.c b/sequencer.c
index 4ddb12e..683b89f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1813,8 +1813,13 @@ static int commit_staged_changes(struct replay_opts *opts)
 
 	if (has_unstaged_changes(1))
 		return error(_("Cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0))
+	if (!has_uncommitted_changes(0)) {
+		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error("Could not remove CHERRY_PICK_HEAD");
 		return 0;
+	}
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 13/34] sequencer (rebase -i): the todo can be empty when continuing

When the last command of an interactive rebase fails, the user needs to resolve the problem and then continue the interactive rebase. Naturally, the todo script is empty by then. So let's not complain about that!

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
Show changes to sequencer.c +3 −2
diff --git a/sequencer.c b/sequencer.c
index 7f017a8..d4437f5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1249,8 +1249,6 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 			fixup_okay = 1;
 		p = *eol ? eol + 1 : eol;
 	}
-	if (!todo_list->nr)
-		return error(_("No commits parsed."));
 	return res;
 }
 
@@ -1273,6 +1271,9 @@ static int read_populate_todo(struct todo_list *todo_list,
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
 		return error(_("Unusable instruction sheet: %s"), todo_file);
+	if (!todo_list->nr &&
+	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
+		return error(_("No commits parsed."));
 	return 0;
 }
 
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 10/34] sequencer (rebase -i): allow continuing with staged changes

When an interactive rebase is interrupted, the user may stage changes before continuing, and we need to commit those changes in that case.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
Show changes to sequencer.c +39 −0
diff --git a/sequencer.c b/sequencer.c
index ef818a2..4ddb12e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1807,6 +1807,41 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
+static int commit_staged_changes(struct replay_opts *opts)
+{
+	int amend = 0;
+
+	if (has_unstaged_changes(1))
+		return error(_("Cannot rebase: You have unstaged changes."));
+	if (!has_uncommitted_changes(0))
+		return 0;
+
+	if (file_exists(rebase_path_amend())) {
+		struct strbuf rev = STRBUF_INIT;
+		unsigned char head[20], to_amend[20];
+
+		if (get_sha1("HEAD", head))
+			return error("Cannot amend non-existing commit");
+		if (!read_oneliner(&rev, rebase_path_amend(), 0))
+			return error("Invalid file: %s", rebase_path_amend());
+		if (get_sha1_hex(rev.buf, to_amend))
+			return error("Invalid contents: %s",
+				rebase_path_amend());
+		if (hashcmp(head, to_amend))
+			return error("\nYou have uncommitted changes in your "
+				"working tree. Please, commit them\nfirst and "
+				"then run 'git rebase --continue' again.");
+
+		strbuf_release(&rev);
+		amend = 1;
+	}
+
+	if (sequencer_commit(rebase_path_message(), opts, 1, 1, amend, 0))
+		return error("Could not commit staged changes.");
+	unlink(rebase_path_amend());
+	return 0;
+}
+
 int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
@@ -1815,6 +1850,10 @@ int sequencer_continue(struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
+	if (is_rebase_i(opts)) {
+		if (commit_staged_changes(opts))
+			return -1;
+	}
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 16/34] sequencer (rebase -i): implement the 'reword' command

This is now trivial, as all the building blocks are in place: all we need to do is to flip the "edit" switch when committing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
Show changes to sequencer.c +7 −2
diff --git a/sequencer.c b/sequencer.c
index 9a06b40..64fec47 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -746,6 +746,7 @@ enum todo_command {
 	TODO_PICK,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_REWORD,
 	TODO_FIXUP,
 	TODO_SQUASH,
 	TODO_EXEC,
@@ -759,6 +760,7 @@ static struct {
 	{ 'p', "pick" },
 	{ 0,   "revert" },
 	{ 'e', "edit" },
+	{ 'r', "reword" },
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
@@ -1004,7 +1006,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (is_fixup(command)) {
+	if (command == TODO_REWORD)
+		edit = 1;
+	else if (is_fixup(command)) {
 		if (update_squash_messages(command, commit, opts))
 			return -1;
 		amend = 1;
@@ -1755,7 +1759,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			}
 			else if (res && is_rebase_i(opts))
 				return res | error_with_patch(item->commit,
-					item->arg, item->arg_len, opts, res, 0);
+					item->arg, item->arg_len, opts, res,
+					item->command == TODO_REWORD);
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 19/34] sequencer (rebase -i): set the reflog message consistently

We already used the same reflog message as the scripted version of rebase -i when finishing. With this commit, we do that also for all the commands before that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)
Show changes to sequencer.c +4 −0
diff --git a/sequencer.c b/sequencer.c
index 71730e6..54f654c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1768,6 +1768,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			unlink(rebase_path_amend());
 		}
 		if (item->command <= TODO_SQUASH) {
+			if (is_rebase_i(opts))
+				setenv("GIT_REFLOG_ACTION", reflog_message(opts,
+					command_to_string(item->command), NULL),
+					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 18/34] sequencer (rebase -i): refactor setting the reflog message

This makes the code DRYer, with the obvious benefit that we can enhance the code further in a single place.

We can also reuse the functionality elsewhere by calling this new function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)
Show changes to sequencer.c +26 −7
diff --git a/sequencer.c b/sequencer.c
index 399fc8d..71730e6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1726,6 +1726,26 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...)
+{
+	va_list ap;
+	static struct strbuf buf = STRBUF_INIT;
+
+	va_start(ap, fmt);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, action_name(opts));
+	if (sub_action)
+		strbuf_addf(&buf, " (%s)", sub_action);
+	if (fmt) {
+		strbuf_addstr(&buf, ": ");
+		strbuf_vaddf(&buf, fmt, ap);
+	}
+	va_end(ap);
+
+	return buf.buf;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1796,6 +1816,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
 				starts_with(head_ref.buf, "refs/")) {
+			const char *msg;
 			unsigned char head[20], orig[20];
 
 			if (get_sha1("HEAD", head))
@@ -1803,19 +1824,17 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
 					get_sha1_hex(buf.buf, orig))
 				return error("Could not read orig-head");
-			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
-				head_ref.buf);
 			if (!read_oneliner(&buf, rebase_path_onto(), 0))
 				return error("Could not read 'onto'");
-			if (update_ref(buf.buf, head_ref.buf, head, orig,
+			msg = reflog_message(opts, "finish", "%s onto %s",
+				head_ref.buf, buf.buf);
+			if (update_ref(msg, head_ref.buf, head, orig,
 					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 				return error("Could not update %s",
 					head_ref.buf);
-			strbuf_reset(&buf);
-			strbuf_addf(&buf,
-				"rebase -i (finish): returning to %s",
+			msg = reflog_message(opts, "finish", "returning to %s",
 				head_ref.buf);
-			if (create_symref("HEAD", head_ref.buf, buf.buf))
+			if (create_symref("HEAD", head_ref.buf, msg))
 				return error("Could not update HEAD to %s",
 					head_ref.buf);
 			strbuf_reset(&buf);
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 21/34] sequencer (rebase -i): record interrupted commits in rewritten, too

When continuing after a `pick` command failed, we want that commit to show up in the rewritten-list (and its notes to be rewritten), too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
Show changes to sequencer.c +11 −0
diff --git a/sequencer.c b/sequencer.c
index a5e18f3..a00d349 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2023,6 +2023,17 @@ int sequencer_continue(struct replay_opts *opts)
 			return error_dirty_index(opts);
 		todo_list.current++;
 	}
+	else if (file_exists(rebase_path_stopped_sha())) {
+		struct strbuf buf = STRBUF_INIT;
+		struct object_id oid;
+
+		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1)) {
+			if (!get_sha1_committish(buf.buf, oid.hash))
+				record_in_rewritten(&oid,
+						peek_command(&todo_list, 0));
+		}
+		strbuf_release(&buf);
+	}
 
 	res = pick_commits(&todo_list, opts);
 	todo_list_release(&todo_list);
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 20/34] sequencer (rebase -i): copy commit notes at end

When rebasing commits that have commit notes attached, the interactive rebase rewrites those notes faithfully at the end. The sequencer must do this, too, if it wishes to do interactive rebase's job.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)
Show changes to sequencer.c +75 −0
diff --git a/sequencer.c b/sequencer.c
index 54f654c..a5e18f3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -94,6 +94,15 @@ static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
  */
 static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 /*
+ * For the post-rewrite hook, we make a list of rewritten commits and
+ * their new sha1s.  The rewritten-pending list keeps the sha1s of
+ * commits that have been processed, but not committed yet,
+ * e.g. because they are waiting for a 'squash' command.
+ */
+static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
+static GIT_PATH_FUNC(rebase_path_rewritten_pending,
+	"rebase-merge/rewritten-pending")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -886,6 +895,43 @@ static int update_squash_messages(enum todo_command command,
 	return write_message(&buf, rebase_path_squash_msg());
 }
 
+static void flush_rewritten_pending(void) {
+	struct strbuf buf = STRBUF_INIT;
+	unsigned char newsha1[20];
+	FILE *out;
+
+	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
+			!get_sha1("HEAD", newsha1) &&
+			(out = fopen(rebase_path_rewritten_list(), "a"))) {
+		char *bol = buf.buf, *eol;
+
+		while (*bol) {
+			eol = strchrnul(bol, '\n');
+			fprintf(out, "%.*s %s\n", (int)(eol - bol),
+					bol, sha1_to_hex(newsha1));
+			if (!*eol)
+				break;
+			bol = eol + 1;
+		}
+		fclose(out);
+		unlink(rebase_path_rewritten_pending());
+	}
+}
+
+static void record_in_rewritten(struct object_id *oid,
+		enum todo_command next_command) {
+	FILE *out = fopen(rebase_path_rewritten_pending(), "a");
+
+	if (!out)
+		return;
+
+	fprintf(out, "%s\n", oid_to_hex(oid));
+	fclose(out);
+
+	if (!is_fixup(next_command))
+		flush_rewritten_pending();
+}
+
 static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct replay_opts *opts, int final_fixup)
 {
@@ -1726,6 +1772,17 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static enum todo_command peek_command(struct todo_list *todo_list, int offset)
+{
+	int i;
+
+	for (i = todo_list->current + offset; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_NOOP)
+			return todo_list->items[i].command;
+
+	return -1;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1784,6 +1841,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (is_rebase_i(opts) && !res)
+				record_in_rewritten(&item->commit->object.oid,
+					peek_command(todo_list, 1));
 			if (res && is_fixup(item->command)) {
 				if (res == 1)
 					intend_to_amend();
@@ -1813,6 +1873,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 	if (is_rebase_i(opts)) {
 		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
+		struct stat st;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
@@ -1857,6 +1918,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			run_command_v_opt(argv, RUN_GIT_CMD);
 			strbuf_reset(&buf);
 		}
+		flush_rewritten_pending();
+		if (!stat(rebase_path_rewritten_list(), &st) &&
+				st.st_size > 0) {
+			struct child_process child = CHILD_PROCESS_INIT;
+
+			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			child.git_cmd = 1;
+			argv_array_push(&child.args, "notes");
+			argv_array_push(&child.args, "copy");
+			argv_array_push(&child.args, "--for-rewrite=rebase");
+			/* we don't care if this copying failed */
+			run_command(&child);
+		}
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.10.0.rc2.102.g5c102ec
Dennis Kaarsemaker· Sep 1, 2016, 07:20 UTC · re: Johannes Schindelin · lore

Re: [PATCH 20/34] sequencer (rebase -i): copy commit notes at end

On wo, 2016-08-31 at 10:55 +0200, Johannes Schindelin wrote:
Show 12 quoted lines
> +               if (!stat(rebase_path_rewritten_list(), &st) &&
> +                               st.st_size > 0) {
> +                       struct child_process child = CHILD_PROCESS_INIT;
> +
> +                       child.in = open(rebase_path_rewritten_list(), O_RDONLY);
> +                       child.git_cmd = 1;
> +                       argv_array_push(&child.args, "notes");
> +                       argv_array_push(&child.args, "copy");
> +                       argv_array_push(&child.args, "--for-rewrite=rebase");
> +                       /* we don't care if this copying failed */
> +                       run_command(&child);
> +               }

I know this is a strict port of git-rebase--interactive.sh, but shouldn't we at least warn the user that the copy failed?

D.
Johannes Schindelin· Sep 1, 2016, 14:57 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 20/34] sequencer (rebase -i): copy commit notes at end

Hi Dennis,
On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
Show 16 quoted lines
> On wo, 2016-08-31 at 10:55 +0200, Johannes Schindelin wrote:
> > +               if (!stat(rebase_path_rewritten_list(), &st) &&
> > +                               st.st_size > 0) {
> > +                       struct child_process child = CHILD_PROCESS_INIT;
> > +
> > +                       child.in = open(rebase_path_rewritten_list(), O_RDONLY);
> > +                       child.git_cmd = 1;
> > +                       argv_array_push(&child.args, "notes");
> > +                       argv_array_push(&child.args, "copy");
> > +                       argv_array_push(&child.args, "--for-rewrite=rebase");
> > +                       /* we don't care if this copying failed */
> > +                       run_command(&child);
> > +               }
> 
> I know this is a strict port of git-rebase--interactive.sh, but
> shouldn't we at least warn the user that the copy failed?

At this point, I want to have as faithful a conversion as possible (except that I do not care for the speed of git-rebase--interactive.sh, of course).

Besides, I am fairly certain that a failure will result in an error message. We just do not act on the exit value.

Ciao, Dscho

Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 22/34] sequencer (rebase -i): run the post-rewrite hook, if needed

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
Show changes to sequencer.c +13 −0
diff --git a/sequencer.c b/sequencer.c
index a00d349..153116e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1922,6 +1922,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process child = CHILD_PROCESS_INIT;
+			const char *post_rewrite_hook =
+				find_hook("post-rewrite");
 
 			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			child.git_cmd = 1;
@@ -1930,6 +1932,17 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			argv_array_push(&child.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
 			run_command(&child);
+
+			if (post_rewrite_hook) {
+				struct child_process hook = CHILD_PROCESS_INIT;
+
+				hook.in = open(rebase_path_rewritten_list(),
+					O_RDONLY);
+				argv_array_push(&hook.args, post_rewrite_hook);
+				argv_array_push(&hook.args, "rebase");
+				/* we don't care if this hook failed */
+				run_command(&hook);
+			}
 		}
 
 		strbuf_release(&buf);
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 26/34] sequencer (rebase -i): implement the 'drop' command

The parsing part of a 'drop' command is almost identical to parsing a 'pick', while the operation is the same as that of a 'noop'.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
Show changes to sequencer.c +7 −5
diff --git a/sequencer.c b/sequencer.c
index 366ee78..51c2f76 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -762,7 +762,8 @@ enum todo_command {
 	TODO_FIXUP,
 	TODO_SQUASH,
 	TODO_EXEC,
-	TODO_NOOP
+	TODO_NOOP,
+	TODO_DROP
 };
 
 static struct {
@@ -776,7 +777,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
-	{ 0,   "noop" }
+	{ 0,   "noop" },
+	{ 'd', "drop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -1309,7 +1311,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		else if (is_fixup(item->command))
 			return error("Cannot '%s' without a previous commit",
 				command_to_string(item->command));
-		else if (item->command != TODO_NOOP)
+		else if (item->command < TODO_NOOP)
 			fixup_okay = 1;
 		p = *eol ? eol + 1 : eol;
 	}
@@ -1800,7 +1802,7 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	int i;
 
 	for (i = todo_list->current + offset; i < todo_list->nr; i++)
-		if (todo_list->items[i].command != TODO_NOOP)
+		if (todo_list->items[i].command < TODO_NOOP)
 			return todo_list->items[i].command;
 
 	return -1;
@@ -1933,7 +1935,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_exec(item->arg);
 			*end_of_arg = saved;
 		}
-		else if (item->command != TODO_NOOP)
+		else if (item->command < TODO_NOOP)
 			return error("Unknown command %d", item->command);
 
 		todo_list->current++;
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 23/34] sequencer (rebase -i): respect the rebase.autostash setting

Git's `rebase` command inspects the `rebase.autostash` config setting to determine whether it should stash any uncommitted changes before rebasing and re-apply them afterwards.

As we introduce more bits and pieces to let the sequencer act as interactive rebase's backend, here is the part that adds support for the autostash feature.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
Show changes to sequencer.c +43 −0
diff --git a/sequencer.c b/sequencer.c
index 153116e..2922837 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -111,6 +111,7 @@ static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
+static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -1783,6 +1784,47 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	return -1;
 }
 
+static int apply_autostash(struct replay_opts *opts)
+{
+	struct strbuf stash_sha1 = STRBUF_INIT;
+	struct child_process child = CHILD_PROCESS_INIT;
+	int ret = 0;
+
+	if (!read_oneliner(&stash_sha1, rebase_path_autostash(), 1)) {
+		strbuf_release(&stash_sha1);
+		return 0;
+	}
+	strbuf_trim(&stash_sha1);
+
+	child.git_cmd = 1;
+	argv_array_push(&child.args, "stash");
+	argv_array_push(&child.args, "apply");
+	argv_array_push(&child.args, stash_sha1.buf);
+	if (!run_command(&child))
+		printf(_("Applied autostash."));
+	else {
+		struct child_process store = CHILD_PROCESS_INIT;
+
+		store.git_cmd = 1;
+		argv_array_push(&store.args, "stash");
+		argv_array_push(&store.args, "store");
+		argv_array_push(&store.args, "-m");
+		argv_array_push(&store.args, "autostash");
+		argv_array_push(&store.args, "-q");
+		argv_array_push(&store.args, stash_sha1.buf);
+		if (run_command(&store))
+			ret = error(_("Cannot store %s"), stash_sha1.buf);
+		else
+			printf(_("Applying autostash resulted in conflicts.\n"
+				"Your changes are safe in the stash.\n"
+				"You can run \"git stash pop\" or"
+				" \"git stash drop\" at any time.\n"));
+	}
+
+	strbuf_release(&stash_sha1);
+	return ret;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1944,6 +1986,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				run_command(&hook);
 			}
 		}
+		apply_autostash(opts);
 
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

In the upcoming patch, we will support rebase -i's progress reporting. The progress skips comments but counts 'noop's.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
Show changes to sequencer.c +8 −6
diff --git a/sequencer.c b/sequencer.c
index 51c2f76..4c902e5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -763,7 +763,8 @@ enum todo_command {
 	TODO_SQUASH,
 	TODO_EXEC,
 	TODO_NOOP,
-	TODO_DROP
+	TODO_DROP,
+	TODO_COMMENT
 };
 
 static struct {
@@ -778,12 +779,13 @@ static struct {
 	{ 's', "squash" },
 	{ 'x', "exec" },
 	{ 0,   "noop" },
-	{ 'd', "drop" }
+	{ 'd', "drop" },
+	{ 0,   NULL }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if (command < ARRAY_SIZE(todo_command_info))
+	if (command < TODO_COMMENT)
 		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
@@ -1235,14 +1237,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	bol += strspn(bol, " \t");
 
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
-		item->command = TODO_NOOP;
+		item->command = TODO_COMMENT;
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = eol - bol;
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+	for (i = 0; i < TODO_COMMENT; i++)
 		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
@@ -1252,7 +1254,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_info))
+	if (i >= TODO_COMMENT)
 		return -1;
 
 	if (item->command == TODO_NOOP) {
-- 
2.10.0.rc2.102.g5c102ec
Dennis Kaarsemaker· Sep 1, 2016, 08:33 UTC · re: Johannes Schindelin · lore

Re: [PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
Show 12 quoted lines
> diff --git a/sequencer.c b/sequencer.c
> index 51c2f76..4c902e5 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -763,7 +763,8 @@ enum todo_command {
>         TODO_SQUASH,
>         TODO_EXEC,
>         TODO_NOOP,
> -       TODO_DROP
> +       TODO_DROP,
> +       TODO_COMMENT
>  };
(picking a random commit that touches this enum)

In a few places you now make comparisons like "< TODO_NOOP", so I think it would be good to have a comment near the definition of this enum that says that ordering matters and why, so people don't attempt to add a new TODO_FOOBAR at the end.

D. 
Johannes Schindelin· Sep 1, 2016, 15:32 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

Hi Dennis,
On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
Show 20 quoted lines
> On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
> > diff --git a/sequencer.c b/sequencer.c
> > index 51c2f76..4c902e5 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -763,7 +763,8 @@ enum todo_command {
> >         TODO_SQUASH,
> >         TODO_EXEC,
> >         TODO_NOOP,
> > -       TODO_DROP
> > +       TODO_DROP,
> > +       TODO_COMMENT
> >  };
> 
> (picking a random commit that touches this enum)
> 
> In a few places you now make comparisons like "< TODO_NOOP", so I think
> it would be good to have a comment near the definition of this enum
> that says that ordering matters and why, so people don't attempt to add
> a new TODO_FOOBAR at the end.
True.

It does not seem that we have a precedent for that. The closest is what I had in an early iteration of the fsck message IDs, and subsequently things were refactored so that it is not the order, but a flag, that determines what the command does.

Not sure how to do this elegantly. Maybe like this?
	enum todo_command {
		TODO_PICK_COMMANDS = 0,
		TODO_PICK = TODO_PICK_COMMANDS,
		TODO_SQUASH,
		TODO_NON_PICK_COMMANDS,
		TODO_EXEC = TODO_NON_PICK_COMMANDS,
		TODO_NOOP_COMMANDS,
		TODO_NOOP = TODO_NOOP_COMMANDS,
		TODO_DROP
		TODO_DROP,
		TODO_LAST_COMMAND,
		TODO_COMMENT = TODO_LAST_COMMAND
	};
But that is so god-awful to read.

Still unsure, Dscho

Dennis Kaarsemaker· Sep 1, 2016, 16:15 UTC · re: Johannes Schindelin · lore

Re: [PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

On do, 2016-09-01 at 17:32 +0200, Johannes Schindelin wrote:
Show 57 quoted lines
> Hi Dennis,
> 
> On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
> 
> > 
> > On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
> > > 
> > > diff --git a/sequencer.c b/sequencer.c
> > > index 51c2f76..4c902e5 100644
> > > --- a/sequencer.c
> > > +++ b/sequencer.c
> > > @@ -763,7 +763,8 @@ enum todo_command {
> > >         TODO_SQUASH,
> > >         TODO_EXEC,
> > >         TODO_NOOP,
> > > -       TODO_DROP
> > > +       TODO_DROP,
> > > +       TODO_COMMENT
> > >  };
> > (picking a random commit that touches this enum)
> > 
> > In a few places you now make comparisons like "< TODO_NOOP", so I
> > think
> > it would be good to have a comment near the definition of this enum
> > that says that ordering matters and why, so people don't attempt to
> > add
> > a new TODO_FOOBAR at the end.
> True.
> 
> It does not seem that we have a precedent for that. The closest is
> what I
> had in an early iteration of the fsck message IDs, and subsequently
> things
> were refactored so that it is not the order, but a flag, that
> determines
> what the command does.
> 
> Not sure how to do this elegantly. Maybe like this?
> 
> 	enum todo_command {
> 		TODO_PICK_COMMANDS = 0,
> 		TODO_PICK = TODO_PICK_COMMANDS,
> 		TODO_SQUASH,
> 
> 		TODO_NON_PICK_COMMANDS,
> 		TODO_EXEC = TODO_NON_PICK_COMMANDS,
> 
> 		TODO_NOOP_COMMANDS,
> 		TODO_NOOP = TODO_NOOP_COMMANDS,
> 		TODO_DROP
> 		TODO_DROP,
> 
> 		TODO_LAST_COMMAND,
> 		TODO_COMMENT = TODO_LAST_COMMAND
> 	};
> 
> But that is so god-awful to read.
Agreed, that sure is awful.
How about something like
/*
 * Note that ordering matters in this enum. Not only must it match the
 * mapping below, it is also divided into several sections that matter.
 * When adding new commands, make sure you add it in the right section.
 */
enum todo_command {
	/* All commands that handle commits */
	TODO_PICK,
	...
	/* All commands that do something else than pick */
	TODO_EXEC,
	...
	/* All commands that do nothing but are counted for reporting progress */
	TODO_NOOP,
	...
	/* Comments, which are not counted
	TODO_COMMENT
}
Johannes Schindelin· Sep 2, 2016, 07:32 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

Hi Dennis,
On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
Show 18 quoted lines
> /*
>  * Note that ordering matters in this enum. Not only must it match the
>  * mapping below, it is also divided into several sections that matter.
>  * When adding new commands, make sure you add it in the right section.
>  */
> enum todo_command {
> 	/* All commands that handle commits */
> 	TODO_PICK,
> 	...
> 	/* All commands that do something else than pick */
> 	TODO_EXEC,
> 	...
> 	/* All commands that do nothing but are counted for reporting progress */
> 	TODO_NOOP,
> 	...
> 	/* Comments, which are not counted
> 	TODO_COMMENT
> }
I like it! Changed accordingly.

Thanks! Dscho

Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 24/34] sequencer (rebase -i): respect strategy/strategy_opts settings

The sequencer already has an idea about using different merge strategies. We just piggy-back on top of that, using rebase -i's own settings, when running the sequencer in interactive rebase mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
Show changes to sequencer.c +22 −0
diff --git a/sequencer.c b/sequencer.c
index 2922837..5f5fb9f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -112,6 +112,8 @@ static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
+static GIT_PATH_FUNC(rebase_path_strategy, "rebase-merge/strategy")
+static GIT_PATH_FUNC(rebase_path_strategy_opts, "rebase-merge/strategy_opts")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -1404,6 +1406,26 @@ static int read_populate_opts(struct replay_opts *opts)
 		if (file_exists(rebase_path_verbose()))
 			opts->verbose = 1;
 
+		if (read_oneliner(&buf, rebase_path_strategy(), 0)) {
+			opts->strategy =
+				sequencer_entrust(opts,
+						  strbuf_detach(&buf, NULL));
+			if (read_oneliner(&buf,
+					  rebase_path_strategy_opts(), 0)) {
+				int i;
+				opts->xopts_nr = split_cmdline(buf.buf,
+					&opts->xopts);
+				for (i = 0; i < opts->xopts_nr; i++)
+					skip_prefix(opts->xopts[i], "--",
+						    &opts->xopts[i]);
+				if (opts->xopts_nr)
+					sequencer_entrust(opts,
+						strbuf_detach(&buf, NULL));
+				else
+					strbuf_release(&buf);
+			}
+		}
+
 		return 0;
 	}
 
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 25/34] sequencer (rebase -i): allow rescheduling commands

The interactive rebase has the very special magic that a cherry-pick that exits with a status different from 0 and 1 signifies a failure to even record that a cherry-pick was started.

This can happen e.g. when a fast-forward fails because it would overwrite untracked files.

In that case, we must reschedule the command that we thought we already had at least started successfully.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++++
 1 file changed, 6 insertions(+)
Show changes to sequencer.c +6 −0
diff --git a/sequencer.c b/sequencer.c
index 5f5fb9f..366ee78 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1895,6 +1895,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
+			if (is_rebase_i(opts) && res < 0) {
+				/* Reschedule */
+				todo_list->current--;
+				if (save_todo(todo_list, opts))
+					return -1;
+			}
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 17/34] sequencer (rebase -i): allow fast-forwarding for edit/reword

The sequencer already knew how to fast-forward instead of cherry-picking, if possible.

We want to continue to do this, of course, but in case of the 'reword' command, we will need to call `git commit` after fast-forwarding.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
Show changes to sequencer.c +17 −8
diff --git a/sequencer.c b/sequencer.c
index 64fec47..399fc8d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -896,7 +896,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res = 0, unborn = 0, amend = 0, allow;
+	int res = 0, unborn = 0, amend = 0, allow = 0;
 
 	if (opts->no_commit) {
 		/*
@@ -942,11 +942,23 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
+	if (get_message(commit, &msg) != 0)
+		return error(_("Cannot get commit message for %s"),
+			oid_to_hex(&commit->object.oid));
+
 	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
-	     (!parent && unborn)))
-		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
-
+	     (!parent && unborn))) {
+		if (is_rebase_i(opts))
+			write_author_script(msg.message);
+		res |= fast_forward_to(commit->object.oid.hash, head, unborn,
+			opts);
+		if (res || command != TODO_REWORD)
+			goto leave;
+		edit = amend = 1;
+		msg_file = NULL;
+		goto fast_forward_edit;
+	}
 	if (parent && parse_commit(parent) < 0)
 		/* TRANSLATORS: The first %s will be "revert" or
 		   "cherry-pick", the second %s a SHA1 */
@@ -954,10 +966,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			command_to_string(command),
 			oid_to_hex(&parent->object.oid));
 
-	if (get_message(commit, &msg) != 0)
-		return error(_("Cannot get commit message for %s"),
-			oid_to_hex(&commit->object.oid));
-
 	/*
 	 * "commit" is an existing commit.  We would want to apply
 	 * the difference it introduces since its first parent "prev"
@@ -1083,6 +1091,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
+fast_forward_edit:
 		res |= sequencer_commit(msg_file, opts, allow, edit, amend,
 			cleanup_commit_message);
 
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 28/34] run_command_opt(): optionally hide stderr when the command succeeds

This will be needed to hide the output of `git commit` when the sequencer handles an interactive rebase's script.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 run-command.c | 23 +++++++++++++++++++++++
 run-command.h |  1 +
 2 files changed, 24 insertions(+)
Show changes to 2 files +24 −0

run-command.c, run-command.h

diff --git a/run-command.c b/run-command.c
index 5a4dbb6..921e43c 100644
--- a/run-command.c
+++ b/run-command.c
@@ -575,6 +575,29 @@ int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const
 	cmd.clean_on_exit = opt & RUN_CLEAN_ON_EXIT ? 1 : 0;
 	cmd.dir = dir;
 	cmd.env = env;
+
+	if (opt & RUN_HIDE_STDERR_ON_SUCCESS) {
+		struct strbuf buf = STRBUF_INIT;
+		int res;
+
+		cmd.err = -1;
+		if (start_command(&cmd) < 0)
+			return -1;
+
+		if (strbuf_read(&buf, cmd.err, 0) < 0) {
+			close(cmd.err);
+			finish_command(&cmd); /* throw away exit code */
+			return -1;
+		}
+
+		close(cmd.err);
+		res = finish_command(&cmd);
+		if (res)
+			fputs(buf.buf, stderr);
+		strbuf_release(&buf);
+		return res;
+	}
+
 	return run_command(&cmd);
 }
 
diff --git a/run-command.h b/run-command.h
index 5066649..f87d01a 100644
--- a/run-command.h
+++ b/run-command.h
@@ -70,6 +70,7 @@ extern int run_hook_ve(const char *const *env, const char *name, va_list args);
 #define RUN_SILENT_EXEC_FAILURE 8
 #define RUN_USING_SHELL 16
 #define RUN_CLEAN_ON_EXIT 32
+#define RUN_HIDE_STDERR_ON_SUCCESS 64
 int run_command_v_opt(const char **argv, int opt);
 
 /*
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 29/34] sequencer (rebase -i): show only failed `git commit`'s output

This is the behavior of the shell script version of the interactive rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
Show changes to sequencer.c +7 −2
diff --git a/sequencer.c b/sequencer.c
index 4c902e5..22568d2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -640,10 +640,15 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 {
 	char **env = NULL;
 	struct argv_array array;
-	int rc;
+	int opt = RUN_GIT_CMD, rc;
 	const char *value;
 
 	if (is_rebase_i(opts)) {
+		if (!edit) {
+			opt |= RUN_COMMAND_STDOUT_TO_STDERR;
+			opt |= RUN_HIDE_STDERR_ON_SUCCESS;
+		}
+
 		env = read_author_script();
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
@@ -688,7 +693,7 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
 			(const char *const *)env);
 	argv_array_clear(&array);
 	free(env);
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 31/34] sequencer (rebase -i): suggest --edit-todo upon unknown command

This is the same behavior as known from `git rebase -i`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Show changes to sequencer.c +5 −1
diff --git a/sequencer.c b/sequencer.c
index b34c381..89fd625 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1346,8 +1346,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (res)
+	if (res) {
+		if (is_rebase_i(opts))
+			return error("Please fix this using "
+				"'git rebase --edit-todo'.");
 		return error(_("Unusable instruction sheet: %s"), todo_file);
+	}
 	if (!todo_list->nr &&
 	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
 		return error(_("No commits parsed."));
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 32/34] sequencer (rebase -i): show the progress

The interactive rebase keeps the user informed about its progress. If the sequencer wants to do the grunt work of the interactive rebase, it also needs to show that progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
Show changes to sequencer.c +33 −0
diff --git a/sequencer.c b/sequencer.c
index 89fd625..e8c6daf 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1218,6 +1218,7 @@ struct todo_list {
 	struct strbuf buf;
 	struct todo_item *items;
 	int nr, alloc, current;
+	int done_nr, total_nr;
 };
 
 #define TODO_LIST_INIT { STRBUF_INIT }
@@ -1329,6 +1330,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	return res;
 }
 
+static int count_commands(struct todo_list *todo_list)
+{
+	int count = 0, i;
+
+	for (i = 0; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_COMMENT)
+			count++;
+
+	return count;
+}
+
 static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
@@ -1355,6 +1367,22 @@ static int read_populate_todo(struct todo_list *todo_list,
 	if (!todo_list->nr &&
 	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
 		return error(_("No commits parsed."));
+
+	if (is_rebase_i(opts)) {
+		struct todo_list done = TODO_LIST_INIT;
+
+		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
+				!parse_insn_buffer(done.buf.buf, &done))
+			todo_list->done_nr = count_commands(&done);
+		else
+			todo_list->done_nr = 0;
+
+		todo_list->total_nr = todo_list->done_nr
+			+ count_commands(todo_list);
+
+		todo_list_release(&done);
+	}
+
 	return 0;
 }
 
@@ -1900,6 +1928,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
+			if (item->command != TODO_COMMENT)
+				fprintf(stderr, "Rebasing (%d/%d)%s",
+					++(todo_list->done_nr),
+					todo_list->total_nr,
+					opts->verbose ? "\n" : "\r");
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.10.0.rc2.102.g5c102ec
Dennis Kaarsemaker· Sep 1, 2016, 08:33 UTC · re: Johannes Schindelin · lore

Re: [PATCH 32/34] sequencer (rebase -i): show the progress

On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
Show 74 quoted lines
> The interactive rebase keeps the user informed about its progress.
> If the sequencer wants to do the grunt work of the interactive
> rebase, it also needs to show that progress.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 89fd625..e8c6daf 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1218,6 +1218,7 @@ struct todo_list {
>  	struct strbuf buf;
>  	struct todo_item *items;
>  	int nr, alloc, current;
> +	int done_nr, total_nr;
>  };
>  
>  #define TODO_LIST_INIT { STRBUF_INIT }
> @@ -1329,6 +1330,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
>  	return res;
>  }
>  
> +static int count_commands(struct todo_list *todo_list)
> +{
> +	int count = 0, i;
> +
> +	for (i = 0; i < todo_list->nr; i++)
> +		if (todo_list->items[i].command != TODO_COMMENT)
> +			count++;
> +
> +	return count;
> +}
> +
>  static int read_populate_todo(struct todo_list *todo_list,
>  			struct replay_opts *opts)
>  {
> @@ -1355,6 +1367,22 @@ static int read_populate_todo(struct todo_list *todo_list,
>  	if (!todo_list->nr &&
>  	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
>  		return error(_("No commits parsed."));
> +
> +	if (is_rebase_i(opts)) {
> +		struct todo_list done = TODO_LIST_INIT;
> +
> +		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
> +				!parse_insn_buffer(done.buf.buf, &done))
> +			todo_list->done_nr = count_commands(&done);
> +		else
> +			todo_list->done_nr = 0;
> +
> +		todo_list->total_nr = todo_list->done_nr
> +			+ count_commands(todo_list);
> +
> +		todo_list_release(&done);
> +	}
> +
>  	return 0;
>  }
>  
> @@ -1900,6 +1928,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  		if (save_todo(todo_list, opts))
>  			return -1;
>  		if (is_rebase_i(opts)) {
> +			if (item->command != TODO_COMMENT)
> +				fprintf(stderr, "Rebasing (%d/%d)%s",
> +					++(todo_list->done_nr),
> +					todo_list->total_nr,
> +					opts->verbose ? "\n" : "\r");
>  			unlink(rebase_path_message());
>  			unlink(rebase_path_author_script());
>  			unlink(rebase_path_stopped_sha());
(picking a random commit that shows this 'symptom')

You're sprinking a lot of is_rebase_i's around sequencer.c to make sure there are no changes in behaviour. I wonder if the right balance has been struck between 'no changes in behaviour' and 'common behaviour'. For instance, in this case, maybe it would be a better idea for non- rebase uses of the sequencer to also show progress.

D.
Johannes Schindelin· Sep 1, 2016, 15:19 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 32/34] sequencer (rebase -i): show the progress

Hi Dennis,
On Thu, 1 Sep 2016, Dennis Kaarsemaker wrote:
Show 83 quoted lines
> On wo, 2016-08-31 at 10:56 +0200, Johannes Schindelin wrote:
> > The interactive rebase keeps the user informed about its progress.
> > If the sequencer wants to do the grunt work of the interactive
> > rebase, it also needs to show that progress.
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 33 +++++++++++++++++++++++++++++++++
> >  1 file changed, 33 insertions(+)
> > 
> > diff --git a/sequencer.c b/sequencer.c
> > index 89fd625..e8c6daf 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -1218,6 +1218,7 @@ struct todo_list {
> >  	struct strbuf buf;
> >  	struct todo_item *items;
> >  	int nr, alloc, current;
> > +	int done_nr, total_nr;
> >  };
> >  
> >  #define TODO_LIST_INIT { STRBUF_INIT }
> > @@ -1329,6 +1330,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
> >  	return res;
> >  }
> >  
> > +static int count_commands(struct todo_list *todo_list)
> > +{
> > +	int count = 0, i;
> > +
> > +	for (i = 0; i < todo_list->nr; i++)
> > +		if (todo_list->items[i].command != TODO_COMMENT)
> > +			count++;
> > +
> > +	return count;
> > +}
> > +
> >  static int read_populate_todo(struct todo_list *todo_list,
> >  			struct replay_opts *opts)
> >  {
> > @@ -1355,6 +1367,22 @@ static int read_populate_todo(struct todo_list *todo_list,
> >  	if (!todo_list->nr &&
> >  	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
> >  		return error(_("No commits parsed."));
> > +
> > +	if (is_rebase_i(opts)) {
> > +		struct todo_list done = TODO_LIST_INIT;
> > +
> > +		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
> > +				!parse_insn_buffer(done.buf.buf, &done))
> > +			todo_list->done_nr = count_commands(&done);
> > +		else
> > +			todo_list->done_nr = 0;
> > +
> > +		todo_list->total_nr = todo_list->done_nr
> > +			+ count_commands(todo_list);
> > +
> > +		todo_list_release(&done);
> > +	}
> > +
> >  	return 0;
> >  }
> >  
> > @@ -1900,6 +1928,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  		if (save_todo(todo_list, opts))
> >  			return -1;
> >  		if (is_rebase_i(opts)) {
> > +			if (item->command != TODO_COMMENT)
> > +				fprintf(stderr, "Rebasing (%d/%d)%s",
> > +					++(todo_list->done_nr),
> > +					todo_list->total_nr,
> > +					opts->verbose ? "\n" : "\r");
> >  			unlink(rebase_path_message());
> >  			unlink(rebase_path_author_script());
> >  			unlink(rebase_path_stopped_sha());
> 
> (picking a random commit that shows this 'symptom')
> 
> You're sprinking a lot of is_rebase_i's around sequencer.c to make sure
> there are no changes in behaviour. I wonder if the right balance has
> been struck between 'no changes in behaviour' and 'common behaviour'.
> For instance, in this case, maybe it would be a better idea for non-
> rebase uses of the sequencer to also show progress.

Actually, adding progress would make for a fine add-on patch series. I still would like to have as faithful a conversion as possible, and everything else can come after that.

This strategy has a couple of advantages:
- we can concentrate on correctness for now,
- I get something to show for my time with Git for Windows v2.10.0, and
- the add-on patches do not have to be done by *me* ;-)

Ciao, Dscho

Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 33/34] sequencer (rebase -i): write the progress into files

For the benefit of e.g. the shell prompt, the interactive rebase not only displays the progress for the user to see, but also writes it into the msgnum/end files in the state directory.

Teach the sequencer this new trick.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
Show changes to sequencer.c +27 −3
diff --git a/sequencer.c b/sequencer.c
index e8c6daf..9a8069e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -45,6 +45,16 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The file to keep track of how many commands were already processed (e.g.
+ * for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgnum, "rebase-merge/msgnum");
+/*
+ * The file to keep track of how many commands are to be processed in total
+ * (e.g. for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgtotal, "rebase-merge/end");
+/*
  * The commit message that is planned to be used for any changes that
  * need to be committed following a user interaction.
  */
@@ -1370,17 +1380,22 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 	if (is_rebase_i(opts)) {
 		struct todo_list done = TODO_LIST_INIT;
+		FILE *f = fopen(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
 				!parse_insn_buffer(done.buf.buf, &done))
 			todo_list->done_nr = count_commands(&done);
 		else
 			todo_list->done_nr = 0;
+		todo_list_release(&done);
 
 		todo_list->total_nr = todo_list->done_nr
 			+ count_commands(todo_list);
 
-		todo_list_release(&done);
+		if (f) {
+			fprintf(f, "%d\n", todo_list->total_nr);
+			fclose(f);
+		}
 	}
 
 	return 0;
@@ -1928,11 +1943,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
-			if (item->command != TODO_COMMENT)
+			if (item->command != TODO_COMMENT) {
+				FILE *f = fopen(rebase_path_msgnum(), "w");
+
+				todo_list->done_nr++;
+
+				if (f) {
+					fprintf(f, "%d\n", todo_list->done_nr);
+					fclose(f);
+				}
 				fprintf(stderr, "Rebasing (%d/%d)%s",
-					++(todo_list->done_nr),
+					todo_list->done_nr,
 					todo_list->total_nr,
 					opts->verbose ? "\n" : "\r");
+			}
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 30/34] sequencer (rebase -i): show only failed cherry-picks' output

This is the behavior of the shell script version of the interactive rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)
Show changes to sequencer.c +4 −0
diff --git a/sequencer.c b/sequencer.c
index 22568d2..b34c381 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -477,6 +477,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	o.ancestor = base ? base_label : "(empty tree)";
 	o.branch1 = "HEAD";
 	o.branch2 = next ? next_label : "(empty tree)";
+	if (is_rebase_i(opts))
+		o.buffer_output = 2;
 
 	head_tree = parse_tree_indirect(head);
 	next_tree = next ? next->tree : empty_tree();
@@ -488,6 +490,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (is_rebase_i(opts) && clean <= 0)
+		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:56 UTC · re: Johannes Schindelin · lore

[PATCH 34/34] sequencer (rebase -i): write out the final message

The shell script version of the interactive rebase has a very specific final message. Teach the sequencer to print the same.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)
Show changes to sequencer.c +3 −0
diff --git a/sequencer.c b/sequencer.c
index 9a8069e..6064018 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2090,6 +2090,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
+		fprintf(stderr, "Successfully rebased and updated %s.\n",
+			head_ref.buf);
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.10.0.rc2.102.g5c102ec
Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 15/34] sequencer (rebase -i): leave a patch upon error

Just like the interactive rebase, we want to leave a 'patch' file for further inspection by the user (even if we never tried to actually apply that patch, since we're cherry-picking instead).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)
Show changes to sequencer.c +3 −0
diff --git a/sequencer.c b/sequencer.c
index 7662222..9a06b40 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1753,6 +1753,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
 			}
+			else if (res && is_rebase_i(opts))
+				return res | error_with_patch(item->commit,
+					item->arg, item->arg_len, opts, res, 0);
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-- 
2.10.0.rc2.102.g5c102ec
Dennis Kaarsemaker· Aug 31, 2016, 18:39 UTC · re: Johannes Schindelin · lore

Re: [PATCH 15/34] sequencer (rebase -i): leave a patch upon error

On wo, 2016-08-31 at 10:55 +0200, Johannes Schindelin wrote:
Show 5 quoted lines
> Just like the interactive rebase, we want to leave a 'patch' file for
> further inspection by the user (even if we never tried to actually apply
> that patch, since we're cherry-picking instead).
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

This commit message confuses me. Did you mean s/Just like the/When doing an/? 

D.
Johannes Schindelin· Sep 1, 2016, 13:03 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 15/34] sequencer (rebase -i): leave a patch upon error

Hi Dennis,
On Wed, 31 Aug 2016, Dennis Kaarsemaker wrote:
Show 9 quoted lines
> On wo, 2016-08-31 at 10:55 +0200, Johannes Schindelin wrote:
> > Just like the interactive rebase, we want to leave a 'patch' file for
> > further inspection by the user (even if we never tried to actually apply
> > that patch, since we're cherry-picking instead).
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> This commit message confuses me. Did you mean s/Just like the/When
> doing an/? 

What I meant is this: when calling `git rebase -i` right now, i.e. before any of my rebase--helper work, a failing rebase will leave a `patch` file in the `.git/rebase-merge/` directory. Since the sequencer is in the process of learning how to do an interactive rebase, it needs to learn the same trick.

But I guess that your suggested edit makes things much clearer.

Thanks, Dscho

Johannes Schindelin· Aug 31, 2016, 08:54 UTC · re: Johannes Schindelin · lore

[PATCH 06/34] sequencer (rebase -i): write the 'done' file

In the interactive rebase, commands that were successfully processed are not simply discarded, but appended to the 'done' file instead. This is used e.g. to display the current state to the user in the output of `git status` or the progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
Show changes to sequencer.c +20 −0
diff --git a/sequencer.c b/sequencer.c
index 95c31bb..3bff3d9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -39,6 +39,12 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  */
 static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
+ * The rebase command lines that have already been processed. A line
+ * is moved here when it is first handled, before any associated user
+ * actions.
+ */
+static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -1272,6 +1278,20 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 			todo_path, strerror(errno));
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("Error wrapping up %s."), todo_path);
+
+	if (is_rebase_i(opts)) {
+		const char *done_path = rebase_path_done();
+		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int prev_offset = !next ? 0 :
+			todo_list->items[next - 1].offset_in_buf;
+
+		if (offset > prev_offset && write_in_full(fd,
+				todo_list->buf.buf + prev_offset,
+				offset - prev_offset) < 0)
+			return error(_("Could not write to %s (%s)"),
+				done_path, strerror(errno));
+		close(fd);
+	}
 	return 0;
 }
 
-- 
2.10.0.rc2.102.g5c102ec
Dennis Kaarsemaker· Aug 31, 2016, 18:39 UTC · re: Johannes Schindelin · lore

Re: [PATCH 06/34] sequencer (rebase -i): write the 'done' file

On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> In the interactive rebase, commands that were successfully processed are
> not simply discarded, but appended to the 'done' file instead. This is
> used e.g. to display the current state to the user in the output of
> `git status` or the progress.

Wouldn't it make more sense to have this patch before the ones that implement the actual rebase commands?

Hmm, and after reading more of this series, I think the same applies to some other patches too, e.g. 08/34 and 14/34, so I'm probably missing something. So before I make a fool of myself and suggest that the implementation of the actual commands should come at the end, maybe you could tell me what I'm missing :) 

D.
Johannes Schindelin· Sep 1, 2016, 08:51 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 06/34] sequencer (rebase -i): write the 'done' file

Hi Dennis,
On Wed, 31 Aug 2016, Dennis Kaarsemaker wrote:
Show 8 quoted lines
> On wo, 2016-08-31 at 10:54 +0200, Johannes Schindelin wrote:
> > In the interactive rebase, commands that were successfully processed are
> > not simply discarded, but appended to the 'done' file instead. This is
> > used e.g. to display the current state to the user in the output of
> > `git status` or the progress.
> 
> Wouldn't it make more sense to have this patch before the ones that
> implement the actual rebase commands?

I waffled about the order so many times that I don't know anymore. The thing is, while the sequencer is taught incrementally to understand all of the rebase -i functionality, rebase -i itself is not touched, on purpose.

In the case of the "done" file, my thoughts were: the commands do not need this file *at all*. In fact, if we did not write the "done" file at all, the only two types of test failures in the test suite would be 1) git status' output and 2) the prompt testing for the progress.

So you see, functionally, the "done" file is only relevant to the progress part of the patch series.

As such, I'd rather keep this patch in the current place, just before introducing the progress.

Show 5 quoted lines
> Hmm, and after reading more of this series, I think the same applies to
> some other patches too, e.g. 08/34 and 14/34, so I'm probably missing
> something. So before I make a fool of myself and suggest that the
> implementation of the actual commands should come at the end, maybe you
> could tell me what I'm missing :) 

No, no, don't hesitate to suggest reorderings. I am really thankful for the discussion we are having, so that the outcome is better than what I have right now. If the outcome would be the very same patches, but with more confidence, it would still be better than what I have right now ;-)

Ciao, Dscho

Johannes Schindelin· Aug 31, 2016, 08:55 UTC · re: Johannes Schindelin · lore

[PATCH 14/34] sequencer (rebase -i): update refs after a successful rebase

An interactive rebase operates on a detached HEAD (to keep the reflog of the original branch relatively clean), and updates the branch only at the end.

Now that the sequencer learns to perform interactive rebases, it also needs to learn the trick to update the branch before removing the directory containing the state of the interactive rebase.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)
Show changes to sequencer.c +31 −1
diff --git a/sequencer.c b/sequencer.c
index d4437f5..7662222 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -100,6 +100,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
+static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
+static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -1769,12 +1771,39 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
-		struct strbuf buf = STRBUF_INIT;
+		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
 
+		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
+				starts_with(head_ref.buf, "refs/")) {
+			unsigned char head[20], orig[20];
+
+			if (get_sha1("HEAD", head))
+				return error("Cannot read HEAD");
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
+					get_sha1_hex(buf.buf, orig))
+				return error("Could not read orig-head");
+			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
+				head_ref.buf);
+			if (!read_oneliner(&buf, rebase_path_onto(), 0))
+				return error("Could not read 'onto'");
+			if (update_ref(buf.buf, head_ref.buf, head, orig,
+					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
+				return error("Could not update %s",
+					head_ref.buf);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf,
+				"rebase -i (finish): returning to %s",
+				head_ref.buf);
+			if (create_symref("HEAD", head_ref.buf, buf.buf))
+				return error("Could not update HEAD to %s",
+					head_ref.buf);
+			strbuf_reset(&buf);
+		}
+
 		if (opts->verbose) {
 			const char *argv[] = {
 				"diff-tree", "--stat", NULL, NULL
@@ -1789,6 +1818,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			strbuf_reset(&buf);
 		}
 		strbuf_release(&buf);
+		strbuf_release(&head_ref);
 	}
 
 	/*
-- 
2.10.0.rc2.102.g5c102ec
Dennis Kaarsemaker· Aug 31, 2016, 18:39 UTC · re: Johannes Schindelin · lore

Re: [PATCH 00/34] Teach the sequencer to act as rebase -i's backend

On wo, 2016-08-31 at 10:53 +0200, Johannes Schindelin wrote:
> This marks the count down to '3': two more patch series after this
> (really tiny ones) and we have a faster rebase -i.

I got to 16/34 (and skipped 07/34), will continue tomorrow. I hope the comments are useful.

D.
Johannes Schindelin· Sep 1, 2016, 13:05 UTC · re: Dennis Kaarsemaker · lore

Re: [PATCH 00/34] Teach the sequencer to act as rebase -i's backend

Hi Dennis,
On Wed, 31 Aug 2016, Dennis Kaarsemaker wrote:
Show 6 quoted lines
> On wo, 2016-08-31 at 10:53 +0200, Johannes Schindelin wrote:
> > This marks the count down to '3': two more patch series after this
> > (really tiny ones) and we have a faster rebase -i.
> 
> I got to 16/34 (and skipped 07/34), will continue tomorrow. I hope the
> comments are useful.

Much appreciated! Dscho

Johannes Schindelin· Dec 13, 2016, 15:29 UTC · re: Johannes Schindelin · lore

[PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

This marks the count down to '3': two more patch series after this (really tiny ones) and we have a faster rebase -i.

The idea of this patch series is to teach the sequencer to understand all of the commands in `git-rebase-todo` scripts, to execute them and to behave pretty much very the same as `git rebase -i --continue` when called with the newly-introduced REPLAY_INTERACTIVE_REBASE mode.

Most of these patches should be pretty much straight-forward. When not, I tried to make a point of describing enough background in the commit message. Please feel free to point out where my explanations fall short.

Note that even after this patch series is applied, rebase -i is still unaffected. It will require the next patch series which introduces the rebase--helper that essentially implements `git rebase -i --continue` by calling the sequencer with the appropriate options.

The final patch series will move a couple of pre- and post-processing steps into the rebase--helper/sequencer (such as expanding/shrinking the SHA-1s, reordering the fixup!/squash! lines, etc). This might sound like a mere add-on, but it is essential for the speed improvements: those stupid little processing steps really dominated the execution time in my tests.

Apart from mostly cosmetic patches (and the occasional odd bug that I fixed promptly), I used these patches since mid May to perform all of my interactive rebases. In mid June, I had the idea to teach rebase -i to run *both* scripted rebase and rebase--helper and to cross-validate the results. This slowed down all my interactive rebases since, but helped me catch three rather obscure bugs (e.g. that git commit --fixup unfolds long onelines and rebase -i still finds the correct original commit).

This is all only to say that I am rather confident that the current code does the job.

Since sending out v1, I integrated all of these patch series into Git for Windows v2.10.0, where they have been live ever since, and used by myself (also in a Linux VM, as Git for Windows' master branch always builds also on Linux and passes the test suite, too).

Just to reiterate why I do all this: it speeds up the interactive rebase substantially. Even with a not yet fully builtin rebase -i, but just the part after the user edited the `git-rebase-todo` script.

The performance test I introduced to demonstrate this (p3404) shows a speed-up of +380% here (i.e. roughly 5x), from ~8.8 seconds to ~1.8 seconds. This is on Windows, where the performance impact of avoiding shell scripting is most noticable.

On MacOSX and on Linux, the speed-up is less pronounced, but still noticable, at least if you trust Travis CI, which I abused to perform that test for me. Check for yourself (searching for "3404.2") here: https://travis-ci.org/git/git/builds/156295227. According to those logs, p3404 is speeded up from ~0.45 seconds to ~0.12 seconds on Linux (read: about 3.5x) and from ~1.7 seconds to ~0.5 seconds on MacOSX (read: almost 4x).

Please note that the interdiff vs v1 is only of limited use: too many things changed in the meantime, in particular the prepare-sequencer branch that went through a couple of iterations before it found its way into git.git's master branch. So please take the interdiff with a mountain range of salt.

Changes since v1:
- some grammar touch-ups.
- simplified determining the command string in
  walk_revs_populate_todo().
- removed the beautiful ordinal logic (to print out "1st", "2nd", "3rd"
  etc) and made things consistent with the current `rebase -i`.
- while at it, marked more messages for translation.
- added code-comments to clarify the order, and the sections, of the
  todo_command enum.
- replaced one error(..., strerror(...)) to an error_errno(...).
- downcased error messages
- marked error messages for translation
- adjusted the patches to account for sequencer_entrust() having been
  removed from the prepare-sequencer patch series by request of Junio.
- moved the introduction of write_message_gently() into the patch
  introducing its first usage, i.e. the support for the 'edit' command.
- adjusted some indentations
- prevented an write_in_full() from being called after a failed open()
- inserted a few forgotten strbuf_release() calls
Johannes Schindelin (34):
  sequencer: support a new action: 'interactive rebase'
  sequencer (rebase -i): implement the 'noop' command
  sequencer (rebase -i): implement the 'edit' command
  sequencer (rebase -i): implement the 'exec' command
  sequencer (rebase -i): learn about the 'verbose' mode
  sequencer (rebase -i): write the 'done' file
  sequencer (rebase -i): add support for the 'fixup' and 'squash'
    commands
  sequencer (rebase -i): implement the short commands
  sequencer (rebase -i): write an author-script file
  sequencer (rebase -i): allow continuing with staged changes
  sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed
  sequencer (rebase -i): skip some revert/cherry-pick specific code path
  sequencer (rebase -i): the todo can be empty when continuing
  sequencer (rebase -i): update refs after a successful rebase
  sequencer (rebase -i): leave a patch upon error
  sequencer (rebase -i): implement the 'reword' command
  sequencer (rebase -i): allow fast-forwarding for edit/reword
  sequencer (rebase -i): refactor setting the reflog message
  sequencer (rebase -i): set the reflog message consistently
  sequencer (rebase -i): copy commit notes at end
  sequencer (rebase -i): record interrupted commits in rewritten, too
  sequencer (rebase -i): run the post-rewrite hook, if needed
  sequencer (rebase -i): respect the rebase.autostash setting
  sequencer (rebase -i): respect strategy/strategy_opts settings
  sequencer (rebase -i): allow rescheduling commands
  sequencer (rebase -i): implement the 'drop' command
  sequencer (rebase -i): differentiate between comments and 'noop'
  run_command_opt(): optionally hide stderr when the command succeeds
  sequencer (rebase -i): show only failed `git commit`'s output
  sequencer (rebase -i): show only failed cherry-picks' output
  sequencer (rebase -i): suggest --edit-todo upon unknown command
  sequencer (rebase -i): show the progress
  sequencer (rebase -i): write the progress into files
  sequencer (rebase -i): write out the final message
 run-command.c |   23 ++
 run-command.h |    1 +
 sequencer.c   | 1003 ++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 sequencer.h   |    4 +-
 4 files changed, 983 insertions(+), 48 deletions(-)
base-commit: 8d7a455ed52e2a96debc080dfc011b6bb00db5d2
Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v2
Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v2
Interdiff vs v1:
 diff --git a/sequencer.c b/sequencer.c
 index 6ca9d1e09d..41be4cde16 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -125,7 +125,6 @@ static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
  static GIT_PATH_FUNC(rebase_path_strategy, "rebase-merge/strategy")
  static GIT_PATH_FUNC(rebase_path_strategy_opts, "rebase-merge/strategy_opts")
  
 -/* We will introduce the 'interactive rebase' mode later */
  static inline int is_rebase_i(const struct replay_opts *opts)
  {
  	return opts->action == REPLAY_INTERACTIVE_REBASE;
 @@ -259,13 +258,13 @@ static const char *action_name(const struct replay_opts *opts)
  {
  	switch (opts->action) {
  	case REPLAY_REVERT:
 -		return "revert";
 +		return N_("revert");
  	case REPLAY_PICK:
 -		return "cherry-pick";
 +		return N_("cherry-pick");
  	case REPLAY_INTERACTIVE_REBASE:
 -		return "rebase -i";
 +		return N_("rebase -i");
  	}
 -	die("Unknown action: %d", opts->action);
 +	die(_("Unknown action: %d"), opts->action);
  }
  
  struct commit_message {
 @@ -548,6 +547,7 @@ static int write_author_script(const char *message)
  {
  	struct strbuf buf = STRBUF_INIT;
  	const char *eol;
 +	int res;
  
  	for (;;)
  		if (!*message || starts_with(message, "\n")) {
 @@ -585,8 +585,9 @@ static int write_author_script(const char *message)
  			strbuf_addch(&buf, *(message++));
  		else
  			strbuf_addf(&buf, "'\\\\%c'", *(message++));
 -	strbuf_addstr(&buf, "'\n");
 -	return write_message(&buf, rebase_path_author_script());
 +	res = write_message(buf.buf, buf.len, rebase_path_author_script(), 1);
 +	strbuf_release(&buf);
 +	return res;
  }
  
  /*
 @@ -771,16 +772,25 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
  		return 1;
  }
  
 +/*
 + * Note that ordering matters in this enum. Not only must it match the mapping
 + * below, it is also divided into several sections that matter.  When adding
 + * new commands, make sure you add it in the right section.
 + */
  enum todo_command {
 +	/* commands that handle commits */
  	TODO_PICK = 0,
  	TODO_REVERT,
  	TODO_EDIT,
  	TODO_REWORD,
  	TODO_FIXUP,
  	TODO_SQUASH,
 +	/* commands that do something else than handling a single commit */
  	TODO_EXEC,
 +	/* commands that do nothing but are counted for reporting progress */
  	TODO_NOOP,
  	TODO_DROP,
 +	/* comments (not counted for reporting progress) */
  	TODO_COMMENT
  };
  
 @@ -812,47 +822,34 @@ static int is_fixup(enum todo_command command)
  	return command == TODO_FIXUP || command == TODO_SQUASH;
  }
  
 -static const char *nth_for_number(int n)
 -{
 -	int n1 = n % 10, n10 = n % 100;
 -
 -	if (n1 == 1 && n10 != 11)
 -		return "st";
 -	if (n1 == 2 && n10 != 12)
 -		return "nd";
 -	if (n1 == 3 && n10 != 13)
 -		return "rd";
 -	return "th";
 -}
 -
  static int update_squash_messages(enum todo_command command,
  		struct commit *commit, struct replay_opts *opts)
  {
  	struct strbuf buf = STRBUF_INIT;
 -	int count;
 +	int count, res;
  	const char *message, *body;
  
  	if (file_exists(rebase_path_squash_msg())) {
  		char *p, *p2;
  
  		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
 -			return error("Could not read %s",
 +			return error(_("could not read '%s'"),
  				rebase_path_squash_msg());
  
 -		if (buf.buf[0] == '\n' || !skip_prefix(buf.buf + 1,
 -				" This is a combination of ",
 -				(const char **)&p))
 -			return error("Unexpected 1st line of squash message:\n"
 -				"\n\t%.*s",
 -				(int)(strchrnul(buf.buf, '\n') - buf.buf),
 -				buf.buf);
 +		if (buf.buf[0] != comment_line_char ||
 +		    !skip_prefix(buf.buf + 1, " This is a combination of ",
 +				 (const char **)&p))
 +			return error(_("unexpected 1st line of squash message:"
 +				       "\n\n\t%.*s"),
 +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
 +				     buf.buf);
  		count = strtol(p, &p2, 10);
  
  		if (count < 1 || *p2 != ' ')
 -			return error("Invalid 1st line of squash message:\n"
 -				"\n\t%.*s",
 -				(int)(strchrnul(buf.buf, '\n') - buf.buf),
 -				buf.buf);
 +			return error(_("invalid 1st line of squash message:\n"
 +				       "\n\t%.*s"),
 +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
 +				     buf.buf);
  
  		sprintf((char *)p, "%d", ++count);
  		if (!*p2)
 @@ -868,32 +865,33 @@ static int update_squash_messages(enum todo_command command,
  		const char *head_message, *body;
  
  		if (get_sha1("HEAD", head))
 -			return error("Need a HEAD to fixup");
 +			return error(_("need a HEAD to fixup"));
  		if (!(head_commit = lookup_commit_reference(head)))
 -			return error("Could not read HEAD");
 +			return error(_("could not read HEAD"));
  		if (!(head_message = get_commit_buffer(head_commit, NULL)))
 -			return error("Could not read HEAD's commit message");
 +			return error(_("could not read HEAD's commit message"));
  
  		body = strstr(head_message, "\n\n");
  		if (!body)
  			body = "";
  		else
  			body = skip_blank_lines(body + 2);
 -		if (write_file_gently(rebase_path_fixup_msg(), body, 0))
 -			return error("Cannot write %s",
 -				rebase_path_fixup_msg());
 +		if (write_message(body, strlen(body),
 +				  rebase_path_fixup_msg(), 0))
 +			return error(_("cannot write '%s'"),
 +				     rebase_path_fixup_msg());
  
  		count = 2;
 -		strbuf_addf(&buf, "%c This is a combination of 2 commits.\n"
 -			"%c The first commit's message is:\n\n%s",
 -			comment_line_char, comment_line_char, body);
 +		strbuf_addf(&buf, _("%c This is a combination of 2 commits.\n"
 +				    "%c The first commit's message is:\n\n%s"),
 +			    comment_line_char, comment_line_char, body);
  
  		unuse_commit_buffer(head_commit, head_message);
  	}
  
  	if (!(message = get_commit_buffer(commit, NULL)))
 -		return error("Could not read commit message of %s",
 -			oid_to_hex(&commit->object.oid));
 +		return error(_("could not read commit message of %s"),
 +			     oid_to_hex(&commit->object.oid));
  	body = strstr(message, "\n\n");
  	if (!body)
  		body = "";
 @@ -902,21 +900,23 @@ static int update_squash_messages(enum todo_command command,
  
  	if (command == TODO_SQUASH) {
  		unlink(rebase_path_fixup_msg());
 -		strbuf_addf(&buf, "\n%c This is the %d%s commit message:\n\n%s",
 -			comment_line_char,
 -			count, nth_for_number(count), body);
 +		strbuf_addf(&buf, _("\n%c This is the commit message #%d:\n"
 +				    "\n%s"),
 +			    comment_line_char, count, body);
  	}
  	else if (command == TODO_FIXUP) {
 -		strbuf_addf(&buf,
 -			"\n%c The %d%s commit message will be skipped:\n\n",
 -			comment_line_char, count, nth_for_number(count));
 +		strbuf_addf(&buf, _("\n%c The commit message #%d "
 +				    "will be skipped:\n\n"),
 +			    comment_line_char, count);
  		strbuf_add_commented_lines(&buf, body, strlen(body));
  	}
  	else
 -		return error("Unknown command: %d", command);
 +		return error(_("unknown command: %d"), command);
  	unuse_commit_buffer(commit, message);
  
 -	return write_message(&buf, rebase_path_squash_msg());
 +	res = write_message(buf.buf, buf.len, rebase_path_squash_msg(), 0);
 +	strbuf_release(&buf);
 +	return res;
  }
  
  static void flush_rewritten_pending(void) {
 @@ -940,6 +940,7 @@ static void flush_rewritten_pending(void) {
  		fclose(out);
  		unlink(rebase_path_rewritten_pending());
  	}
 +	strbuf_release(&buf);
  }
  
  static void record_in_rewritten(struct object_id *oid,
 @@ -1013,7 +1014,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		parent = commit->parents->item;
  
  	if (get_message(commit, &msg) != 0)
 -		return error(_("Cannot get commit message for %s"),
 +		return error(_("cannot get commit message for %s"),
  			oid_to_hex(&commit->object.oid));
  
  	if (opts->allow_ff && !is_fixup(command) &&
 @@ -1021,7 +1022,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	     (!parent && unborn))) {
  		if (is_rebase_i(opts))
  			write_author_script(msg.message);
 -		res |= fast_forward_to(commit->object.oid.hash, head, unborn,
 +		res = fast_forward_to(commit->object.oid.hash, head, unborn,
  			opts);
  		if (res || command != TODO_REWORD)
  			goto leave;
 @@ -1100,8 +1101,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  			const char *dest = git_path("SQUASH_MSG");
  			unlink(dest);
  			if (copy_file(dest, rebase_path_squash_msg(), 0666))
 -				return error("Could not rename %s to "
 -					"%s", rebase_path_squash_msg(), dest);
 +				return error(_("could not rename '%s' to '%s'"),
 +					     rebase_path_squash_msg(), dest);
  			unlink(git_path("MERGE_MSG"));
  			msg_file = dest;
  			edit = 1;
 @@ -1109,7 +1110,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	}
  
  	if (is_rebase_i(opts) && write_author_script(msg.message) < 0)
 -		res |= -1;
 +		res = -1;
  	else if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
  		res = do_recursive_merge(base, next, base_label, next_label,
  					 head, &msgbuf, opts);
 @@ -1166,8 +1167,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	}
  	if (!opts->no_commit)
  fast_forward_edit:
 -		res |= sequencer_commit(msg_file, opts, allow, edit, amend,
 -			cleanup_commit_message);
 +		res = run_git_commit(msg_file, opts, allow, edit, amend,
 +				     cleanup_commit_message);
  
  	if (!res && final_fixup) {
  		unlink(rebase_path_fixup_msg());
 @@ -1317,7 +1318,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  {
  	struct todo_item *item;
 -	char *p = buf;
 +	char *p = buf, *next_p;
  	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
  
  	for (i = 1; *p; i++, p = next_p) {
 @@ -1335,15 +1336,16 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  				i, (int)(eol - p), p);
  			item->command = TODO_NOOP;
  		}
 +
  		if (fixup_okay)
  			; /* do nothing */
  		else if (is_fixup(item->command))
 -			return error("Cannot '%s' without a previous commit",
 +			return error(_("cannot '%s' without a previous commit"),
  				command_to_string(item->command));
  		else if (item->command < TODO_NOOP)
  			fixup_okay = 1;
 -		p = *eol ? eol + 1 : eol;
  	}
 +
  	return res;
  }
  
 @@ -1377,13 +1379,14 @@ static int read_populate_todo(struct todo_list *todo_list,
  	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
  	if (res) {
  		if (is_rebase_i(opts))
 -			return error("Please fix this using "
 -				"'git rebase --edit-todo'.");
 -		return error(_("Unusable instruction sheet: %s"), todo_file);
 +			return error(_("please fix this using "
 +				       "'git rebase --edit-todo'."));
 +		return error(_("unusable instruction sheet: '%s'"), todo_file);
  	}
 +
  	if (!todo_list->nr &&
  	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
 -		return error(_("No commits parsed."));
 +		return error(_("no commits parsed."));
  
  	if (!is_rebase_i(opts)) {
  		enum todo_command valid =
 @@ -1408,10 +1411,10 @@ static int read_populate_todo(struct todo_list *todo_list,
  			todo_list->done_nr = count_commands(&done);
  		else
  			todo_list->done_nr = 0;
 -		todo_list_release(&done);
  
  		todo_list->total_nr = todo_list->done_nr
  			+ count_commands(todo_list);
 +		todo_list_release(&done);
  
  		if (f) {
  			fprintf(f, "%d\n", todo_list->total_nr);
 @@ -1467,6 +1470,26 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
  	return 0;
  }
  
 +static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
 +{
 +	int i;
 +
 +	strbuf_reset(buf);
 +	if (!read_oneliner(buf, rebase_path_strategy(), 0))
 +		return;
 +	opts->strategy = strbuf_detach(buf, NULL);
 +	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
 +		return;
 +
 +	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
 +	for (i = 0; i < opts->xopts_nr; i++) {
 +		const char *arg = opts->xopts[i];
 +
 +		skip_prefix(arg, "--", &arg);
 +		opts->xopts[i] = xstrdup(arg);
 +	}
 +}
 +
  static int read_populate_opts(struct replay_opts *opts)
  {
  	if (is_rebase_i(opts)) {
 @@ -1480,30 +1503,12 @@ static int read_populate_opts(struct replay_opts *opts)
  				opts->gpg_sign = xstrdup(buf.buf + 2);
  			}
  		}
 -		strbuf_release(&buf);
  
  		if (file_exists(rebase_path_verbose()))
  			opts->verbose = 1;
  
 -		if (read_oneliner(&buf, rebase_path_strategy(), 0)) {
 -			opts->strategy =
 -				sequencer_entrust(opts,
 -						  strbuf_detach(&buf, NULL));
 -			if (read_oneliner(&buf,
 -					  rebase_path_strategy_opts(), 0)) {
 -				int i;
 -				opts->xopts_nr = split_cmdline(buf.buf,
 -					&opts->xopts);
 -				for (i = 0; i < opts->xopts_nr; i++)
 -					skip_prefix(opts->xopts[i], "--",
 -						    &opts->xopts[i]);
 -				if (opts->xopts_nr)
 -					sequencer_entrust(opts,
 -						strbuf_detach(&buf, NULL));
 -				else
 -					strbuf_release(&buf);
 -			}
 -		}
 +		read_strategy_opts(opts, &buf);
 +		strbuf_release(&buf);
  
  		return 0;
  	}
 @@ -1527,7 +1532,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
  {
  	enum todo_command command = opts->action == REPLAY_PICK ?
  		TODO_PICK : TODO_REVERT;
 -	const char *command_string = todo_command_strings[command];
 +	const char *command_string = todo_command_info[command].str;
  	struct commit *commit;
  
  	if (prepare_revs(opts))
 @@ -1681,12 +1686,15 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
  		int prev_offset = !next ? 0 :
  			todo_list->items[next - 1].offset_in_buf;
  
 -		if (offset > prev_offset && write_in_full(fd,
 -				todo_list->buf.buf + prev_offset,
 -				offset - prev_offset) < 0)
 -			return error(_("Could not write to %s (%s)"),
 -				done_path, strerror(errno));
 -		close(fd);
 +		if (fd >= 0 && offset > prev_offset &&
 +		    write_in_full(fd, todo_list->buf.buf + prev_offset,
 +				  offset - prev_offset) < 0) {
 +			close(fd);
 +			return error_errno(_("could not write to '%s'"),
 +					   done_path);
 +		}
 +		if (fd >= 0)
 +			close(fd);
  	}
  	return 0;
  }
 @@ -1730,11 +1738,11 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  {
  	struct strbuf buf = STRBUF_INIT;
  	struct rev_info log_tree_opt;
 -	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject;
 +	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
  	int res = 0;
  
 -	if (write_file_gently(rebase_path_stopped_sha(),
 -			      short_commit_name(commit), 1) < 0)
 +	p = short_commit_name(commit);
 +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
  		return -1;
  
  	strbuf_addf(&buf, "%s/patch", get_dir(opts));
 @@ -1748,7 +1756,7 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
  	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
  	if (!log_tree_opt.diffopt.file)
 -		res |= error_errno("could not open '%s'", buf.buf);
 +		res |= error_errno(_("could not open '%s'"), buf.buf);
  	else {
  		res |= log_tree_commit(&log_tree_opt, commit);
  		fclose(log_tree_opt.diffopt.file);
 @@ -1758,7 +1766,7 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  	strbuf_addf(&buf, "%s/message", get_dir(opts));
  	if (!file_exists(buf.buf)) {
  		find_commit_subject(commit_buffer, &subject);
 -		res |= write_file_gently(buf.buf, subject, 1);
 +		res |= write_message(subject, strlen(subject), buf.buf, 1);
  		unuse_commit_buffer(commit, commit_buffer);
  	}
  	strbuf_release(&buf);
 @@ -1769,11 +1777,13 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  static int intend_to_amend(void)
  {
  	unsigned char head[20];
 +	char *p;
  
  	if (get_sha1("HEAD", head))
 -		return error("Cannot read HEAD");
 +		return error(_("cannot read HEAD"));
  
 -	return write_file_gently(rebase_path_amend(), sha1_to_hex(head), 1);
 +	p = sha1_to_hex(head);
 +	return write_message(p, strlen(p), rebase_path_amend(), 1);
  }
  
  static int error_with_patch(struct commit *commit,
 @@ -1806,13 +1816,13 @@ static int error_failed_squash(struct commit *commit,
  	struct replay_opts *opts, int subject_len, const char *subject)
  {
  	if (rename(rebase_path_squash_msg(), rebase_path_message()))
 -		return error("Could not rename %s to %s",
 +		return error(_("could not rename '%s' to '%s'"),
  			rebase_path_squash_msg(), rebase_path_message());
  	unlink(rebase_path_fixup_msg());
  	unlink(git_path("MERGE_MSG"));
  	if (copy_file(git_path("MERGE_MSG"), rebase_path_message(), 0666))
 -		return error("Could not copy %s to %s", rebase_path_message(),
 -			git_path("MERGE_MSG"));
 +		return error(_("could not copy '%s' to '%s'"),
 +			     rebase_path_message(), git_path("MERGE_MSG"));
  	return error_with_patch(commit, subject, subject_len, opts, 1, 0);
  }
  
 @@ -1827,30 +1837,30 @@ static int do_exec(const char *command_line)
  
  	/* force re-reading of the cache */
  	if (discard_cache() < 0 || read_cache() < 0)
 -		return error(_("Could not read index"));
 +		return error(_("could not read index"));
  
  	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
  
  	if (status) {
 -		warning("Execution failed: %s\n%s"
 -			"You can fix the problem, and then run\n"
 -			"\n"
 -			"  git rebase --continue\n"
 -			"\n",
 +		warning(_("execution failed: %s\n%s"
 +			  "You can fix the problem, and then run\n"
 +			  "\n"
 +			  "  git rebase --continue\n"
 +			  "\n"),
  			command_line,
 -			dirty ? "and made changes to the index and/or the "
 -				"working tree\n" : "");
 +			dirty ? N_("and made changes to the index and/or the "
 +				"working tree\n") : "");
  		if (status == 127)
  			/* command not found */
  			status = 1;
  	}
  	else if (dirty) {
 -		warning("Execution succeeded: %s\nbut "
 -			"left changes to the index and/or the working tree\n"
 -			"Commit or stash your changes, and then run\n"
 -			"\n"
 -			"  git rebase --continue\n"
 -			"\n", command_line);
 +		warning(_("execution succeeded: %s\nbut "
 +			  "left changes to the index and/or the working tree\n"
 +			  "Commit or stash your changes, and then run\n"
 +			  "\n"
 +			  "  git rebase --continue\n"
 +			  "\n"), command_line);
  		status = 1;
  	}
  
 @@ -1912,7 +1922,7 @@ static int apply_autostash(struct replay_opts *opts)
  		argv_array_push(&store.args, "-q");
  		argv_array_push(&store.args, stash_sha1.buf);
  		if (run_command(&store))
 -			ret = error(_("Cannot store %s"), stash_sha1.buf);
 +			ret = error(_("cannot store %s"), stash_sha1.buf);
  		else
  			printf(_("Applying autostash resulted in conflicts.\n"
  				"Your changes are safe in the stash.\n"
 @@ -1995,7 +2005,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			if (item->command == TODO_EDIT) {
  				struct commit *commit = item->commit;
  				if (!res)
 -					warning("Stopped at %s... %.*s",
 +					warning(_("stopped at %s... %.*s"),
  						short_commit_name(commit),
  						item->arg_len, item->arg);
  				return error_with_patch(commit,
 @@ -2025,7 +2035,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			*end_of_arg = saved;
  		}
  		else if (item->command < TODO_NOOP)
 -			return error("Unknown command %d", item->command);
 +			return error(_("unknown command %d"), item->command);
  
  		todo_list->current++;
  		if (res)
 @@ -2046,22 +2056,22 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			unsigned char head[20], orig[20];
  
  			if (get_sha1("HEAD", head))
 -				return error("Cannot read HEAD");
 +				return error(_("cannot read HEAD"));
  			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
  					get_sha1_hex(buf.buf, orig))
 -				return error("Could not read orig-head");
 +				return error(_("could not read orig-head"));
  			if (!read_oneliner(&buf, rebase_path_onto(), 0))
 -				return error("Could not read 'onto'");
 +				return error(_("could not read 'onto'"));
  			msg = reflog_message(opts, "finish", "%s onto %s",
  				head_ref.buf, buf.buf);
  			if (update_ref(msg, head_ref.buf, head, orig,
  					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 -				return error("Could not update %s",
 +				return error(_("could not update %s"),
  					head_ref.buf);
  			msg = reflog_message(opts, "finish", "returning to %s",
  				head_ref.buf);
  			if (create_symref("HEAD", head_ref.buf, msg))
 -				return error("Could not update HEAD to %s",
 +				return error(_("could not update HEAD to %s"),
  					head_ref.buf);
  			strbuf_reset(&buf);
  		}
 @@ -2072,7 +2082,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  			};
  
  			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
 -				return error("Could not read %s",
 +				return error(_("could not read '%s'"),
  					rebase_path_orig_head());
  			strbuf_addstr(&buf, "..HEAD");
  			argv[2] = buf.buf;
 @@ -2099,6 +2109,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  
  				hook.in = open(rebase_path_rewritten_list(),
  					O_RDONLY);
 +				hook.stdout_to_stderr = 1;
  				argv_array_push(&hook.args, post_rewrite_hook);
  				argv_array_push(&hook.args, "rebase");
  				/* we don't care if this hook failed */
 @@ -2136,12 +2147,12 @@ static int commit_staged_changes(struct replay_opts *opts)
  	int amend = 0;
  
  	if (has_unstaged_changes(1))
 -		return error(_("Cannot rebase: You have unstaged changes."));
 +		return error(_("cannot rebase: You have unstaged changes."));
  	if (!has_uncommitted_changes(0)) {
  		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
  
  		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
 -			return error("Could not remove CHERRY_PICK_HEAD");
 +			return error(_("could not remove CHERRY_PICK_HEAD"));
  		return 0;
  	}
  
 @@ -2150,23 +2161,24 @@ static int commit_staged_changes(struct replay_opts *opts)
  		unsigned char head[20], to_amend[20];
  
  		if (get_sha1("HEAD", head))
 -			return error("Cannot amend non-existing commit");
 +			return error(_("cannot amend non-existing commit"));
  		if (!read_oneliner(&rev, rebase_path_amend(), 0))
 -			return error("Invalid file: %s", rebase_path_amend());
 +			return error(_("invalid file: '%s'"), rebase_path_amend());
  		if (get_sha1_hex(rev.buf, to_amend))
 -			return error("Invalid contents: %s",
 +			return error(_("invalid contents: '%s'"),
  				rebase_path_amend());
  		if (hashcmp(head, to_amend))
 -			return error("\nYou have uncommitted changes in your "
 -				"working tree. Please, commit them\nfirst and "
 -				"then run 'git rebase --continue' again.");
 +			return error(_("\nYou have uncommitted changes in your "
 +				       "working tree. Please, commit them\n"
 +				       "first and then run 'git rebase "
 +				       "--continue' again."));
  
  		strbuf_release(&rev);
  		amend = 1;
  	}
  
 -	if (sequencer_commit(rebase_path_message(), opts, 1, 1, amend, 0))
 -		return error("Could not commit staged changes.");
 +	if (run_git_commit(rebase_path_message(), opts, 1, 1, amend, 0))
 +		return error(_("could not commit staged changes."));
  	unlink(rebase_path_amend());
  	return 0;
  }
 @@ -2194,23 +2206,23 @@ int sequencer_continue(struct replay_opts *opts)
  		/* Verify that the conflict has been resolved */
  		if (file_exists(git_path_cherry_pick_head()) ||
  		    file_exists(git_path_revert_head())) {
 -			int ret = continue_single_pick();
 -			if (ret)
 -				return ret;
 +			res = continue_single_pick();
 +			if (res)
 +				goto release_todo_list;
 +		}
 +		if (index_differs_from("HEAD", 0, 0)) {
 +			res = error_dirty_index(opts);
 +			goto release_todo_list;
  		}
 -		if (index_differs_from("HEAD", 0))
 -			return error_dirty_index(opts);
  		todo_list.current++;
  	}
  	else if (file_exists(rebase_path_stopped_sha())) {
  		struct strbuf buf = STRBUF_INIT;
  		struct object_id oid;
  
 -		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1)) {
 -			if (!get_sha1_committish(buf.buf, oid.hash))
 -				record_in_rewritten(&oid,
 -						peek_command(&todo_list, 0));
 -		}
 +		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1) &&
 +		    !get_sha1_committish(buf.buf, oid.hash))
 +			record_in_rewritten(&oid, peek_command(&todo_list, 0));
  		strbuf_release(&buf);
  	}
  
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:29 UTC · re: Johannes Schindelin · lore

[PATCH v2 04/34] sequencer (rebase -i): implement the 'exec' command

The 'exec' command is a little special among rebase -i's commands, as it does *not* have a SHA-1 as first parameter. Instead, everything after the `exec` command is treated as command-line to execute.

Let's reuse the arg/arg_len fields of the todo_item structure (which hold the oneline for pick/edit commands) to point to the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
Show changes to sequencer.c +59 −0
diff --git a/sequencer.c b/sequencer.c
index 68e2c84803..700b7575ed 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -17,6 +17,7 @@
 #include "argv-array.h"
 #include "quote.h"
 #include "log-tree.h"
+#include "wt-status.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -664,6 +665,8 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	/* commands that do something else than handling a single commit */
+	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP
 };
@@ -672,6 +675,7 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"exec",
 	"noop"
 };
 
@@ -971,6 +975,12 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return -1;
 	bol += padding;
 
+	if (item->command == TODO_EXEC) {
+		item->arg = bol;
+		item->arg_len = (int)(eol - bol);
+		return 0;
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -1396,6 +1406,47 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int do_exec(const char *command_line)
+{
+	const char *child_argv[] = { NULL, NULL };
+	int dirty, status;
+
+	fprintf(stderr, "Executing: %s\n", command_line);
+	child_argv[0] = command_line;
+	status = run_command_v_opt(child_argv, RUN_USING_SHELL);
+
+	/* force re-reading of the cache */
+	if (discard_cache() < 0 || read_cache() < 0)
+		return error(_("could not read index"));
+
+	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
+
+	if (status) {
+		warning(_("execution failed: %s\n%s"
+			  "You can fix the problem, and then run\n"
+			  "\n"
+			  "  git rebase --continue\n"
+			  "\n"),
+			command_line,
+			dirty ? N_("and made changes to the index and/or the "
+				"working tree\n") : "");
+		if (status == 127)
+			/* command not found */
+			status = 1;
+	}
+	else if (dirty) {
+		warning(_("execution succeeded: %s\nbut "
+			  "left changes to the index and/or the working tree\n"
+			  "Commit or stash your changes, and then run\n"
+			  "\n"
+			  "  git rebase --continue\n"
+			  "\n"), command_line);
+		status = 1;
+	}
+
+	return status;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1425,6 +1476,14 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					!res);
 			}
 		}
+		else if (item->command == TODO_EXEC) {
+			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			int saved = *end_of_arg;
+
+			*end_of_arg = '\0';
+			res = do_exec(item->arg);
+			*end_of_arg = saved;
+		}
 		else if (item->command != TODO_NOOP)
 			return error(_("unknown command %d"), item->command);
 
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 13, 2016, 21:35 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 04/34] sequencer (rebase -i): implement the 'exec' command

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 40 quoted lines
> +static int do_exec(const char *command_line)
> +{
> +	const char *child_argv[] = { NULL, NULL };
> +	int dirty, status;
> +
> +	fprintf(stderr, "Executing: %s\n", command_line);
> +	child_argv[0] = command_line;
> +	status = run_command_v_opt(child_argv, RUN_USING_SHELL);
> +
> +	/* force re-reading of the cache */
> +	if (discard_cache() < 0 || read_cache() < 0)
> +		return error(_("could not read index"));
> +
> +	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
> +
> +	if (status) {
> +		warning(_("execution failed: %s\n%s"
> +			  "You can fix the problem, and then run\n"
> +			  "\n"
> +			  "  git rebase --continue\n"
> +			  "\n"),
> +			command_line,
> +			dirty ? N_("and made changes to the index and/or the "
> +				"working tree\n") : "");
> +		if (status == 127)
> +			/* command not found */
> +			status = 1;
> +	}
> +	else if (dirty) {
> +		warning(_("execution succeeded: %s\nbut "
> +			  "left changes to the index and/or the working tree\n"
> +			  "Commit or stash your changes, and then run\n"
> +			  "\n"
> +			  "  git rebase --continue\n"
> +			  "\n"), command_line);
> +		status = 1;
> +	}
> +
> +	return status;
> +}

OK, this looks like a faithful reproduction of what the scripted version does inside do_next() helper function.

Please have "else if" on the same line as "}" that closes the "if (...) {" in the same if/else if/else cascade.

Johannes Schindelin· Dec 19, 2016, 14:06 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 04/34] sequencer (rebase -i): implement the 'exec' command

Hi Junio,
On Tue, 13 Dec 2016, Junio C Hamano wrote:
> Please have "else if" on the same line as "}" that closes the
> "if (...) {" in the same if/else if/else cascade.

Fixed. This affected quite a few more patches than just this one, and added two more to this already large patch series.

Ciao, Dscho

Johannes Schindelin· Dec 13, 2016, 15:29 UTC · re: Johannes Schindelin · lore

[PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

This patch is a straight-forward reimplementation of the `edit` operation of the interactive rebase command.

Well, not *quite* straight-forward: when stopping, the `edit` command wants to write the `patch` file (which is not only the patch, but includes the commit message and author information). To that end, this patch requires the earlier work that taught the log-tree machinery to respect the `file` setting of rev_info->diffopt to write to a file stream different than stdout.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 114 insertions(+), 2 deletions(-)
Show changes to sequencer.c +114 −2
diff --git a/sequencer.c b/sequencer.c
index 1224799286..68e2c84803 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -16,6 +16,7 @@
 #include "refs.h"
 #include "argv-array.h"
 #include "quote.h"
+#include "log-tree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -43,6 +44,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
 /*
+ * When an "edit" rebase command is being processed, the SHA1 of the
+ * commit to be edited is recorded in this file.  When "git rebase
+ * --continue" is executed, if there are any staged changes then they
+ * will be amended to the HEAD commit, but only provided the HEAD
+ * commit is still the commit to be edited.  When any other rebase
+ * command is processed, this file is deleted.
+ */
+static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
+/*
+ * When we stop at a given patch via the "edit" command, this file contains
+ * the long commit name of the corresponding patch.
+ */
+static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -648,6 +663,7 @@ enum todo_command {
 	/* commands that handle commits */
 	TODO_PICK = 0,
 	TODO_REVERT,
+	TODO_EDIT,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP
 };
@@ -655,6 +671,7 @@ enum todo_command {
 static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
+	"edit",
 	"noop"
 };
 
@@ -1301,9 +1318,87 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
+static int make_patch(struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct rev_info log_tree_opt;
+	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
+	int res = 0;
+
+	p = short_commit_name(commit);
+	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
+		return -1;
+
+	strbuf_addf(&buf, "%s/patch", get_dir(opts));
+	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
+	init_revisions(&log_tree_opt, NULL);
+	log_tree_opt.abbrev = 0;
+	log_tree_opt.diff = 1;
+	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
+	log_tree_opt.disable_stdin = 1;
+	log_tree_opt.no_commit_id = 1;
+	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
+	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
+	if (!log_tree_opt.diffopt.file)
+		res |= error_errno(_("could not open '%s'"), buf.buf);
+	else {
+		res |= log_tree_commit(&log_tree_opt, commit);
+		fclose(log_tree_opt.diffopt.file);
+	}
+	strbuf_reset(&buf);
+
+	strbuf_addf(&buf, "%s/message", get_dir(opts));
+	if (!file_exists(buf.buf)) {
+		find_commit_subject(commit_buffer, &subject);
+		res |= write_message(subject, strlen(subject), buf.buf, 1);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
+	strbuf_release(&buf);
+
+	return res;
+}
+
+static int intend_to_amend(void)
+{
+	unsigned char head[20];
+	char *p;
+
+	if (get_sha1("HEAD", head))
+		return error(_("cannot read HEAD"));
+
+	p = sha1_to_hex(head);
+	return write_message(p, strlen(p), rebase_path_amend(), 1);
+}
+
+static int error_with_patch(struct commit *commit,
+	const char *subject, int subject_len,
+	struct replay_opts *opts, int exit_code, int to_amend)
+{
+	if (make_patch(commit, opts))
+		return -1;
+
+	if (to_amend) {
+		if (intend_to_amend())
+			return -1;
+
+		fprintf(stderr, "You can amend the commit now, with\n"
+			"\n"
+			"  git commit --amend %s\n"
+			"\n"
+			"Once you are satisfied with your changes, run\n"
+			"\n"
+			"  git rebase --continue\n", gpg_sign_opt_quoted(opts));
+	}
+	else if (exit_code)
+		fprintf(stderr, "Could not apply %s... %.*s\n",
+			short_commit_name(commit), subject_len, subject);
+
+	return exit_code;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	int res;
+	int res = 0;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	if (opts->allow_ff)
@@ -1316,9 +1411,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_REVERT)
+		if (item->command <= TODO_EDIT) {
 			res = do_pick_commit(item->command, item->commit,
 					opts);
+			if (item->command == TODO_EDIT) {
+				struct commit *commit = item->commit;
+				if (!res)
+					warning(_("stopped at %s... %.*s"),
+						short_commit_name(commit),
+						item->arg_len, item->arg);
+				return error_with_patch(commit,
+					item->arg, item->arg_len, opts, res,
+					!res);
+			}
+		}
 		else if (item->command != TODO_NOOP)
 			return error(_("unknown command %d"), item->command);
 
@@ -1327,6 +1433,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			return res;
 	}
 
+	if (is_rebase_i(opts)) {
+		/* Stopped in the middle, as planned? */
+		if (todo_list->current < todo_list->nr)
+			return 0;
+	}
+
 	/*
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 13, 2016, 21:30 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> @@ -43,6 +44,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
>   */
>  static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
>  /*

It is minor, but please have a blank line to separate the logical blocks. If you have "comment for thing A" before "thing A", then having a blank after that before "comment for thing B" and "thing B" that follow would help. Otherwise "thing A" immediately followed by "comment for thing B" are (mis)read together, leading to nonsense.

Show 11 quoted lines
> + * When an "edit" rebase command is being processed, the SHA1 of the
> + * commit to be edited is recorded in this file.  When "git rebase
> + * --continue" is executed, if there are any staged changes then they
> + * will be amended to the HEAD commit, but only provided the HEAD
> + * commit is still the commit to be edited.  When any other rebase
> + * command is processed, this file is deleted.
> + */
> +static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
> +/*
> + * When we stop at a given patch via the "edit" command, this file contains
> + * the long commit name of the corresponding patch.

If you abbreviate an object name to 38-hex that is still long but that is not full; if you meant full 40-hex, better spell it out as "full"---that conveys useful information to programmers (e.g. they can just use get_sha1_hex()).

But I think you are writing short_commit_name() to it? So perhaps "an abbreviated commit object name"?

Show 42 quoted lines
> @@ -1301,9 +1318,87 @@ static int save_opts(struct replay_opts *opts)
>  	return res;
>  }
>  
> +static int make_patch(struct commit *commit, struct replay_opts *opts)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	struct rev_info log_tree_opt;
> +	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
> +	int res = 0;
> +
> +	p = short_commit_name(commit);
> +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
> +		return -1;
> +
> +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
> +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
> +	init_revisions(&log_tree_opt, NULL);
> +	log_tree_opt.abbrev = 0;
> +	log_tree_opt.diff = 1;
> +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
> +	log_tree_opt.disable_stdin = 1;
> +	log_tree_opt.no_commit_id = 1;
> +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
> +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
> +	if (!log_tree_opt.diffopt.file)
> +		res |= error_errno(_("could not open '%s'"), buf.buf);
> +	else {
> +		res |= log_tree_commit(&log_tree_opt, commit);
> +		fclose(log_tree_opt.diffopt.file);
> +	}
> +	strbuf_reset(&buf);
> +	strbuf_addf(&buf, "%s/message", get_dir(opts));
> +	if (!file_exists(buf.buf)) {
> +		find_commit_subject(commit_buffer, &subject);
> +		res |= write_message(subject, strlen(subject), buf.buf, 1);
> +		unuse_commit_buffer(commit, commit_buffer);
> +	}
> +	strbuf_release(&buf);
> +
> +	return res;
> +}

OK. This seems to match what scripted make_patch does in a handful of lines. We probably should have given you a helper to reduce boilerplate that sets up log_tree_opt so that this function does not have to be this long, but that is a separate topic.

Does it matter output_format is set to FORMAT_PATCH here, though? With --no-commit-id set, I suspect there is no log message or authorship information given to the output.

As you are only interested in seeing the patch/diff in this file and the log is stored in a separate "message" file, as long as "patch" file gets the patch correctly, it is not a problem, but it just looked strange to see FORMAT_PATCH there.

Johannes Schindelin· Dec 19, 2016, 13:46 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

Hi Junio,
On Tue, 13 Dec 2016, Junio C Hamano wrote:
Show 9 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -43,6 +44,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
> >   */
> >  static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
> >  /*
> 
> It is minor, but please have a blank line to separate the logical
> blocks.

Ah, but where to draw the line? These comments in front of some of the rebase_path_* functions clarify the particular role of the corresponding paths, but all of those functions belong to the same block of rebase_path_* functions. That latter circumstance is what made me decide to *not* insert blank lines here, but only a blank line before that block (to separate from the sequencer path functions) and after that block (to separate from the rest of the code).

> If you have "comment for thing A" before "thing A", then having a blank
> after that before "comment for thing B" and "thing B" that follow would
> help.  Otherwise "thing A" immediately followed by "comment for thing B"
> are (mis)read together, leading to nonsense.

In this case, I think it is quite obvious that the comments belong to the immediately following line, and that all of the path functions are part of a bigger block.

Show 19 quoted lines
> > + * When an "edit" rebase command is being processed, the SHA1 of the
> > + * commit to be edited is recorded in this file.  When "git rebase
> > + * --continue" is executed, if there are any staged changes then they
> > + * will be amended to the HEAD commit, but only provided the HEAD
> > + * commit is still the commit to be edited.  When any other rebase
> > + * command is processed, this file is deleted.
> > + */
> > +static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
> > +/*
> > + * When we stop at a given patch via the "edit" command, this file contains
> > + * the long commit name of the corresponding patch.
> 
> If you abbreviate an object name to 38-hex that is still long but
> that is not full; if you meant full 40-hex, better spell it out as
> "full"---that conveys useful information to programmers (e.g. they
> can just use get_sha1_hex()).
> 
> But I think you are writing short_commit_name() to it?  So perhaps
> "an abbreviated commit object name"?
Fixed.
Show 56 quoted lines
> > @@ -1301,9 +1318,87 @@ static int save_opts(struct replay_opts *opts)
> >  	return res;
> >  }
> >  
> > +static int make_patch(struct commit *commit, struct replay_opts *opts)
> > +{
> > +	struct strbuf buf = STRBUF_INIT;
> > +	struct rev_info log_tree_opt;
> > +	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
> > +	int res = 0;
> > +
> > +	p = short_commit_name(commit);
> > +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
> > +		return -1;
> > +
> > +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
> > +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
> > +	init_revisions(&log_tree_opt, NULL);
> > +	log_tree_opt.abbrev = 0;
> > +	log_tree_opt.diff = 1;
> > +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
> > +	log_tree_opt.disable_stdin = 1;
> > +	log_tree_opt.no_commit_id = 1;
> > +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
> > +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
> > +	if (!log_tree_opt.diffopt.file)
> > +		res |= error_errno(_("could not open '%s'"), buf.buf);
> > +	else {
> > +		res |= log_tree_commit(&log_tree_opt, commit);
> > +		fclose(log_tree_opt.diffopt.file);
> > +	}
> > +	strbuf_reset(&buf);
> > +	strbuf_addf(&buf, "%s/message", get_dir(opts));
> > +	if (!file_exists(buf.buf)) {
> > +		find_commit_subject(commit_buffer, &subject);
> > +		res |= write_message(subject, strlen(subject), buf.buf, 1);
> > +		unuse_commit_buffer(commit, commit_buffer);
> > +	}
> > +	strbuf_release(&buf);
> > +
> > +	return res;
> > +}
> 
> OK.  This seems to match what scripted make_patch does in a handful
> of lines.  We probably should have given you a helper to reduce
> boilerplate that sets up log_tree_opt so that this function does not
> have to be this long, but that is a separate topic.
> 
> Does it matter output_format is set to FORMAT_PATCH here, though?
> With --no-commit-id set, I suspect there is no log message or
> authorship information given to the output.
> 
> As you are only interested in seeing the patch/diff in this file and
> the log is stored in a separate "message" file, as long as "patch"
> file gets the patch correctly, it is not a problem, but it just
> looked strange to see FORMAT_PATCH there.

I am indifferent as to FORMAT_PATCH. It is there simply as a direct translation of the `diff-tree -p` command in https://github.com/git/git/blob/v2.11.0/git-rebase--interactive.sh#L188

Ciao, Dscho

Junio C Hamano· Dec 19, 2016, 18:48 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 03/34] sequencer (rebase -i): implement the 'edit' command

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 36 quoted lines
>> > +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
>> > +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
>> > +	init_revisions(&log_tree_opt, NULL);
>> > +	log_tree_opt.abbrev = 0;
>> > +	log_tree_opt.diff = 1;
>> > +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
>> > +	log_tree_opt.disable_stdin = 1;
>> > +	log_tree_opt.no_commit_id = 1;
>> > +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
>> > +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
>> > +	if (!log_tree_opt.diffopt.file)
>> > +		res |= error_errno(_("could not open '%s'"), buf.buf);
>> > +	else {
>> > +		res |= log_tree_commit(&log_tree_opt, commit);
>> > +		fclose(log_tree_opt.diffopt.file);
>> > +	}
>> > +	strbuf_reset(&buf);
>> > +	strbuf_addf(&buf, "%s/message", get_dir(opts));
>> > +	if (!file_exists(buf.buf)) {
>> > +		find_commit_subject(commit_buffer, &subject);
>> > +		res |= write_message(subject, strlen(subject), buf.buf, 1);
>> > +		unuse_commit_buffer(commit, commit_buffer);
>> > +	}
>> > +	strbuf_release(&buf);
>> > +
>> > +	return res;
>> > +}
>> 
>> OK.  This seems to match what scripted make_patch does in a handful
>> of lines.  We probably should have given you a helper to reduce
>> boilerplate that sets up log_tree_opt so that this function does not
>> have to be this long, but that is a separate topic.
>> 
>> Does it matter output_format is set to FORMAT_PATCH here, though?
>> With --no-commit-id set, I suspect there is no log message or
>> authorship information given to the output.
Sorry, this was me being stupid.  

FORMAT_PATCH here does not have anythning to do with "git format-patch" (and "git log --pretty=email"). The PATCH there is as opposed to things like --stat and --raw. We want patch text that can be fed to "git apply" and it is absolutely the right thing to use here.

Johannes Schindelin· Dec 13, 2016, 15:30 UTC · re: Johannes Schindelin · lore

[PATCH v2 12/34] sequencer (rebase -i): skip some revert/cherry-pick specific code path

When a cherry-pick continues without a "todo script", the intention is simply to pick a single commit.

However, when an interactive rebase is continued without a "todo script", it means that the last command has been completed and that we now need to clean up.

This commit guards the revert/cherry-pick specific steps so that they are not executed in rebase -i mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)
Show changes to sequencer.c +14 −11
diff --git a/sequencer.c b/sequencer.c
index abffaf3b40..4ceb6f3ac5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1882,25 +1882,28 @@ int sequencer_continue(struct replay_opts *opts)
 		if (commit_staged_changes(opts))
 			return -1;
 	}
-	if (!file_exists(get_todo_path(opts)))
+	else if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
 		return -1;
 	if ((res = read_populate_todo(&todo_list, opts)))
 		goto release_todo_list;
 
-	/* Verify that the conflict has been resolved */
-	if (file_exists(git_path_cherry_pick_head()) ||
-	    file_exists(git_path_revert_head())) {
-		res = continue_single_pick();
-		if (res)
+	if (!is_rebase_i(opts)) {
+		/* Verify that the conflict has been resolved */
+		if (file_exists(git_path_cherry_pick_head()) ||
+		    file_exists(git_path_revert_head())) {
+			res = continue_single_pick();
+			if (res)
+				goto release_todo_list;
+		}
+		if (index_differs_from("HEAD", 0, 0)) {
+			res = error_dirty_index(opts);
 			goto release_todo_list;
+		}
+		todo_list.current++;
 	}
-	if (index_differs_from("HEAD", 0, 0)) {
-		res = error_dirty_index(opts);
-		goto release_todo_list;
-	}
-	todo_list.current++;
+
 	res = pick_commits(&todo_list, opts);
 release_todo_list:
 	todo_list_release(&todo_list);
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:30 UTC · re: Johannes Schindelin · lore

[PATCH v2 14/34] sequencer (rebase -i): update refs after a successful rebase

An interactive rebase operates on a detached HEAD (to keep the reflog of the original branch relatively clean), and updates the branch only at the end.

Now that the sequencer learns to perform interactive rebases, it also needs to learn the trick to update the branch before removing the directory containing the state of the interactive rebase.

We introduce a new head_ref variable in a wider scope than necessary at the moment, to allow for a later patch that prints out "Successfully rebased and updated <ref>".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)
Show changes to sequencer.c +31 −1
diff --git a/sequencer.c b/sequencer.c
index a6625e765d..a4e9b326ba 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -100,6 +100,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
+static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
+static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1793,12 +1795,39 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
-		struct strbuf buf = STRBUF_INIT;
+		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
 
+		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
+				starts_with(head_ref.buf, "refs/")) {
+			unsigned char head[20], orig[20];
+
+			if (get_sha1("HEAD", head))
+				return error(_("cannot read HEAD"));
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
+					get_sha1_hex(buf.buf, orig))
+				return error(_("could not read orig-head"));
+			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
+				head_ref.buf);
+			if (!read_oneliner(&buf, rebase_path_onto(), 0))
+				return error(_("could not read 'onto'"));
+			if (update_ref(buf.buf, head_ref.buf, head, orig,
+					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
+				return error(_("could not update %s"),
+					head_ref.buf);
+			strbuf_reset(&buf);
+			strbuf_addf(&buf,
+				"rebase -i (finish): returning to %s",
+				head_ref.buf);
+			if (create_symref("HEAD", head_ref.buf, buf.buf))
+				return error(_("could not update HEAD to %s"),
+					head_ref.buf);
+			strbuf_reset(&buf);
+		}
+
 		if (opts->verbose) {
 			const char *argv[] = {
 				"diff-tree", "--stat", NULL, NULL
@@ -1813,6 +1842,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			strbuf_reset(&buf);
 		}
 		strbuf_release(&buf);
+		strbuf_release(&head_ref);
 	}
 
 	/*
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 16, 2016, 19:19 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 14/34] sequencer (rebase -i): update refs after a successful rebase

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 65 quoted lines
> An interactive rebase operates on a detached HEAD (to keep the reflog
> of the original branch relatively clean), and updates the branch only
> at the end.
>
> Now that the sequencer learns to perform interactive rebases, it also
> needs to learn the trick to update the branch before removing the
> directory containing the state of the interactive rebase.
>
> We introduce a new head_ref variable in a wider scope than necessary at
> the moment, to allow for a later patch that prints out "Successfully
> rebased and updated <ref>".
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 32 +++++++++++++++++++++++++++++++-
>  1 file changed, 31 insertions(+), 1 deletion(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index a6625e765d..a4e9b326ba 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -100,6 +100,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
>  static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
>  static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
>  static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
> +static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
> +static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
>  
>  static inline int is_rebase_i(const struct replay_opts *opts)
>  {
> @@ -1793,12 +1795,39 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  	}
>  
>  	if (is_rebase_i(opts)) {
> -		struct strbuf buf = STRBUF_INIT;
> +		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
>  
>  		/* Stopped in the middle, as planned? */
>  		if (todo_list->current < todo_list->nr)
>  			return 0;
>  
> +		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
> +				starts_with(head_ref.buf, "refs/")) {
> +			unsigned char head[20], orig[20];
> +
> +			if (get_sha1("HEAD", head))
> +				return error(_("cannot read HEAD"));
> +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
> +					get_sha1_hex(buf.buf, orig))
> +				return error(_("could not read orig-head"));
> +			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
> +				head_ref.buf);
> +			if (!read_oneliner(&buf, rebase_path_onto(), 0))
> +				return error(_("could not read 'onto'"));
> +			if (update_ref(buf.buf, head_ref.buf, head, orig,
> +					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
> +				return error(_("could not update %s"),
> +					head_ref.buf);
> +			strbuf_reset(&buf);
> +			strbuf_addf(&buf,
> +				"rebase -i (finish): returning to %s",
> +				head_ref.buf);
> +			if (create_symref("HEAD", head_ref.buf, buf.buf))
> +				return error(_("could not update HEAD to %s"),
> +					head_ref.buf);

All of the above return error() calls leak head_ref.buf; in addition some leak buf.buf, too.

Show 14 quoted lines
> +			strbuf_reset(&buf);
> +		}
> +
>  		if (opts->verbose) {
>  			const char *argv[] = {
>  				"diff-tree", "--stat", NULL, NULL
> @@ -1813,6 +1842,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  			strbuf_reset(&buf);
>  		}
>  		strbuf_release(&buf);
> +		strbuf_release(&head_ref);
>  	}
>  
>  	/*
Johannes Schindelin· Dec 19, 2016, 17:33 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 14/34] sequencer (rebase -i): update refs after a successful rebase

Hi Junio,
On Fri, 16 Dec 2016, Junio C Hamano wrote:
Show 70 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > An interactive rebase operates on a detached HEAD (to keep the reflog
> > of the original branch relatively clean), and updates the branch only
> > at the end.
> >
> > Now that the sequencer learns to perform interactive rebases, it also
> > needs to learn the trick to update the branch before removing the
> > directory containing the state of the interactive rebase.
> >
> > We introduce a new head_ref variable in a wider scope than necessary at
> > the moment, to allow for a later patch that prints out "Successfully
> > rebased and updated <ref>".
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 32 +++++++++++++++++++++++++++++++-
> >  1 file changed, 31 insertions(+), 1 deletion(-)
> >
> > diff --git a/sequencer.c b/sequencer.c
> > index a6625e765d..a4e9b326ba 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -100,6 +100,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
> >  static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
> >  static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
> >  static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
> > +static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
> > +static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
> >  
> >  static inline int is_rebase_i(const struct replay_opts *opts)
> >  {
> > @@ -1793,12 +1795,39 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  	}
> >  
> >  	if (is_rebase_i(opts)) {
> > -		struct strbuf buf = STRBUF_INIT;
> > +		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
> >  
> >  		/* Stopped in the middle, as planned? */
> >  		if (todo_list->current < todo_list->nr)
> >  			return 0;
> >  
> > +		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
> > +				starts_with(head_ref.buf, "refs/")) {
> > +			unsigned char head[20], orig[20];
> > +
> > +			if (get_sha1("HEAD", head))
> > +				return error(_("cannot read HEAD"));
> > +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
> > +					get_sha1_hex(buf.buf, orig))
> > +				return error(_("could not read orig-head"));
> > +			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
> > +				head_ref.buf);
> > +			if (!read_oneliner(&buf, rebase_path_onto(), 0))
> > +				return error(_("could not read 'onto'"));
> > +			if (update_ref(buf.buf, head_ref.buf, head, orig,
> > +					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
> > +				return error(_("could not update %s"),
> > +					head_ref.buf);
> > +			strbuf_reset(&buf);
> > +			strbuf_addf(&buf,
> > +				"rebase -i (finish): returning to %s",
> > +				head_ref.buf);
> > +			if (create_symref("HEAD", head_ref.buf, buf.buf))
> > +				return error(_("could not update HEAD to %s"),
> > +					head_ref.buf);
> 
> All of the above return error() calls leak head_ref.buf; in addition
> some leak buf.buf, too.
Thanks, fixed.

Ciao, Dscho

Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

In the upcoming patch, we will support rebase -i's progress reporting. The progress skips comments but counts 'noop's.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
Show changes to sequencer.c +9 −6
diff --git a/sequencer.c b/sequencer.c
index 1f314b2743..63f6f25ced 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -770,7 +770,9 @@ enum todo_command {
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
-	TODO_DROP
+	TODO_DROP,
+	/* comments (not counted for reporting progress) */
+	TODO_COMMENT
 };
 
 static struct {
@@ -785,12 +787,13 @@ static struct {
 	{ 's', "squash" },
 	{ 'x', "exec" },
 	{ 0,   "noop" },
-	{ 'd', "drop" }
+	{ 'd', "drop" },
+	{ 0,   NULL }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if ((size_t)command < ARRAY_SIZE(todo_command_info))
+	if (command < TODO_COMMENT)
 		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
@@ -1237,14 +1240,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	bol += strspn(bol, " \t");
 
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
-		item->command = TODO_NOOP;
+		item->command = TODO_COMMENT;
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = eol - bol;
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+	for (i = 0; i < TODO_COMMENT; i++)
 		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
@@ -1254,7 +1257,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_info))
+	if (i >= TODO_COMMENT)
 		return -1;
 
 	if (item->command == TODO_NOOP) {
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 19, 2016, 19:04 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 27/34] sequencer (rebase -i): differentiate between comments and 'noop'

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 7 quoted lines
> In the upcoming patch, we will support rebase -i's progress
> reporting. The progress skips comments but counts 'noop's.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
Show 15 quoted lines
> diff --git a/sequencer.c b/sequencer.c
> index 1f314b2743..63f6f25ced 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -770,7 +770,9 @@ enum todo_command {
>  	TODO_EXEC,
>  	/* commands that do nothing but are counted for reporting progress */
>  	TODO_NOOP,
> -	TODO_DROP
> +	TODO_DROP,
> +	/* comments (not counted for reporting progress) */
> +	TODO_COMMENT
>  };
>  
>  static struct {

Makes sense. I would have done this immediately after introducing NOOP if I were doing this series, if only because by having the unchanging last element early in enum {} definition, we can avoid having to deal with the "last element cannot have comma", but that is not a big issue.

Show 16 quoted lines
> @@ -785,12 +787,13 @@ static struct {
>  	{ 's', "squash" },
>  	{ 'x', "exec" },
>  	{ 0,   "noop" },
> -	{ 'd', "drop" }
> +	{ 'd', "drop" },
> +	{ 0,   NULL }
>  };
>  
>  static const char *command_to_string(const enum todo_command command)
>  {
> -	if ((size_t)command < ARRAY_SIZE(todo_command_info))
> +	if (command < TODO_COMMENT)
>  		return todo_command_info[command].str;
>  	die("Unknown command: %d", command);
>  }

The same comment as "instead of comparing with TODO_NOOP, you would want is_noop()" applies to three instances of comparing with TODO_COMMENT we can see in this patch, I think.

"is_counted()" perhaps?
Johannes Schindelin· Dec 13, 2016, 15:29 UTC · re: Johannes Schindelin · lore

[PATCH v2 06/34] sequencer (rebase -i): write the 'done' file

In the interactive rebase, commands that were successfully processed are not simply discarded, but appended to the 'done' file instead. This is used e.g. to display the current state to the user in the output of `git status` or the progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
Show changes to sequencer.c +23 −0
diff --git a/sequencer.c b/sequencer.c
index 1ab50884bd..f6e20b142a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -39,6 +39,12 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  */
 static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
+ * The rebase command lines that have already been processed. A line
+ * is moved here when it is first handled, before any associated user
+ * actions.
+ */
+static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -1295,6 +1301,23 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("failed to finalize '%s'."), todo_path);
+
+	if (is_rebase_i(opts)) {
+		const char *done_path = rebase_path_done();
+		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int prev_offset = !next ? 0 :
+			todo_list->items[next - 1].offset_in_buf;
+
+		if (fd >= 0 && offset > prev_offset &&
+		    write_in_full(fd, todo_list->buf.buf + prev_offset,
+				  offset - prev_offset) < 0) {
+			close(fd);
+			return error_errno(_("could not write to '%s'"),
+					   done_path);
+		}
+		if (fd >= 0)
+			close(fd);
+	}
 	return 0;
 }
 
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 13, 2016, 21:52 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 06/34] sequencer (rebase -i): write the 'done' file

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 7 quoted lines
> In the interactive rebase, commands that were successfully processed are
> not simply discarded, but appended to the 'done' file instead. This is
> used e.g. to display the current state to the user in the output of
> `git status` or the progress.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
Looks good.

I'd stop at this point for now, and start working on other things for the rest of the day. I might find time to come back to it later tonight.

So far, looking mostly good.
Johannes Schindelin· Dec 13, 2016, 15:29 UTC · re: Johannes Schindelin · lore

[PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

When calling `git rebase -i -v`, the user wants to see some statistics after the commits were rebased. Let's show some.

The strbuf we use to perform that task will be used for other things in subsequent commits, hence it is declared and initialized in a wider scope than strictly needed here.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 ++++++++++++++++++++++
 sequencer.h |  1 +
 2 files changed, 23 insertions(+)
Show changes to 2 files +23 −0

sequencer.c, sequencer.h

diff --git a/sequencer.c b/sequencer.c
index 700b7575ed..1ab50884bd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -63,6 +63,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  * command-line (and are only consumed, not modified, by the sequencer).
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
+static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
+static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1121,6 +1123,9 @@ static int read_populate_opts(struct replay_opts *opts)
 		}
 		strbuf_release(&buf);
 
+		if (file_exists(rebase_path_verbose()))
+			opts->verbose = 1;
+
 		return 0;
 	}
 
@@ -1493,9 +1498,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
+
+		if (opts->verbose) {
+			const char *argv[] = {
+				"diff-tree", "--stat", NULL, NULL
+			};
+
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
+				return error(_("could not read '%s'"),
+					rebase_path_orig_head());
+			strbuf_addstr(&buf, "..HEAD");
+			argv[2] = buf.buf;
+			run_command_v_opt(argv, RUN_GIT_CMD);
+			strbuf_reset(&buf);
+		}
+		strbuf_release(&buf);
 	}
 
 	/*
diff --git a/sequencer.h b/sequencer.h
index cb21cfddee..f885b68395 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -24,6 +24,7 @@ struct replay_opts {
 	int allow_empty;
 	int allow_empty_message;
 	int keep_redundant_commits;
+	int verbose;
 
 	int mainline;
 
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 13, 2016, 21:51 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 25 quoted lines
> @@ -1493,9 +1498,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  	}
>  
>  	if (is_rebase_i(opts)) {
> +		struct strbuf buf = STRBUF_INIT;
> +
>  		/* Stopped in the middle, as planned? */
>  		if (todo_list->current < todo_list->nr)
>  			return 0;
> +
> +		if (opts->verbose) {
> +			const char *argv[] = {
> +				"diff-tree", "--stat", NULL, NULL
> +			};
> +
> +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
> +				return error(_("could not read '%s'"),
> +					rebase_path_orig_head());
> +			strbuf_addstr(&buf, "..HEAD");
> +			argv[2] = buf.buf;
> +			run_command_v_opt(argv, RUN_GIT_CMD);
> +			strbuf_reset(&buf);
> +		}
> +		strbuf_release(&buf);
>  	}

It's a bit curious that the previous step avoided running a separate process and instead did "diff-tree -p" all in C, but this one does not.

I think it is because this one is outside the loop? The original, being a scripted Porcelain, formulates a lazy and loose command line, but you may want to tighten it up a bit if you spawn a process. If your user happens to have a file whose name is $orig_head..HEAD, the command line you are creating (which is identical to the scripted version) will barf with "ambiguous argument".

One good thing about a complete C rewrite is that it won't have an issue like this one because you'd be working with in-core objects.

Show 11 quoted lines
> diff --git a/sequencer.h b/sequencer.h
> index cb21cfddee..f885b68395 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -24,6 +24,7 @@ struct replay_opts {
>  	int allow_empty;
>  	int allow_empty_message;
>  	int keep_redundant_commits;
> +	int verbose;
>  
>  	int mainline;
Junio C Hamano· Dec 14, 2016, 06:59 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

Junio C Hamano <gitster@pobox.com> writes:
Show 17 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> ...
>> +		if (opts->verbose) {
>> +			const char *argv[] = {
>> +				"diff-tree", "--stat", NULL, NULL
>> +			};
>> + ...
>> +			run_command_v_opt(argv, RUN_GIT_CMD);
>> +			strbuf_reset(&buf);
>> +		}
>> +		strbuf_release(&buf);
>>  	}
>
> It's a bit curious that the previous step avoided running a separate
> process and instead did "diff-tree -p" all in C, but this one does not.
>
> I think it is because this one is outside the loop?

Nah, this guess of mine "The patch file generation done in 03/34 avoids spawn because it is in a loop" is off the mark. It is done before "edit" gives control back to the end user and it is not like we write one patch file per iteration of the loop we want to get maximum speed out of.

Johannes Schindelin· Jan 2, 2017, 15:11 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

Hi Junio,
On Tue, 13 Dec 2016, Junio C Hamano wrote:
Show 30 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -1493,9 +1498,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  	}
> >  
> >  	if (is_rebase_i(opts)) {
> > +		struct strbuf buf = STRBUF_INIT;
> > +
> >  		/* Stopped in the middle, as planned? */
> >  		if (todo_list->current < todo_list->nr)
> >  			return 0;
> > +
> > +		if (opts->verbose) {
> > +			const char *argv[] = {
> > +				"diff-tree", "--stat", NULL, NULL
> > +			};
> > +
> > +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
> > +				return error(_("could not read '%s'"),
> > +					rebase_path_orig_head());
> > +			strbuf_addstr(&buf, "..HEAD");
> > +			argv[2] = buf.buf;
> > +			run_command_v_opt(argv, RUN_GIT_CMD);
> > +			strbuf_reset(&buf);
> > +		}
> > +		strbuf_release(&buf);
> >  	}
> 
> It's a bit curious that the previous step avoided running a separate
> process and instead did "diff-tree -p" all in C, but this one does not.
I guess my only defence is that I tried to be a little lazy.

Fixed. Dscho

Junio C Hamano· Jan 7, 2017, 21:48 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 36 quoted lines
> Hi Junio,
>
> On Tue, 13 Dec 2016, Junio C Hamano wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> > @@ -1493,9 +1498,26 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>> >  	}
>> >  
>> >  	if (is_rebase_i(opts)) {
>> > +		struct strbuf buf = STRBUF_INIT;
>> > +
>> >  		/* Stopped in the middle, as planned? */
>> >  		if (todo_list->current < todo_list->nr)
>> >  			return 0;
>> > +
>> > +		if (opts->verbose) {
>> > +			const char *argv[] = {
>> > +				"diff-tree", "--stat", NULL, NULL
>> > +			};
>> > +
>> > +			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
>> > +				return error(_("could not read '%s'"),
>> > +					rebase_path_orig_head());
>> > +			strbuf_addstr(&buf, "..HEAD");
>> > +			argv[2] = buf.buf;
>> > +			run_command_v_opt(argv, RUN_GIT_CMD);
>> > +			strbuf_reset(&buf);
>> > +		}
>> > +		strbuf_release(&buf);
>> >  	}
>> 
>> It's a bit curious that the previous step avoided running a separate
>> process and instead did "diff-tree -p" all in C, but this one does not.
>
> I guess my only defence is that I tried to be a little lazy.

I actually was alluding to going the other way around, spawning "diff-tree -p" in the other codepath like this one does.

Junio C Hamano· Jan 8, 2017, 01:57 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 05/34] sequencer (rebase -i): learn about the 'verbose' mode

Junio C Hamano <gitster@pobox.com> writes:
Show 8 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> ...
>>
>> I guess my only defence is that I tried to be a little lazy.
>
> I actually was alluding to going the other way around, spawning
> "diff-tree -p" in the other codepath like this one does.

Pre-emptively, because I expect it will take a while for me to clear the backlog to get to your reroll already on the mailing list, I do not mean to say that you must rewrite the other one to spawn to match this one. I meant that I wouldn't have minded if the series were done that way, as this kind of being "little lazy" would reduce the chance of regression by reducing the number of exchanged parts and it is not necessarily a bad thing.

Johannes Schindelin· Dec 13, 2016, 15:30 UTC · re: Johannes Schindelin · lore

[PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

This is a huge patch, and at the same time a huge step forward to execute the performance-critical parts of the interactive rebase in a builtin command.

Since 'fixup' and 'squash' are not only similar, but also need to know about each other (we want to reduce a series of fixups/squashes into a single, final commit message edit, from the user's point of view), we really have to implement them both at the same time.

Most of the actual work is done by the existing code path that already handles the "pick" and the "edit" commands; We added support for other features (e.g. to amend the commit message) in the patches leading up to this one, yet there are still quite a few bits in this patch that simply would not make sense as individual patches (such as: determining whether there was anything to "fix up" in the "todo" script, etc).

In theory, it would be possible to reuse the fast-forward code path also for the fixup and the squash code paths, but in practice this would make the code less readable. The end result cannot be fast-forwarded anyway, therefore let's just extend the cherry-picking code path for now.

Since the sequencer parses the entire `git-rebase-todo` script in one go, fixup or squash commands without a preceding pick can be reported early (in git-rebase--interactive, we could only report such errors just before executing the fixup/squash).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 232 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 222 insertions(+), 10 deletions(-)
Show changes to sequencer.c +222 −10
diff --git a/sequencer.c b/sequencer.c
index f6e20b142a..271c21581d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -45,6 +45,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The commit message that is planned to be used for any changes that
+ * need to be committed following a user interaction.
+ */
+static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
+/*
+ * The file into which is accumulated the suggested commit message for
+ * squash/fixup commands. When the first of a series of squash/fixups
+ * is seen, the file is created and the commit message from the
+ * previous commit and from the first squash/fixup commit are written
+ * to it. The commit message for each subsequent squash/fixup commit
+ * is appended to the file as it is processed.
+ *
+ * The first line of the file is of the form
+ *     # This is a combination of $count commits.
+ * where $count is the number of commits whose messages have been
+ * written to the file so far (including the initial "pick" commit).
+ * Each time that a commit message is processed, this line is read and
+ * updated. It is deleted just before the combined commit is made.
+ */
+static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
+/*
+ * If the current series of squash/fixups has not yet included a squash
+ * command, then this file exists and holds the commit message of the
+ * original "pick" commit.  (If the series ends without a "squash"
+ * command, then this can be used as the commit message of the combined
+ * commit without opening the editor.)
+ */
+static GIT_PATH_FUNC(rebase_path_fixup_msg, "rebase-merge/message-fixup")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -673,6 +702,8 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_FIXUP,
+	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
@@ -683,6 +714,8 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"fixup",
+	"squash",
 	"exec",
 	"noop"
 };
@@ -694,16 +727,119 @@ static const char *command_to_string(const enum todo_command command)
 	die("Unknown command: %d", command);
 }
 
+static int is_fixup(enum todo_command command)
+{
+	return command == TODO_FIXUP || command == TODO_SQUASH;
+}
+
+static int update_squash_messages(enum todo_command command,
+		struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int count, res;
+	const char *message, *body;
+
+	if (file_exists(rebase_path_squash_msg())) {
+		char *p, *p2;
+
+		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
+			return error(_("could not read '%s'"),
+				rebase_path_squash_msg());
+
+		if (buf.buf[0] != comment_line_char ||
+		    !skip_prefix(buf.buf + 1, " This is a combination of ",
+				 (const char **)&p))
+			return error(_("unexpected 1st line of squash message:"
+				       "\n\n\t%.*s"),
+				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
+				     buf.buf);
+		count = strtol(p, &p2, 10);
+
+		if (count < 1 || *p2 != ' ')
+			return error(_("invalid 1st line of squash message:\n"
+				       "\n\t%.*s"),
+				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
+				     buf.buf);
+
+		sprintf((char *)p, "%d", ++count);
+		if (!*p2)
+			*p2 = ' ';
+		else {
+			*(++p2) = 'c';
+			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
+		}
+	}
+	else {
+		unsigned char head[20];
+		struct commit *head_commit;
+		const char *head_message, *body;
+
+		if (get_sha1("HEAD", head))
+			return error(_("need a HEAD to fixup"));
+		if (!(head_commit = lookup_commit_reference(head)))
+			return error(_("could not read HEAD"));
+		if (!(head_message = get_commit_buffer(head_commit, NULL)))
+			return error(_("could not read HEAD's commit message"));
+
+		body = strstr(head_message, "\n\n");
+		if (!body)
+			body = "";
+		else
+			body = skip_blank_lines(body + 2);
+		if (write_message(body, strlen(body),
+				  rebase_path_fixup_msg(), 0))
+			return error(_("cannot write '%s'"),
+				     rebase_path_fixup_msg());
+
+		count = 2;
+		strbuf_addf(&buf, _("%c This is a combination of 2 commits.\n"
+				    "%c The first commit's message is:\n\n%s"),
+			    comment_line_char, comment_line_char, body);
+
+		unuse_commit_buffer(head_commit, head_message);
+	}
+
+	if (!(message = get_commit_buffer(commit, NULL)))
+		return error(_("could not read commit message of %s"),
+			     oid_to_hex(&commit->object.oid));
+	body = strstr(message, "\n\n");
+	if (!body)
+		body = "";
+	else
+		body = skip_blank_lines(body + 2);
+
+	if (command == TODO_SQUASH) {
+		unlink(rebase_path_fixup_msg());
+		strbuf_addf(&buf, _("\n%c This is the commit message #%d:\n"
+				    "\n%s"),
+			    comment_line_char, count, body);
+	}
+	else if (command == TODO_FIXUP) {
+		strbuf_addf(&buf, _("\n%c The commit message #%d "
+				    "will be skipped:\n\n"),
+			    comment_line_char, count);
+		strbuf_add_commented_lines(&buf, body, strlen(body));
+	}
+	else
+		return error(_("unknown command: %d"), command);
+	unuse_commit_buffer(commit, message);
+
+	res = write_message(buf.buf, buf.len, rebase_path_squash_msg(), 0);
+	strbuf_release(&buf);
+	return res;
+}
 
 static int do_pick_commit(enum todo_command command, struct commit *commit,
-		struct replay_opts *opts)
+		struct replay_opts *opts, int final_fixup)
 {
+	int edit = opts->edit, cleanup_commit_message = 0;
+	const char *msg_file = edit ? NULL : git_path_merge_msg();
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, allow;
+	int res, unborn = 0, amend = 0, allow;
 
 	if (opts->no_commit) {
 		/*
@@ -749,7 +885,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
-	if (opts->allow_ff &&
+	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
 	     (!parent && unborn)))
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
@@ -813,6 +949,28 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
+	if (is_fixup(command)) {
+		if (update_squash_messages(command, commit, opts))
+			return -1;
+		amend = 1;
+		if (!final_fixup)
+			msg_file = rebase_path_squash_msg();
+		else if (file_exists(rebase_path_fixup_msg())) {
+			cleanup_commit_message = 1;
+			msg_file = rebase_path_fixup_msg();
+		}
+		else {
+			const char *dest = git_path("SQUASH_MSG");
+			unlink(dest);
+			if (copy_file(dest, rebase_path_squash_msg(), 0666))
+				return error(_("could not rename '%s' to '%s'"),
+					     rebase_path_squash_msg(), dest);
+			unlink(git_path("MERGE_MSG"));
+			msg_file = dest;
+			edit = 1;
+		}
+	}
+
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
@@ -868,8 +1026,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit, 0, 0);
+		res = run_git_commit(msg_file, opts, allow, edit, amend,
+				     cleanup_commit_message);
+
+	if (!res && final_fixup) {
+		unlink(rebase_path_fixup_msg());
+		unlink(rebase_path_squash_msg());
+	}
 
 leave:
 	free_message(commit, &msg);
@@ -1009,7 +1172,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
-	int i, res = 0;
+	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
@@ -1024,8 +1187,16 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		if (parse_insn_line(item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = -1;
+			item->command = TODO_NOOP;
 		}
+
+		if (fixup_okay)
+			; /* do nothing */
+		else if (is_fixup(item->command))
+			return error(_("cannot '%s' without a previous commit"),
+				command_to_string(item->command));
+		else if (item->command != TODO_NOOP)
+			fixup_okay = 1;
 	}
 	if (!todo_list->nr)
 		return error(_("no commits parsed."));
@@ -1434,6 +1605,20 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int error_failed_squash(struct commit *commit,
+	struct replay_opts *opts, int subject_len, const char *subject)
+{
+	if (rename(rebase_path_squash_msg(), rebase_path_message()))
+		return error(_("could not rename '%s' to '%s'"),
+			rebase_path_squash_msg(), rebase_path_message());
+	unlink(rebase_path_fixup_msg());
+	unlink(git_path("MERGE_MSG"));
+	if (copy_file(git_path("MERGE_MSG"), rebase_path_message(), 0666))
+		return error(_("could not copy '%s' to '%s'"),
+			     rebase_path_message(), git_path("MERGE_MSG"));
+	return error_with_patch(commit, subject, subject_len, opts, 1, 0);
+}
+
 static int do_exec(const char *command_line)
 {
 	const char *child_argv[] = { NULL, NULL };
@@ -1475,6 +1660,21 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int is_final_fixup(struct todo_list *todo_list)
+{
+	int i = todo_list->current;
+
+	if (!is_fixup(todo_list->items[i].command))
+		return 0;
+
+	while (++i < todo_list->nr)
+		if (is_fixup(todo_list->items[i].command))
+			return 0;
+		else if (todo_list->items[i].command < TODO_NOOP)
+			break;
+	return 1;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1490,9 +1690,15 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_EDIT) {
+		if (is_rebase_i(opts)) {
+			unlink(rebase_path_message());
+			unlink(rebase_path_author_script());
+			unlink(rebase_path_stopped_sha());
+			unlink(rebase_path_amend());
+		}
+		if (item->command <= TODO_SQUASH) {
 			res = do_pick_commit(item->command, item->commit,
-					opts);
+					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
@@ -1503,6 +1709,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (res && is_fixup(item->command)) {
+				if (res == 1)
+					intend_to_amend();
+				return error_failed_squash(item->commit, opts,
+					item->arg_len, item->arg);
+			}
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
@@ -1597,7 +1809,7 @@ static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	return do_pick_commit(opts->action == REPLAY_PICK ?
-		TODO_PICK : TODO_REVERT, cmit, opts);
+		TODO_PICK : TODO_REVERT, cmit, opts, 0);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 15, 2016, 18:42 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 15 quoted lines
> diff --git a/sequencer.c b/sequencer.c
> index f6e20b142a..271c21581d 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -45,6 +45,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
>   */
>  static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
>  /*
> + * The commit message that is planned to be used for any changes that
> + * need to be committed following a user interaction.
> + */
> +static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
> +/*
> + * The file into which is accumulated the suggested commit message for
> + * squash/fixup commands. When the first of a series of squash/fixups

The same comment as 03/34 applies here, regarding blank line to separate logical unit.

Show 30 quoted lines
> +static int update_squash_messages(enum todo_command command,
> +		struct commit *commit, struct replay_opts *opts)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	int count, res;
> +	const char *message, *body;
> +
> +	if (file_exists(rebase_path_squash_msg())) {
> +		char *p, *p2;
> +
> +		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
> +			return error(_("could not read '%s'"),
> +				rebase_path_squash_msg());
> +
> +		if (buf.buf[0] != comment_line_char ||
> +		    !skip_prefix(buf.buf + 1, " This is a combination of ",
> +				 (const char **)&p))
> +			return error(_("unexpected 1st line of squash message:"
> +				       "\n\n\t%.*s"),
> +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
> +				     buf.buf);
> +		count = strtol(p, &p2, 10);
> +
> +		if (count < 1 || *p2 != ' ')
> +			return error(_("invalid 1st line of squash message:\n"
> +				       "\n\t%.*s"),
> +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
> +				     buf.buf);
> +
> +		sprintf((char *)p, "%d", ++count);

Do we know the area pointed at p (which is inside buf) long enough not to overflow? If the original were 9 and you incremented to get 10, you would need one extra byte.

> +		if (!*p2)
> +			*p2 = ' ';
> +		else {
> +			*(++p2) = 'c';

p2 points into buf; do we know this increment does not step beyond its end? What is the meaning of a letter 'c' here (I do not see a corresponding one in the scripted update_squash_messages)?

> +			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
> +		}
> +	}
> +	else {
Style: "} else {" (I won't repeat this, as it will become too noisy).
Show 16 quoted lines
> +		unsigned char head[20];
> +		struct commit *head_commit;
> +		const char *head_message, *body;
> +
> +		if (get_sha1("HEAD", head))
> +			return error(_("need a HEAD to fixup"));
> +		if (!(head_commit = lookup_commit_reference(head)))
> +			return error(_("could not read HEAD"));
> +		if (!(head_message = get_commit_buffer(head_commit, NULL)))
> +			return error(_("could not read HEAD's commit message"));
> +
> +		body = strstr(head_message, "\n\n");
> +		if (!body)
> +			body = "";
> +		else
> +			body = skip_blank_lines(body + 2);

I think I saw you used a helper function find_commit_subject() to do the above in an earlier patch for "edit" in this series. Would it make this part (and another one for "commit" we have after this if/else) shorter?

Show 45 quoted lines
>  static int do_pick_commit(enum todo_command command, struct commit *commit,
> -		struct replay_opts *opts)
> +		struct replay_opts *opts, int final_fixup)
>  {
> +	int edit = opts->edit, cleanup_commit_message = 0;
> +	const char *msg_file = edit ? NULL : git_path_merge_msg();
>  	unsigned char head[20];
>  	struct commit *base, *next, *parent;
>  	const char *base_label, *next_label;
>  	struct commit_message msg = { NULL, NULL, NULL, NULL };
>  	struct strbuf msgbuf = STRBUF_INIT;
> -	int res, unborn = 0, allow;
> +	int res, unborn = 0, amend = 0, allow;
>  
>  	if (opts->no_commit) {
>  		/*
> @@ -749,7 +885,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	else
>  		parent = commit->parents->item;
>  
> -	if (opts->allow_ff &&
> +	if (opts->allow_ff && !is_fixup(command) &&
>  	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
>  	     (!parent && unborn)))
>  		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
> @@ -813,6 +949,28 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		}
>  	}
>  
> +	if (is_fixup(command)) {
> +		if (update_squash_messages(command, commit, opts))
> +			return -1;
> +		amend = 1;
> +		if (!final_fixup)
> +			msg_file = rebase_path_squash_msg();
> +		else if (file_exists(rebase_path_fixup_msg())) {
> +			cleanup_commit_message = 1;
> +			msg_file = rebase_path_fixup_msg();
> +		}
> +		else {
> +			const char *dest = git_path("SQUASH_MSG");
> +			unlink(dest);
> +			if (copy_file(dest, rebase_path_squash_msg(), 0666))
> +				return error(_("could not rename '%s' to '%s'"),
> +					     rebase_path_squash_msg(), dest);

Perhaps an error from unlink(dest) before copy_file() should also result in an error return? After all, that unlink() was added to mimick the "cp" in the scripted one and copy_file() does not want to overwrite an existing destination.

> +			unlink(git_path("MERGE_MSG"));

Errors from this and other unlink() that emulates "rm -f" were unchecked in the scripted original, so not checking for errors is not a regression. I would check for an error if I were writing this, however, because I know I would forget updating these after I am done with the series.

Show 16 quoted lines
> +			msg_file = dest;
> +			edit = 1;
> +		}
> +	}
> +
>  	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
>  		res = do_recursive_merge(base, next, base_label, next_label,
>  					 head, &msgbuf, opts);
> @@ -868,8 +1026,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		goto leave;
>  	}
>  	if (!opts->no_commit)
> -		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
> -				     opts, allow, opts->edit, 0, 0);
> +		res = run_git_commit(msg_file, opts, allow, edit, amend,
> +				     cleanup_commit_message);

The introduction of msg_file variable made this call (actually, the logic to decide which file is affected) much nicer to understand.

Show 21 quoted lines
> +	if (!res && final_fixup) {
> +		unlink(rebase_path_fixup_msg());
> +		unlink(rebase_path_squash_msg());
> +	}
>  
> @@ -1475,6 +1660,21 @@ static int do_exec(const char *command_line)
>  	return status;
>  }
>  
> +static int is_final_fixup(struct todo_list *todo_list)
> +{
> +	int i = todo_list->current;
> +
> +	if (!is_fixup(todo_list->items[i].command))
> +		return 0;
> +
> +	while (++i < todo_list->nr)
> +		if (is_fixup(todo_list->items[i].command))
> +			return 0;
> +		else if (todo_list->items[i].command < TODO_NOOP)
> +			break;

What follows NOOP are comment and "drop" which is another comment in disguise, so this one is excluding all the no-op commands in various shapes, which makes sense but is clear only to a reader who bothered to go back to "enum todo_command" and checked that fact. If a check for "is it one of the no-op commands?" appears only here, a single liner comment may be sufficient (but necessary) to help readers. Otherwise a single-liner helper function (similar to is_fixup() you have) with a descriptive name would be better than a single liner comment.

Jeff King· Dec 15, 2016, 18:56 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On Thu, Dec 15, 2016 at 10:42:53AM -0800, Junio C Hamano wrote:
Show 5 quoted lines
> > +		sprintf((char *)p, "%d", ++count);
> 
> Do we know the area pointed at p (which is inside buf) long enough
> not to overflow?  If the original were 9 and you incremented to get
> 10, you would need one extra byte.

Even if it is enough, I'd ask to please use xsnprintf(). In the off chance that there's a programming error, we'd get a nice die("BUG") instead of a buffer overflow (and it makes the code base easier to audit for other overflows).

-Peff
Johannes Schindelin· Dec 19, 2016, 16:59 UTC · re: Jeff King · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Hi Peff,
On Thu, 15 Dec 2016, Jeff King wrote:
Show 12 quoted lines
> On Thu, Dec 15, 2016 at 10:42:53AM -0800, Junio C Hamano wrote:
> 
> > > +		sprintf((char *)p, "%d", ++count);
> > 
> > Do we know the area pointed at p (which is inside buf) long enough
> > not to overflow?  If the original were 9 and you incremented to get
> > 10, you would need one extra byte.
> 
> Even if it is enough, I'd ask to please use xsnprintf(). In the off
> chance that there's a programming error, we'd get a nice die("BUG")
> instead of a buffer overflow (and it makes the code base easier to audit
> for other overflows).

I ended up with more verbose, easier-to-read code that does not try to do things in-place, in favor of being slightly more wasteful with strbufs.

Ciao, Dscho

Jeff King· Dec 19, 2016, 17:19 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On Mon, Dec 19, 2016 at 05:59:06PM +0100, Johannes Schindelin wrote:
Show 13 quoted lines
> > > > +		sprintf((char *)p, "%d", ++count);
> > > 
> > > Do we know the area pointed at p (which is inside buf) long enough
> > > not to overflow?  If the original were 9 and you incremented to get
> > > 10, you would need one extra byte.
> > 
> > Even if it is enough, I'd ask to please use xsnprintf(). In the off
> > chance that there's a programming error, we'd get a nice die("BUG")
> > instead of a buffer overflow (and it makes the code base easier to audit
> > for other overflows).
> 
> I ended up with more verbose, easier-to-read code that does not try to do
> things in-place, in favor of being slightly more wasteful with strbufs.
Great. I agree that should make the whole thing way more readable.
-Peff
Johannes Schindelin· Dec 19, 2016, 16:58 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Hi Junio,
On Thu, 15 Dec 2016, Junio C Hamano wrote:
Show 20 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/sequencer.c b/sequencer.c
> > index f6e20b142a..271c21581d 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -45,6 +45,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
> >   */
> >  static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
> >  /*
> > + * The commit message that is planned to be used for any changes that
> > + * need to be committed following a user interaction.
> > + */
> > +static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
> > +/*
> > + * The file into which is accumulated the suggested commit message for
> > + * squash/fixup commands. When the first of a series of squash/fixups
> 
> The same comment as 03/34 applies here, regarding blank line to
> separate logical unit.

Same rationale here: the path functions are one big continuous block, with comments obviously applying to the immediately following line only.

Show 33 quoted lines
> > +static int update_squash_messages(enum todo_command command,
> > +		struct commit *commit, struct replay_opts *opts)
> > +{
> > +	struct strbuf buf = STRBUF_INIT;
> > +	int count, res;
> > +	const char *message, *body;
> > +
> > +	if (file_exists(rebase_path_squash_msg())) {
> > +		char *p, *p2;
> > +
> > +		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
> > +			return error(_("could not read '%s'"),
> > +				rebase_path_squash_msg());
> > +
> > +		if (buf.buf[0] != comment_line_char ||
> > +		    !skip_prefix(buf.buf + 1, " This is a combination of ",
> > +				 (const char **)&p))
> > +			return error(_("unexpected 1st line of squash message:"
> > +				       "\n\n\t%.*s"),
> > +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
> > +				     buf.buf);
> > +		count = strtol(p, &p2, 10);
> > +
> > +		if (count < 1 || *p2 != ' ')
> > +			return error(_("invalid 1st line of squash message:\n"
> > +				       "\n\t%.*s"),
> > +				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
> > +				     buf.buf);
> > +
> > +		sprintf((char *)p, "%d", ++count);
> 
> Do we know the area pointed at p (which is inside buf) long enough
> not to overflow?

Yes, we know that: p2 points to the byte after the parsed number, and said byte is a space (ASCII 0x20), as verified by the if() above.

> If the original were 9 and you incremented to get 10, you would need one
> extra byte.

Exactly. That extra byte (if needed) is 0x20, as verified above, and can be overwritten.

If that extra byte (to which p2 points) is *not* overwritten, i.e. if the new count requires the same amount of space in decimal representation as the previous count, it is now NUL, as tested here:

Show 8 quoted lines
> > +		if (!*p2)
> > +			*p2 = ' ';
> > +		else {
> > +			*(++p2) = 'c';
> 
> p2 points into buf; do we know this increment does not step beyond
> its end?  What is the meaning of a letter 'c' here (I do not see a
> corresponding one in the scripted update_squash_messages)?

This clause is entered only when the space needed by the previous count was not sufficient to hold the new count, at which point we know that not only the space after the old count was overwritten, but also the 'c' of the "commits" string.

Therefore, this clause reinstates the 'c' and inserts the space, so that everything is groovy again:

> > +			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
> > +		}

Having said that, I just realized that this code was only safe as long as the squash messages were not localized.

I changed the code to imitate more closely what the shell script does. It made the code a little more verbose, but it should work better as a consequence, and I am pretty certain you will find it easier to verify that it is correct.

> > +	}
> > +	else {
> 
> Style: "} else {" (I won't repeat this, as it will become too noisy).

It is not necessary to repeat this, either, as I took the first such comment as a strong hint to look at the entire patch series and fix it appropriately.

Show 22 quoted lines
> 
> > +		unsigned char head[20];
> > +		struct commit *head_commit;
> > +		const char *head_message, *body;
> > +
> > +		if (get_sha1("HEAD", head))
> > +			return error(_("need a HEAD to fixup"));
> > +		if (!(head_commit = lookup_commit_reference(head)))
> > +			return error(_("could not read HEAD"));
> > +		if (!(head_message = get_commit_buffer(head_commit, NULL)))
> > +			return error(_("could not read HEAD's commit message"));
> > +
> > +		body = strstr(head_message, "\n\n");
> > +		if (!body)
> > +			body = "";
> > +		else
> > +			body = skip_blank_lines(body + 2);
> 
> I think I saw you used a helper function find_commit_subject() to do
> the above in an earlier patch for "edit" in this series.  Would it
> make this part (and another one for "commit" we have after this
> if/else) shorter?

Right, and by using the helper function, I also fixed a bug handling funny commit objects that had more than one empty line separating header from the message.

Of course, there is a third location in sequencer.c (predating my patches) that uses the very same idiom. Yet another patch added to this growing patch series...

Show 48 quoted lines
> >  static int do_pick_commit(enum todo_command command, struct commit *commit,
> > -		struct replay_opts *opts)
> > +		struct replay_opts *opts, int final_fixup)
> >  {
> > +	int edit = opts->edit, cleanup_commit_message = 0;
> > +	const char *msg_file = edit ? NULL : git_path_merge_msg();
> >  	unsigned char head[20];
> >  	struct commit *base, *next, *parent;
> >  	const char *base_label, *next_label;
> >  	struct commit_message msg = { NULL, NULL, NULL, NULL };
> >  	struct strbuf msgbuf = STRBUF_INIT;
> > -	int res, unborn = 0, allow;
> > +	int res, unborn = 0, amend = 0, allow;
> >  
> >  	if (opts->no_commit) {
> >  		/*
> > @@ -749,7 +885,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
> >  	else
> >  		parent = commit->parents->item;
> >  
> > -	if (opts->allow_ff &&
> > +	if (opts->allow_ff && !is_fixup(command) &&
> >  	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
> >  	     (!parent && unborn)))
> >  		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
> > @@ -813,6 +949,28 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
> >  		}
> >  	}
> >  
> > +	if (is_fixup(command)) {
> > +		if (update_squash_messages(command, commit, opts))
> > +			return -1;
> > +		amend = 1;
> > +		if (!final_fixup)
> > +			msg_file = rebase_path_squash_msg();
> > +		else if (file_exists(rebase_path_fixup_msg())) {
> > +			cleanup_commit_message = 1;
> > +			msg_file = rebase_path_fixup_msg();
> > +		}
> > +		else {
> > +			const char *dest = git_path("SQUASH_MSG");
> > +			unlink(dest);
> > +			if (copy_file(dest, rebase_path_squash_msg(), 0666))
> > +				return error(_("could not rename '%s' to '%s'"),
> > +					     rebase_path_squash_msg(), dest);
> 
> Perhaps an error from unlink(dest) before copy_file() should also
> result in an error return?

No, because the most likely cause for that `unlink()` to fail is that the destination does not exist, and it is fine if it does not exist yet.

No worries, copy_file() will fail if we could not remove any existing file and we still error out in that case.

Show 7 quoted lines
> > +			unlink(git_path("MERGE_MSG"));
> 
> Errors from this and other unlink() that emulates "rm -f" were
> unchecked in the scripted original, so not checking for errors is
> not a regression.  I would check for an error if I were writing
> this, however, because I know I would forget updating these after I
> am done with the series.

The problem is that these files do not necessarily exist. We only unlink() them to make sure that they do not exist afterwards.

In any case, I am still more interested in a faithful translation than already starting to improve the code at this stage.

Show 26 quoted lines
> > @@ -1475,6 +1660,21 @@ static int do_exec(const char *command_line)
> >  	return status;
> >  }
> >  
> > +static int is_final_fixup(struct todo_list *todo_list)
> > +{
> > +	int i = todo_list->current;
> > +
> > +	if (!is_fixup(todo_list->items[i].command))
> > +		return 0;
> > +
> > +	while (++i < todo_list->nr)
> > +		if (is_fixup(todo_list->items[i].command))
> > +			return 0;
> > +		else if (todo_list->items[i].command < TODO_NOOP)
> > +			break;
> 
> What follows NOOP are comment and "drop" which is another comment in
> disguise, so this one is excluding all the no-op commands in various
> shapes, which makes sense but is clear only to a reader who bothered
> to go back to "enum todo_command" and checked that fact.  If a check
> for "is it one of the no-op commands?" appears only here, a single
> liner comment may be sufficient (but necessary) to help readers.
> Otherwise a single-liner helper function (similar to is_fixup() you
> have) with a descriptive name would be better than a single liner
> comment.
True. I introduced is_noop().

Ciao, Dscho

Jeff King· Dec 15, 2016, 19:03 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On Tue, Dec 13, 2016 at 04:30:01PM +0100, Johannes Schindelin wrote:
Show 11 quoted lines
> +	else {
> +		unsigned char head[20];
> +		struct commit *head_commit;
> +		const char *head_message, *body;
> +
> +		if (get_sha1("HEAD", head))
> +			return error(_("need a HEAD to fixup"));
> +		if (!(head_commit = lookup_commit_reference(head)))
> +			return error(_("could not read HEAD"));
> +		if (!(head_message = get_commit_buffer(head_commit, NULL)))
> +			return error(_("could not read HEAD's commit message"));
This get_commit_buffer() may allocate a fresh buffer...
Show 9 quoted lines
> +		body = strstr(head_message, "\n\n");
> +		if (!body)
> +			body = "";
> +		else
> +			body = skip_blank_lines(body + 2);
> +		if (write_message(body, strlen(body),
> +				  rebase_path_fixup_msg(), 0))
> +			return error(_("cannot write '%s'"),
> +				     rebase_path_fixup_msg());

...and then this return leaks the result (the other code path hits unuse_commit_buffer(), and is fine).

This leak was noticed by Coverity. It has a _ton_ of false positives across the whole project, but it sends out a mail with new ones every few days, which is usually short enough that I can process it in 30 seconds or so.

I _think_ that email just goes to me and Stefan right now. You can add yourself at:

  https://scan.coverity.com/projects/git?tab=project_settings

if you already have admin access to the project (which I think you (Dscho) do). I wonder if it would be helpful to send that output to the list.

-Peff
Stefan Beller· Dec 15, 2016, 19:07 UTC · re: Jeff King · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On Thu, Dec 15, 2016 at 11:03 AM, Jeff King <peff@peff.net> wrote:
> wonder if it would be helpful to send that output to the list.
Sure we can try.

Another project I used to run through coverity (Gerrit), shows similar characteristics w.r.t. false positives, so people complained when I was force feeding them the niceties of static analysis.

I'll just try to set it up and see how the mailing list reacts. (Not sure if you can just add emails there or if the email has to be verified or such.)

Thanks, Stefan

Jeff King· Dec 15, 2016, 19:20 UTC · re: Stefan Beller · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On Thu, Dec 15, 2016 at 11:07:34AM -0800, Stefan Beller wrote:
Show 12 quoted lines
> On Thu, Dec 15, 2016 at 11:03 AM, Jeff King <peff@peff.net> wrote:
> > wonder if it would be helpful to send that output to the list.
> 
> Sure we can try.
> 
> Another project I used to run through coverity (Gerrit), shows
> similar characteristics w.r.t. false positives, so people complained
> when I was force feeding them the niceties of static analysis.
> 
> I'll just try to set it up and see how the mailing list reacts.
> (Not sure if you can just add emails there or if the email has
> to be verified or such.)

I see you added it, but I don't see the confirmation email on the list. I wonder if it was HTML mail and vger ate it.

-Peff
Stefan Beller· Dec 15, 2016, 19:27 UTC · re: Jeff King · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

On Thu, Dec 15, 2016 at 11:20 AM, Jeff King <peff@peff.net> wrote:
Show 19 quoted lines
> On Thu, Dec 15, 2016 at 11:07:34AM -0800, Stefan Beller wrote:
>
>> On Thu, Dec 15, 2016 at 11:03 AM, Jeff King <peff@peff.net> wrote:
>> > wonder if it would be helpful to send that output to the list.
>>
>> Sure we can try.
>>
>> Another project I used to run through coverity (Gerrit), shows
>> similar characteristics w.r.t. false positives, so people complained
>> when I was force feeding them the niceties of static analysis.
>>
>> I'll just try to set it up and see how the mailing list reacts.
>> (Not sure if you can just add emails there or if the email has
>> to be verified or such.)
>
> I see you added it, but I don't see the confirmation email on the list.
> I wonder if it was HTML mail and vger ate it.
>
> -Peff

I think I'll setup a dummy gmail account to use for subscription and then I'll configure that to forward all email from coverity to the list. (the actual complaints about memleaks etc are plain text, so that forwarding then should work)

Johannes Schindelin· Dec 19, 2016, 17:14 UTC · re: Jeff King · lore

Re: [PATCH v2 07/34] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

Hi Peff,
On Thu, 15 Dec 2016, Jeff King wrote:
Show 28 quoted lines
> On Tue, Dec 13, 2016 at 04:30:01PM +0100, Johannes Schindelin wrote:
> 
> > +	else {
> > +		unsigned char head[20];
> > +		struct commit *head_commit;
> > +		const char *head_message, *body;
> > +
> > +		if (get_sha1("HEAD", head))
> > +			return error(_("need a HEAD to fixup"));
> > +		if (!(head_commit = lookup_commit_reference(head)))
> > +			return error(_("could not read HEAD"));
> > +		if (!(head_message = get_commit_buffer(head_commit, NULL)))
> > +			return error(_("could not read HEAD's commit message"));
> 
> This get_commit_buffer() may allocate a fresh buffer...
> 
> > +		body = strstr(head_message, "\n\n");
> > +		if (!body)
> > +			body = "";
> > +		else
> > +			body = skip_blank_lines(body + 2);
> > +		if (write_message(body, strlen(body),
> > +				  rebase_path_fixup_msg(), 0))
> > +			return error(_("cannot write '%s'"),
> > +				     rebase_path_fixup_msg());
> 
> ...and then this return leaks the result (the other code path hits
> unuse_commit_buffer(), and is fine).
Good point.
I found another leaked commit buffer in make_patch() and fixed it, too.
> This leak was noticed by Coverity. It has a _ton_ of false positives
> across the whole project, but it sends out a mail with new ones every
> few days, which is usually short enough that I can process it in 30
> seconds or so.

Yeah, I get these mails now, thanks to Stephan adding me in response to some issues I introduced with the builtin difftool (and hence I did not get the warnings when I introduced the problems with sequencer-i).

Ciao, Dscho

Johannes Schindelin· Dec 13, 2016, 15:30 UTC · re: Johannes Schindelin · lore

[PATCH v2 13/34] sequencer (rebase -i): the todo can be empty when continuing

When the last command of an interactive rebase fails, the user needs to resolve the problem and then continue the interactive rebase. Naturally, the todo script is empty by then. So let's not complain about that!

To that end, let's move that test out of the function that parses the todo script, and into the more high-level function read_populate_todo(). This is also necessary by now because the lower-level parse_insn_buffer() has no idea whether we are performing an interactive rebase or not.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
Show changes to sequencer.c +5 −2
diff --git a/sequencer.c b/sequencer.c
index 4ceb6f3ac5..a6625e765d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1255,8 +1255,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		else if (item->command != TODO_NOOP)
 			fixup_okay = 1;
 	}
-	if (!todo_list->nr)
-		return error(_("no commits parsed."));
+
 	return res;
 }
 
@@ -1280,6 +1279,10 @@ static int read_populate_todo(struct todo_list *todo_list,
 	if (res)
 		return error(_("unusable instruction sheet: '%s'"), todo_file);
 
+	if (!todo_list->nr &&
+	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
+		return error(_("no commits parsed."));
+
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
 			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:32 UTC · re: Johannes Schindelin · lore

[PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

This will be needed to hide the output of `git commit` when the sequencer handles an interactive rebase's script.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 run-command.c | 23 +++++++++++++++++++++++
 run-command.h |  1 +
 2 files changed, 24 insertions(+)
Show changes to 2 files +24 −0

run-command.c, run-command.h

diff --git a/run-command.c b/run-command.c
index ca905a9e80..5bb957afdd 100644
--- a/run-command.c
+++ b/run-command.c
@@ -589,6 +589,29 @@ int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const
 	cmd.clean_on_exit = opt & RUN_CLEAN_ON_EXIT ? 1 : 0;
 	cmd.dir = dir;
 	cmd.env = env;
+
+	if (opt & RUN_HIDE_STDERR_ON_SUCCESS) {
+		struct strbuf buf = STRBUF_INIT;
+		int res;
+
+		cmd.err = -1;
+		if (start_command(&cmd) < 0)
+			return -1;
+
+		if (strbuf_read(&buf, cmd.err, 0) < 0) {
+			close(cmd.err);
+			finish_command(&cmd); /* throw away exit code */
+			return -1;
+		}
+
+		close(cmd.err);
+		res = finish_command(&cmd);
+		if (res)
+			fputs(buf.buf, stderr);
+		strbuf_release(&buf);
+		return res;
+	}
+
 	return run_command(&cmd);
 }
 
diff --git a/run-command.h b/run-command.h
index dd1c78c28d..65a21ddd4e 100644
--- a/run-command.h
+++ b/run-command.h
@@ -72,6 +72,7 @@ extern int run_hook_ve(const char *const *env, const char *name, va_list args);
 #define RUN_SILENT_EXEC_FAILURE 8
 #define RUN_USING_SHELL 16
 #define RUN_CLEAN_ON_EXIT 32
+#define RUN_HIDE_STDERR_ON_SUCCESS 64
 int run_command_v_opt(const char **argv, int opt);
 
 /*
-- 
2.11.0.rc3.windows.1
Johannes Sixt· Dec 14, 2016, 08:34 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

Am 13.12.2016 um 16:32 schrieb Johannes Schindelin:
Show 42 quoted lines
> This will be needed to hide the output of `git commit` when the
> sequencer handles an interactive rebase's script.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  run-command.c | 23 +++++++++++++++++++++++
>  run-command.h |  1 +
>  2 files changed, 24 insertions(+)
> 
> diff --git a/run-command.c b/run-command.c
> index ca905a9e80..5bb957afdd 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -589,6 +589,29 @@ int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const
>  	cmd.clean_on_exit = opt & RUN_CLEAN_ON_EXIT ? 1 : 0;
>  	cmd.dir = dir;
>  	cmd.env = env;
> +
> +	if (opt & RUN_HIDE_STDERR_ON_SUCCESS) {
> +		struct strbuf buf = STRBUF_INIT;
> +		int res;
> +
> +		cmd.err = -1;
> +		if (start_command(&cmd) < 0)
> +			return -1;
> +
> +		if (strbuf_read(&buf, cmd.err, 0) < 0) {
> +			close(cmd.err);
> +			finish_command(&cmd); /* throw away exit code */
> +			return -1;
> +		}
> +
> +		close(cmd.err);
> +		res = finish_command(&cmd);
> +		if (res)
> +			fputs(buf.buf, stderr);
> +		strbuf_release(&buf);
> +		return res;
> +	}
> +
>  	return run_command(&cmd);
>  }

Clearly, this is a bolted-on feature. It's not the worst move that you did not advertise the flag in Documentation/technical/api-run-command.txt...

I wanted to see what it would look like if we make it the caller's responsibility to throw away stderr. The patch is below, as fixup of patch 29/34. The change is gross, but the end result is not that bad, though not really a delightful read, either, mostly due to the strange cleanup semantics of the start_command/finish_command combo, so... I dunno.

Show changes to sequencer.c +44 −21
diff --git a/sequencer.c b/sequencer.c
index 41be4cde16..f375880bd7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -660,15 +660,16 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int cleanup_commit_message)
 {
 	char **env = NULL;
-	struct argv_array array;
-	int opt = RUN_GIT_CMD, rc;
+	int rc;
 	const char *value;
+	struct strbuf errout = STRBUF_INIT;
+	struct child_process cmd = CHILD_PROCESS_INIT;
+
+	cmd.git_cmd = 1;
 
 	if (is_rebase_i(opts)) {
-		if (!edit) {
-			opt |= RUN_COMMAND_STDOUT_TO_STDERR;
-			opt |= RUN_HIDE_STDERR_ON_SUCCESS;
-		}
+		if (!edit)
+			cmd.stdout_to_stderr = 1;
 
 		env = read_author_script();
 		if (!env) {
@@ -679,36 +680,58 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		}
 	}
 
-	argv_array_init(&array);
-	argv_array_push(&array, "commit");
-	argv_array_push(&array, "-n");
+	argv_array_push(&cmd.args, "commit");
+	argv_array_push(&cmd.args, "-n");
 
 	if (amend)
-		argv_array_push(&array, "--amend");
+		argv_array_push(&cmd.args, "--amend");
 	if (opts->gpg_sign)
-		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
+		argv_array_pushf(&cmd.args, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
-		argv_array_push(&array, "-s");
+		argv_array_push(&cmd.args, "-s");
 	if (defmsg)
-		argv_array_pushl(&array, "-F", defmsg, NULL);
+		argv_array_pushl(&cmd.args, "-F", defmsg, NULL);
 	if (cleanup_commit_message)
-		argv_array_push(&array, "--cleanup=strip");
+		argv_array_push(&cmd.args, "--cleanup=strip");
 	if (edit)
-		argv_array_push(&array, "-e");
+		argv_array_push(&cmd.args, "-e");
 	else if (!cleanup_commit_message &&
 		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
-		argv_array_push(&array, "--cleanup=verbatim");
+		argv_array_push(&cmd.args, "--cleanup=verbatim");
 
 	if (allow_empty)
-		argv_array_push(&array, "--allow-empty");
+		argv_array_push(&cmd.args, "--allow-empty");
 
 	if (opts->allow_empty_message)
-		argv_array_push(&array, "--allow-empty-message");
+		argv_array_push(&cmd.args, "--allow-empty-message");
+
+	cmd.env = (const char *const *)env;
+
+	if (cmd.stdout_to_stderr) {
+		/* hide stderr on success */
+		cmd.err = -1;
+		rc = -1;
+		if (start_command(&cmd) < 0)
+			goto cleanup;
+
+		if (strbuf_read(&errout, cmd.err, 0) < 0) {
+			close(cmd.err);
+			finish_command(&cmd); /* throw away exit code */
+			goto cleanup;
+		}
+
+		close(cmd.err);
+		rc = finish_command(&cmd);
+		if (rc)
+			fputs(errout.buf, stderr);
+	} else {
+		rc = run_command(&cmd);
+	}
 
-	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
-			(const char *const *)env);
-	argv_array_clear(&array);
+cleanup:
+	child_process_clear(&cmd);
+	strbuf_release(&errout);
 	free(env);
 
 	return rc;
-- 
2.11.0.79.g263f27a
Jeff King· Dec 14, 2016, 12:53 UTC · re: Johannes Sixt · lore

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

On Wed, Dec 14, 2016 at 09:34:20AM +0100, Johannes Sixt wrote:
Show 6 quoted lines
> I wanted to see what it would look like if we make it the caller's
> responsibility to throw away stderr. The patch is below, as fixup
> of patch 29/34. The change is gross, but the end result is not that
> bad, though not really a delightful read, either, mostly due to the
> strange cleanup semantics of the start_command/finish_command combo,
> so... I dunno.

I don't have a strong opinion on the patches under discussion, but here are a few pointers on the run-command interface:

> +	struct child_process cmd = CHILD_PROCESS_INIT;
> [...]
>  		env = read_author_script();
>  		if (!env) {

The child_process struct comes with its own internal env array. So you can do:

  read_author_script(&cmd.env_array);
  if (!cmd.env_array.argc)
	...

and then you don't have to worry about free-ing env, as it happens automatically as part of the child cleanup (I suspect the refactoring may also reduce some of the confusing memory handling in read_author_script()).

Show 20 quoted lines
> +	if (cmd.stdout_to_stderr) {
> +		/* hide stderr on success */
> +		cmd.err = -1;
> +		rc = -1;
> +		if (start_command(&cmd) < 0)
> +			goto cleanup;
> +
> +		if (strbuf_read(&errout, cmd.err, 0) < 0) {
> +			close(cmd.err);
> +			finish_command(&cmd); /* throw away exit code */
> +			goto cleanup;
> +		}
> +
> +		close(cmd.err);
> +		rc = finish_command(&cmd);
> +		if (rc)
> +			fputs(errout.buf, stderr);
> +	} else {
> +		rc = run_command(&cmd);
> +	}

We have a helper function for capturing output, so I think you can write this as:

  if (cmd.err == -1) {
        struct strbuf errout = STRBUF_INIT;
        int rc = pipe_command(&cmd,
		              NULL, 0, /* stdin */
                              NULL, 0, /* stdout */
                              &errout, 0);
        if (rc)
                fputs(errout.buf, stderr);
        strbuf_release(&errout);
  } else
        rc = run_command(&cmd);

and drop the cleanup goto entirely (if you do the "env" thing above, you could even drop "rc" and just return directly from each branch of the conditional).

Show 7 quoted lines
> -	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
> -			(const char *const *)env);
> -	argv_array_clear(&array);
> +cleanup:
> +	child_process_clear(&cmd);
> +	strbuf_release(&errout);
>  	free(env);

Even if you do keep the goto here, I think this child_process_clear() is unnecessary. It should be done automatically either by finish_command(), or by start_command() when it returns an error.

-Peff
Jeff King· Dec 14, 2016, 13:06 UTC · re: Jeff King · lore

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

On Wed, Dec 14, 2016 at 07:53:23AM -0500, Jeff King wrote:
Show 12 quoted lines
> On Wed, Dec 14, 2016 at 09:34:20AM +0100, Johannes Sixt wrote:
> 
> > I wanted to see what it would look like if we make it the caller's
> > responsibility to throw away stderr. The patch is below, as fixup
> > of patch 29/34. The change is gross, but the end result is not that
> > bad, though not really a delightful read, either, mostly due to the
> > strange cleanup semantics of the start_command/finish_command combo,
> > so... I dunno.
> 
> I don't have a strong opinion on the patches under discussion, but here
> are a few pointers on the run-command interface:
> [...]

And here is a patch representing my suggestions, on top of yours. Not tested beyond "make test".

I think read_author_script() could be simplified even more by appending to the env array in the first loop, but I didn't want to refactor the quote handling.

---
 sequencer.c | 65 +++++++++++------------------------
 1 file changed, 20 insertions(+), 45 deletions(-)
Show changes to sequencer.c +20 −45
diff --git a/sequencer.c b/sequencer.c
index f375880bd..27a9eaf15 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -591,18 +591,17 @@ static int write_author_script(const char *message)
 }
 
 /*
- * Read the author-script file into an environment block, ready for use in
- * run_command(), that can be free()d afterwards.
+ * Read the author-script file into an environment block. Returns -1
+ * on error, 0 otherwise.
  */
-static char **read_author_script(void)
+static int read_author_script(struct argv_array *env)
 {
 	struct strbuf script = STRBUF_INIT;
 	int i, count = 0;
-	char *p, *p2, **env;
-	size_t env_size;
+	char *p, *p2;
 
 	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
-		return NULL;
+		return -1;
 
 	for (p = script.buf; *p; p++)
 		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
@@ -614,19 +613,12 @@ static char **read_author_script(void)
 			count++;
 		}
 
-	env_size = (count + 1) * sizeof(*env);
-	strbuf_grow(&script, env_size);
-	memmove(script.buf + env_size, script.buf, script.len);
-	p = script.buf + env_size;
-	env = (char **)strbuf_detach(&script, NULL);
-
 	for (i = 0; i < count; i++) {
-		env[i] = p;
+		argv_array_push(env, p);
 		p += strlen(p) + 1;
 	}
-	env[count] = NULL;
 
-	return env;
+	return 0;
 }
 
 static const char staged_changes_advice[] =
@@ -659,20 +651,18 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty, int edit, int amend,
 			  int cleanup_commit_message)
 {
-	char **env = NULL;
-	int rc;
 	const char *value;
-	struct strbuf errout = STRBUF_INIT;
 	struct child_process cmd = CHILD_PROCESS_INIT;
 
 	cmd.git_cmd = 1;
 
 	if (is_rebase_i(opts)) {
-		if (!edit)
+		if (!edit) {
 			cmd.stdout_to_stderr = 1;
+			cmd.err = -1;
+		}
 
-		env = read_author_script();
-		if (!env) {
+		if (read_author_script(&cmd.env_array)) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
 			return error(_(staged_changes_advice),
@@ -706,35 +696,20 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&cmd.args, "--allow-empty-message");
 
-	cmd.env = (const char *const *)env;
-
-	if (cmd.stdout_to_stderr) {
+	if (cmd.err == -1) {
 		/* hide stderr on success */
-		cmd.err = -1;
-		rc = -1;
-		if (start_command(&cmd) < 0)
-			goto cleanup;
-
-		if (strbuf_read(&errout, cmd.err, 0) < 0) {
-			close(cmd.err);
-			finish_command(&cmd); /* throw away exit code */
-			goto cleanup;
-		}
-
-		close(cmd.err);
-		rc = finish_command(&cmd);
+		struct strbuf errout = STRBUF_INIT;
+		int rc = pipe_command(&cmd,
+				      NULL, 0, /* stdin */
+				      NULL, 0, /* stdout */
+				      &errout, 0);
 		if (rc)
 			fputs(errout.buf, stderr);
-	} else {
-		rc = run_command(&cmd);
+		strbuf_release(&errout);
+		return rc;
 	}
 
-cleanup:
-	child_process_clear(&cmd);
-	strbuf_release(&errout);
-	free(env);
-
-	return rc;
+	return run_command(&cmd);
 }
 
 static int is_original_commit_empty(struct commit *commit)
Johannes Sixt· Dec 14, 2016, 19:17 UTC · re: Jeff King · lore

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

Am 14.12.2016 um 14:06 schrieb Jeff King:
Show 10 quoted lines
> On Wed, Dec 14, 2016 at 07:53:23AM -0500, Jeff King wrote:
>
>> On Wed, Dec 14, 2016 at 09:34:20AM +0100, Johannes Sixt wrote:
>>
>>> I wanted to see what it would look like if we make it the caller's
>>> responsibility to throw away stderr. The patch is below, as fixup
>>> of patch 29/34. The change is gross, but the end result is not that
>>> bad, though not really a delightful read, either, mostly due to the
>>> strange cleanup semantics of the start_command/finish_command combo,
>>> so... I dunno.

The cleanup semantics of start_command and finish_command are not that strange as I thought first. I just hadn't looked well enough.

>>
>> I don't have a strong opinion on the patches under discussion, but here
>> are a few pointers on the run-command interface:
>> [...]
 >
> And here is a patch representing my suggestions, on top of yours. Not
> tested beyond "make test".
Thank you, that looks way better.

If there is agreement that this approach is preferable, I think we can have patches on top of the series; they would be orthogonal and do not have to take hostage of it. (And it looks like I won't be able to follow up until later this week[end].)

-- Hannes
Johannes Schindelin· Jan 2, 2017, 14:38 UTC · re: Johannes Sixt · lore

Re: [PATCH v2 28/34] run_command_opt(): optionally hide stderr when the command succeeds

Hi Hannes,
On Wed, 14 Dec 2016, Johannes Sixt wrote:
Show 16 quoted lines
> Am 14.12.2016 um 14:06 schrieb Jeff King:
> > On Wed, Dec 14, 2016 at 07:53:23AM -0500, Jeff King wrote:
> >
> > > I don't have a strong opinion on the patches under discussion, but
> > > here are a few pointers on the run-command interface:
> > > [...]
> >
> > And here is a patch representing my suggestions, on top of yours. Not
> > tested beyond "make test".
> 
> Thank you, that looks way better.
> 
> If there is agreement that this approach is preferable, I think we can
> have patches on top of the series; they would be orthogonal and do not
> have to take hostage of it. (And it looks like I won't be able to follow
> up until later this week[end].)

Seeing as the original intention was to do away with the RUN_HIDE_STDERR_ON_SUCCESS flag, and that the sequencer-i branch *must* include that functionality somehow, it is unfortunately not really possible to do this on top of the patch series.

I say "unfortunately" because I feel pretty uncomfortable with replacing something that has been tried and tested by something that still awaits the test of time.

So the only possible course of action I see is to go the really long route: incorporate the patches to use pipe_command() instead of introducing a new RUN_* flag (which means basically munch up your patch and Peff's and move it somewhere into the middle of the sequencer-i patch series, which is exactly what I already did locally), cook the patches beyond recognition in `next`, i.e. cook it really long to give it a really good testing before moving the patches to `master`.

Ciao, Johannes

Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 26/34] sequencer (rebase -i): implement the 'drop' command

The parsing part of a 'drop' command is almost identical to parsing a 'pick', while the operation is the same as that of a 'noop'.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
Show changes to sequencer.c +7 −5
diff --git a/sequencer.c b/sequencer.c
index 03256b5b1d..1f314b2743 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -769,7 +769,8 @@ enum todo_command {
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP
+	TODO_NOOP,
+	TODO_DROP
 };
 
 static struct {
@@ -783,7 +784,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
-	{ 0,   "noop" }
+	{ 0,   "noop" },
+	{ 'd', "drop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -1317,7 +1319,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		else if (is_fixup(item->command))
 			return error(_("cannot '%s' without a previous commit"),
 				command_to_string(item->command));
-		else if (item->command != TODO_NOOP)
+		else if (item->command < TODO_NOOP)
 			fixup_okay = 1;
 	}
 
@@ -1827,7 +1829,7 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	int i;
 
 	for (i = todo_list->current + offset; i < todo_list->nr; i++)
-		if (todo_list->items[i].command != TODO_NOOP)
+		if (todo_list->items[i].command < TODO_NOOP)
 			return todo_list->items[i].command;
 
 	return -1;
@@ -1960,7 +1962,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			res = do_exec(item->arg);
 			*end_of_arg = saved;
 		}
-		else if (item->command != TODO_NOOP)
+		else if (item->command < TODO_NOOP)
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 15/34] sequencer (rebase -i): leave a patch upon error

When doing an interactive rebase, we want to leave a 'patch' file for further inspection by the user (even if we never tried to actually apply that patch, since we're cherry-picking instead).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)
Show changes to sequencer.c +3 −0
diff --git a/sequencer.c b/sequencer.c
index a4e9b326ba..4361fe0e94 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1777,6 +1777,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
 			}
+			else if (res && is_rebase_i(opts))
+				return res | error_with_patch(item->commit,
+					item->arg, item->arg_len, opts, res, 0);
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 16, 2016, 19:23 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 15/34] sequencer (rebase -i): leave a patch upon error

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 6 quoted lines
> When doing an interactive rebase, we want to leave a 'patch' file for
> further inspection by the user (even if we never tried to actually apply
> that patch, since we're cherry-picking instead).
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
Yup.  

The other day, I was kind of surprised to see the "patch" file produced when I tried to do "git rebase -i HEAD^^ HEAD" with the one in current Git (not yours), marked the first one "edit", and then it gave me control back. Obviously there was no conflict and I could just do "git show" if I wanted to see what the original change was, but the "patch" file was there. I personally never have looked at the "patch" file in such a situation, and I kind of feel it is wasteful, but I can see people expect to find one there whenever "rebase -i" stops and gives control back. It is good that you are preserving the behaviour.

Show 17 quoted lines
>  sequencer.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/sequencer.c b/sequencer.c
> index a4e9b326ba..4361fe0e94 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1777,6 +1777,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  				return error_failed_squash(item->commit, opts,
>  					item->arg_len, item->arg);
>  			}
> +			else if (res && is_rebase_i(opts))
> +				return res | error_with_patch(item->commit,
> +					item->arg, item->arg_len, opts, res, 0);
>  		}
>  		else if (item->command == TODO_EXEC) {
>  			char *end_of_arg = (char *)(item->arg + item->arg_len);
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 25/34] sequencer (rebase -i): allow rescheduling commands

The interactive rebase has the very special magic that a cherry-pick that exits with a status different from 0 and 1 signifies a failure to even record that a cherry-pick was started.

This can happen e.g. when a fast-forward fails because it would overwrite untracked files.

In that case, we must reschedule the command that we thought we already had at least started successfully.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++++
 1 file changed, 6 insertions(+)
Show changes to sequencer.c +6 −0
diff --git a/sequencer.c b/sequencer.c
index a67ecec961..03256b5b1d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1922,6 +1922,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
+			if (is_rebase_i(opts) && res < 0) {
+				/* Reschedule */
+				todo_list->current--;
+				if (save_todo(todo_list, opts))
+					return -1;
+			}
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 24/34] sequencer (rebase -i): respect strategy/strategy_opts settings

The sequencer already has an idea about using different merge strategies. We just piggy-back on top of that, using rebase -i's own settings, when running the sequencer in interactive rebase mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)
Show changes to sequencer.c +25 −1
diff --git a/sequencer.c b/sequencer.c
index c11eceb70b..a67ecec961 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -112,6 +112,8 @@ static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
+static GIT_PATH_FUNC(rebase_path_strategy, "rebase-merge/strategy")
+static GIT_PATH_FUNC(rebase_path_strategy_opts, "rebase-merge/strategy_opts")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1408,6 +1410,26 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
+static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
+{
+	int i;
+
+	strbuf_reset(buf);
+	if (!read_oneliner(buf, rebase_path_strategy(), 0))
+		return;
+	opts->strategy = strbuf_detach(buf, NULL);
+	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
+		return;
+
+	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
+	for (i = 0; i < opts->xopts_nr; i++) {
+		const char *arg = opts->xopts[i];
+
+		skip_prefix(arg, "--", &arg);
+		opts->xopts[i] = xstrdup(arg);
+	}
+}
+
 static int read_populate_opts(struct replay_opts *opts)
 {
 	if (is_rebase_i(opts)) {
@@ -1421,11 +1443,13 @@ static int read_populate_opts(struct replay_opts *opts)
 				opts->gpg_sign = xstrdup(buf.buf + 2);
 			}
 		}
-		strbuf_release(&buf);
 
 		if (file_exists(rebase_path_verbose()))
 			opts->verbose = 1;
 
+		read_strategy_opts(opts, &buf);
+		strbuf_release(&buf);
+
 		return 0;
 	}
 
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 19, 2016, 18:58 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 24/34] sequencer (rebase -i): respect strategy/strategy_opts settings

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 8 quoted lines
> The sequencer already has an idea about using different merge
> strategies. We just piggy-back on top of that, using rebase -i's
> own settings, when running the sequencer in interactive rebase mode.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 26 +++++++++++++++++++++++++-
>  1 file changed, 25 insertions(+), 1 deletion(-)

A handful of steps before and including this one look quite faithful port to C from the scripted one.

Looking good.
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 20/34] sequencer (rebase -i): copy commit notes at end

When rebasing commits that have commit notes attached, the interactive rebase rewrites those notes faithfully at the end. The sequencer must do this, too, if it wishes to do interactive rebase's job.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)
Show changes to sequencer.c +76 −0
diff --git a/sequencer.c b/sequencer.c
index 118696f6d3..7ab533abd9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -94,6 +94,15 @@ static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
  */
 static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 /*
+ * For the post-rewrite hook, we make a list of rewritten commits and
+ * their new sha1s.  The rewritten-pending list keeps the sha1s of
+ * commits that have been processed, but not committed yet,
+ * e.g. because they are waiting for a 'squash' command.
+ */
+static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
+static GIT_PATH_FUNC(rebase_path_rewritten_pending,
+	"rebase-merge/rewritten-pending")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -883,6 +892,44 @@ static int update_squash_messages(enum todo_command command,
 	return res;
 }
 
+static void flush_rewritten_pending(void) {
+	struct strbuf buf = STRBUF_INIT;
+	unsigned char newsha1[20];
+	FILE *out;
+
+	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
+			!get_sha1("HEAD", newsha1) &&
+			(out = fopen(rebase_path_rewritten_list(), "a"))) {
+		char *bol = buf.buf, *eol;
+
+		while (*bol) {
+			eol = strchrnul(bol, '\n');
+			fprintf(out, "%.*s %s\n", (int)(eol - bol),
+					bol, sha1_to_hex(newsha1));
+			if (!*eol)
+				break;
+			bol = eol + 1;
+		}
+		fclose(out);
+		unlink(rebase_path_rewritten_pending());
+	}
+	strbuf_release(&buf);
+}
+
+static void record_in_rewritten(struct object_id *oid,
+		enum todo_command next_command) {
+	FILE *out = fopen(rebase_path_rewritten_pending(), "a");
+
+	if (!out)
+		return;
+
+	fprintf(out, "%s\n", oid_to_hex(oid));
+	fclose(out);
+
+	if (!is_fixup(next_command))
+		flush_rewritten_pending();
+}
+
 static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct replay_opts *opts, int final_fixup)
 {
@@ -1750,6 +1797,17 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static enum todo_command peek_command(struct todo_list *todo_list, int offset)
+{
+	int i;
+
+	for (i = todo_list->current + offset; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_NOOP)
+			return todo_list->items[i].command;
+
+	return -1;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1808,6 +1866,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (is_rebase_i(opts) && !res)
+				record_in_rewritten(&item->commit->object.oid,
+					peek_command(todo_list, 1));
 			if (res && is_fixup(item->command)) {
 				if (res == 1)
 					intend_to_amend();
@@ -1837,6 +1898,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 	if (is_rebase_i(opts)) {
 		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
+		struct stat st;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
@@ -1881,6 +1943,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			run_command_v_opt(argv, RUN_GIT_CMD);
 			strbuf_reset(&buf);
 		}
+		flush_rewritten_pending();
+		if (!stat(rebase_path_rewritten_list(), &st) &&
+				st.st_size > 0) {
+			struct child_process child = CHILD_PROCESS_INIT;
+
+			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			child.git_cmd = 1;
+			argv_array_push(&child.args, "notes");
+			argv_array_push(&child.args, "copy");
+			argv_array_push(&child.args, "--for-rewrite=rebase");
+			/* we don't care if this copying failed */
+			run_command(&child);
+		}
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 16, 2016, 19:38 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 20/34] sequencer (rebase -i): copy commit notes at end

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 8 quoted lines
> +static void flush_rewritten_pending(void) {
> +	struct strbuf buf = STRBUF_INIT;
> +	unsigned char newsha1[20];
> +	FILE *out;
> +
> +	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
> +			!get_sha1("HEAD", newsha1) &&
> +			(out = fopen(rebase_path_rewritten_list(), "a"))) {
An error in fopen() here ...
Show 11 quoted lines
> + ...
> +	}
> +	strbuf_release(&buf);
> +}
> +
> +static void record_in_rewritten(struct object_id *oid,
> +		enum todo_command next_command) {
> +	FILE *out = fopen(rebase_path_rewritten_pending(), "a");
> +
> +	if (!out)
> +		return;

... and here are ignored as an insignificant error in the scripted version, and this one does the same.

Show 22 quoted lines
> +
> +	fprintf(out, "%s\n", oid_to_hex(oid));
> +	fclose(out);
> +
> +	if (!is_fixup(next_command))
> +		flush_rewritten_pending();
> +}
> +
>  static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		struct replay_opts *opts, int final_fixup)
>  {
> @@ -1750,6 +1797,17 @@ static int is_final_fixup(struct todo_list *todo_list)
>  	return 1;
>  }
>  
> +static enum todo_command peek_command(struct todo_list *todo_list, int offset)
> +{
> +	int i;
> +
> +	for (i = todo_list->current + offset; i < todo_list->nr; i++)
> +		if (todo_list->items[i].command != TODO_NOOP)
> +			return todo_list->items[i].command;

Makes me wonder, after having commented on 07/34 regarding the fact that in the end you would end up having three variants of no-op (i.e. NOOP, DROP and COMMENT), what definition of a "command" this function uses to return its result, when asked to "peek". I suspect that this will be updated in a later patch to do "< TODO_NOOP" instead? If so, then that answers one question in my comment on 07/34, i.e.

    If a check for "is it one of the no-op commands?" appears only
    here, a single liner comment may be sufficient (but necessary)
    to help readers.  Otherwise a single-liner helper function
    (similar to is_fixup() you have) with a descriptive name would
    be better than a single liner comment.

The answer is "no, it is not just there" hence the conclusion is "we want a helper with a descriptive name".

Johannes Schindelin· Jan 2, 2017, 14:51 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 20/34] sequencer (rebase -i): copy commit notes at end

Hi Junio,
On Fri, 16 Dec 2016, Junio C Hamano wrote:
Show 18 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -1750,6 +1797,17 @@ static int is_final_fixup(struct todo_list *todo_list)
> >  	return 1;
> >  }
> >  
> > +static enum todo_command peek_command(struct todo_list *todo_list, int offset)
> > +{
> > +	int i;
> > +
> > +	for (i = todo_list->current + offset; i < todo_list->nr; i++)
> > +		if (todo_list->items[i].command != TODO_NOOP)
> > +			return todo_list->items[i].command;
> 
> Makes me wonder, after having commented on 07/34 regarding the fact
> that in the end you would end up having three variants of no-op
> (i.e. NOOP, DROP and COMMENT), what definition of a "command" this
> function uses to return its result, when asked to "peek".
Well, it uses the todo_command idea of a "command"... ;-)

The only thing we do with this for now is to look whether the next command is a fixup/squash (so that the user gets to edit the commit message just once, for example, and also to record rewritten commits properly).

> I suspect that this will be updated in a later patch to do "< TODO_NOOP"
> instead?
Actually, no. I introduced a new function is_noop() and that is used now.

Ciao, Dscho

Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 23/34] sequencer (rebase -i): respect the rebase.autostash setting

Git's `rebase` command inspects the `rebase.autostash` config setting to determine whether it should stash any uncommitted changes before rebasing and re-apply them afterwards.

As we introduce more bits and pieces to let the sequencer act as interactive rebase's backend, here is the part that adds support for the autostash feature.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
Show changes to sequencer.c +43 −0
diff --git a/sequencer.c b/sequencer.c
index cafd945e83..c11eceb70b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -111,6 +111,7 @@ static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
+static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1808,6 +1809,47 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	return -1;
 }
 
+static int apply_autostash(struct replay_opts *opts)
+{
+	struct strbuf stash_sha1 = STRBUF_INIT;
+	struct child_process child = CHILD_PROCESS_INIT;
+	int ret = 0;
+
+	if (!read_oneliner(&stash_sha1, rebase_path_autostash(), 1)) {
+		strbuf_release(&stash_sha1);
+		return 0;
+	}
+	strbuf_trim(&stash_sha1);
+
+	child.git_cmd = 1;
+	argv_array_push(&child.args, "stash");
+	argv_array_push(&child.args, "apply");
+	argv_array_push(&child.args, stash_sha1.buf);
+	if (!run_command(&child))
+		printf(_("Applied autostash."));
+	else {
+		struct child_process store = CHILD_PROCESS_INIT;
+
+		store.git_cmd = 1;
+		argv_array_push(&store.args, "stash");
+		argv_array_push(&store.args, "store");
+		argv_array_push(&store.args, "-m");
+		argv_array_push(&store.args, "autostash");
+		argv_array_push(&store.args, "-q");
+		argv_array_push(&store.args, stash_sha1.buf);
+		if (run_command(&store))
+			ret = error(_("cannot store %s"), stash_sha1.buf);
+		else
+			printf(_("Applying autostash resulted in conflicts.\n"
+				"Your changes are safe in the stash.\n"
+				"You can run \"git stash pop\" or"
+				" \"git stash drop\" at any time.\n"));
+	}
+
+	strbuf_release(&stash_sha1);
+	return ret;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1970,6 +2012,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				run_command(&hook);
 			}
 		}
+		apply_autostash(opts);
 
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 22/34] sequencer (rebase -i): run the post-rewrite hook, if needed

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
Show changes to sequencer.c +14 −0
diff --git a/sequencer.c b/sequencer.c
index 0233999389..cafd945e83 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1947,6 +1947,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process child = CHILD_PROCESS_INIT;
+			const char *post_rewrite_hook =
+				find_hook("post-rewrite");
 
 			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			child.git_cmd = 1;
@@ -1955,6 +1957,18 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			argv_array_push(&child.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
 			run_command(&child);
+
+			if (post_rewrite_hook) {
+				struct child_process hook = CHILD_PROCESS_INIT;
+
+				hook.in = open(rebase_path_rewritten_list(),
+					O_RDONLY);
+				hook.stdout_to_stderr = 1;
+				argv_array_push(&hook.args, post_rewrite_hook);
+				argv_array_push(&hook.args, "rebase");
+				/* we don't care if this hook failed */
+				run_command(&hook);
+			}
 		}
 
 		strbuf_release(&buf);
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 19/34] sequencer (rebase -i): set the reflog message consistently

We already used the same reflog message as the scripted version of rebase -i when finishing. With this commit, we do that also for all the commands before that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)
Show changes to sequencer.c +4 −0
diff --git a/sequencer.c b/sequencer.c
index d20efad562..118696f6d3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1792,6 +1792,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			unlink(rebase_path_amend());
 		}
 		if (item->command <= TODO_SQUASH) {
+			if (is_rebase_i(opts))
+				setenv("GIT_REFLOG_ACTION", reflog_message(opts,
+					command_to_string(item->command), NULL),
+					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 21/34] sequencer (rebase -i): record interrupted commits in rewritten, too

When continuing after a `pick` command failed, we want that commit to show up in the rewritten-list (and its notes to be rewritten), too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++++
 1 file changed, 9 insertions(+)
Show changes to sequencer.c +9 −0
diff --git a/sequencer.c b/sequencer.c
index 7ab533abd9..0233999389 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2052,6 +2052,15 @@ int sequencer_continue(struct replay_opts *opts)
 		}
 		todo_list.current++;
 	}
+	else if (file_exists(rebase_path_stopped_sha())) {
+		struct strbuf buf = STRBUF_INIT;
+		struct object_id oid;
+
+		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1) &&
+		    !get_sha1_committish(buf.buf, oid.hash))
+			record_in_rewritten(&oid, peek_command(&todo_list, 0));
+		strbuf_release(&buf);
+	}
 
 	res = pick_commits(&todo_list, opts);
 release_todo_list:
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 18/34] sequencer (rebase -i): refactor setting the reflog message

This makes the code DRYer, with the obvious benefit that we can enhance the code further in a single place.

We can also reuse the functionality elsewhere by calling this new function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)
Show changes to sequencer.c +26 −7
diff --git a/sequencer.c b/sequencer.c
index 33fb36dcbe..d20efad562 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1750,6 +1750,26 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...)
+{
+	va_list ap;
+	static struct strbuf buf = STRBUF_INIT;
+
+	va_start(ap, fmt);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, action_name(opts));
+	if (sub_action)
+		strbuf_addf(&buf, " (%s)", sub_action);
+	if (fmt) {
+		strbuf_addstr(&buf, ": ");
+		strbuf_vaddf(&buf, fmt, ap);
+	}
+	va_end(ap);
+
+	return buf.buf;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1820,6 +1840,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
 				starts_with(head_ref.buf, "refs/")) {
+			const char *msg;
 			unsigned char head[20], orig[20];
 
 			if (get_sha1("HEAD", head))
@@ -1827,19 +1848,17 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
 					get_sha1_hex(buf.buf, orig))
 				return error(_("could not read orig-head"));
-			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
-				head_ref.buf);
 			if (!read_oneliner(&buf, rebase_path_onto(), 0))
 				return error(_("could not read 'onto'"));
-			if (update_ref(buf.buf, head_ref.buf, head, orig,
+			msg = reflog_message(opts, "finish", "%s onto %s",
+				head_ref.buf, buf.buf);
+			if (update_ref(msg, head_ref.buf, head, orig,
 					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 				return error(_("could not update %s"),
 					head_ref.buf);
-			strbuf_reset(&buf);
-			strbuf_addf(&buf,
-				"rebase -i (finish): returning to %s",
+			msg = reflog_message(opts, "finish", "returning to %s",
 				head_ref.buf);
-			if (create_symref("HEAD", head_ref.buf, buf.buf))
+			if (create_symref("HEAD", head_ref.buf, msg))
 				return error(_("could not update HEAD to %s"),
 					head_ref.buf);
 			strbuf_reset(&buf);
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 16, 2016, 19:28 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 18/34] sequencer (rebase -i): refactor setting the reflog message

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 38 quoted lines
> This makes the code DRYer, with the obvious benefit that we can enhance
> the code further in a single place.
>
> We can also reuse the functionality elsewhere by calling this new
> function.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 33 ++++++++++++++++++++++++++-------
>  1 file changed, 26 insertions(+), 7 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 33fb36dcbe..d20efad562 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1750,6 +1750,26 @@ static int is_final_fixup(struct todo_list *todo_list)
>  	return 1;
>  }
>  
> +static const char *reflog_message(struct replay_opts *opts,
> +	const char *sub_action, const char *fmt, ...)
> +{
> +	va_list ap;
> +	static struct strbuf buf = STRBUF_INIT;
> +
> +	va_start(ap, fmt);
> +	strbuf_reset(&buf);
> +	strbuf_addstr(&buf, action_name(opts));
> +	if (sub_action)
> +		strbuf_addf(&buf, " (%s)", sub_action);
> +	if (fmt) {
> +		strbuf_addstr(&buf, ": ");
> +		strbuf_vaddf(&buf, fmt, ap);
> +	}
> +	va_end(ap);
> +
> +	return buf.buf;
> +}

It is unlikely for a single caller to want to format another reflog entry after formating one but before consuming it [*1*], so this "call this, and you can use the return value until you call it the next time without worrying about leakage" is quite a reasonable pattern to employ.

[Footnote]
*1* And it is unlikely that this will run in a multi-threaded
    environment, either ;-)
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 17/34] sequencer (rebase -i): allow fast-forwarding for edit/reword

The sequencer already knew how to fast-forward instead of cherry-picking, if possible.

We want to continue to do this, of course, but in case of the 'reword' command, we will need to call `git commit` after fast-forwarding.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
Show changes to sequencer.c +17 −8
diff --git a/sequencer.c b/sequencer.c
index 5a9972fec3..33fb36dcbe 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -893,7 +893,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, amend = 0, allow;
+	int res, unborn = 0, amend = 0, allow = 0;
 
 	if (opts->no_commit) {
 		/*
@@ -939,11 +939,23 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
+	if (get_message(commit, &msg) != 0)
+		return error(_("cannot get commit message for %s"),
+			oid_to_hex(&commit->object.oid));
+
 	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
-	     (!parent && unborn)))
-		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
-
+	     (!parent && unborn))) {
+		if (is_rebase_i(opts))
+			write_author_script(msg.message);
+		res = fast_forward_to(commit->object.oid.hash, head, unborn,
+			opts);
+		if (res || command != TODO_REWORD)
+			goto leave;
+		edit = amend = 1;
+		msg_file = NULL;
+		goto fast_forward_edit;
+	}
 	if (parent && parse_commit(parent) < 0)
 		/* TRANSLATORS: The first %s will be a "todo" command like
 		   "revert" or "pick", the second %s a SHA1. */
@@ -951,10 +963,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			command_to_string(command),
 			oid_to_hex(&parent->object.oid));
 
-	if (get_message(commit, &msg) != 0)
-		return error(_("cannot get commit message for %s"),
-			oid_to_hex(&commit->object.oid));
-
 	/*
 	 * "commit" is an existing commit.  We would want to apply
 	 * the difference it introduces since its first parent "prev"
@@ -1084,6 +1092,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
+fast_forward_edit:
 		res = run_git_commit(msg_file, opts, allow, edit, amend,
 				     cleanup_commit_message);
 
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:31 UTC · re: Johannes Schindelin · lore

[PATCH v2 16/34] sequencer (rebase -i): implement the 'reword' command

This is now trivial, as all the building blocks are in place: all we need to do is to flip the "edit" switch when committing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
Show changes to sequencer.c +7 −2
diff --git a/sequencer.c b/sequencer.c
index 4361fe0e94..5a9972fec3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -751,6 +751,7 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_REWORD,
 	TODO_FIXUP,
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
@@ -766,6 +767,7 @@ static struct {
 	{ 'p', "pick" },
 	{ 0,   "revert" },
 	{ 'e', "edit" },
+	{ 'r', "reword" },
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
@@ -1001,7 +1003,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (is_fixup(command)) {
+	if (command == TODO_REWORD)
+		edit = 1;
+	else if (is_fixup(command)) {
 		if (update_squash_messages(command, commit, opts))
 			return -1;
 		amend = 1;
@@ -1779,7 +1783,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			}
 			else if (res && is_rebase_i(opts))
 				return res | error_with_patch(item->commit,
-					item->arg, item->arg_len, opts, res, 0);
+					item->arg, item->arg_len, opts, res,
+					item->command == TODO_REWORD);
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:32 UTC · re: Johannes Schindelin · lore

[PATCH v2 32/34] sequencer (rebase -i): show the progress

The interactive rebase keeps the user informed about its progress. If the sequencer wants to do the grunt work of the interactive rebase, it also needs to show that progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
Show changes to sequencer.c +32 −0
diff --git a/sequencer.c b/sequencer.c
index 498dd028d1..35d5ef4ef6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1221,6 +1221,7 @@ struct todo_list {
 	struct strbuf buf;
 	struct todo_item *items;
 	int nr, alloc, current;
+	int done_nr, total_nr;
 };
 
 #define TODO_LIST_INIT { STRBUF_INIT }
@@ -1338,6 +1339,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	return res;
 }
 
+static int count_commands(struct todo_list *todo_list)
+{
+	int count = 0, i;
+
+	for (i = 0; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_COMMENT)
+			count++;
+
+	return count;
+}
+
 static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
@@ -1380,6 +1392,21 @@ static int read_populate_todo(struct todo_list *todo_list,
 				return error(_("cannot revert during a cherry-pick."));
 	}
 
+	if (is_rebase_i(opts)) {
+		struct todo_list done = TODO_LIST_INIT;
+
+		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
+				!parse_insn_buffer(done.buf.buf, &done))
+			todo_list->done_nr = count_commands(&done);
+		else
+			todo_list->done_nr = 0;
+
+		todo_list->total_nr = todo_list->done_nr
+			+ count_commands(todo_list);
+
+		todo_list_release(&done);
+	}
+
 	return 0;
 }
 
@@ -1928,6 +1955,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
+			if (item->command != TODO_COMMENT)
+				fprintf(stderr, "Rebasing (%d/%d)%s",
+					++(todo_list->done_nr),
+					todo_list->total_nr,
+					opts->verbose ? "\n" : "\r");
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 19, 2016, 19:18 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 32/34] sequencer (rebase -i): show the progress

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 8 quoted lines
> The interactive rebase keeps the user informed about its progress.
> If the sequencer wants to do the grunt work of the interactive
> rebase, it also needs to show that progress.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 32 ++++++++++++++++++++++++++++++++
>  1 file changed, 32 insertions(+)

Sensible. This further adds two comparisons with TODO_COMMENT and makes need for is_counted() or something like that felt more.

    $ git grep TODO_COMMENT pu sequencer.c

shows that some places "x < TODO_COMMENT" is used while some other places "y != TODO_COMMENT" is used, both for the same purpose, and "z >= TODO_COMMENT" is also seen for the negation of the same.

I think all of them except for one can become is_counted() or !is_counted() to convey what they want to check better while the one used in the loop control:

	for (i = 0; i < TODO_COMMENT; i++)
should probably become:
	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)

as the parsing loop wants to check the input against all known commands and there is no strong reason for that layer to know how the insns are numbered. is_xxx() implementation can take advantage of the way the insns are numbered, but there is no point spreading the knowledge to higher layer in the callchain.

Other than that, all 34 patches looked sensible steps explained as a coherent story that unfolds bit by bit, which was mostly a pleasant read.

Thanks.
Johannes Schindelin· Dec 13, 2016, 15:32 UTC · re: Johannes Schindelin · lore

[PATCH v2 33/34] sequencer (rebase -i): write the progress into files

For the benefit of e.g. the shell prompt, the interactive rebase not only displays the progress for the user to see, but also writes it into the msgnum/end files in the state directory.

Teach the sequencer this new trick.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
Show changes to sequencer.c +27 −3
diff --git a/sequencer.c b/sequencer.c
index 35d5ef4ef6..cb5e7f35fc 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -45,6 +45,16 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The file to keep track of how many commands were already processed (e.g.
+ * for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgnum, "rebase-merge/msgnum");
+/*
+ * The file to keep track of how many commands are to be processed in total
+ * (e.g. for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgtotal, "rebase-merge/end");
+/*
  * The commit message that is planned to be used for any changes that
  * need to be committed following a user interaction.
  */
@@ -1394,6 +1404,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 	if (is_rebase_i(opts)) {
 		struct todo_list done = TODO_LIST_INIT;
+		FILE *f = fopen(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
 				!parse_insn_buffer(done.buf.buf, &done))
@@ -1403,8 +1414,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 		todo_list->total_nr = todo_list->done_nr
 			+ count_commands(todo_list);
-
 		todo_list_release(&done);
+
+		if (f) {
+			fprintf(f, "%d\n", todo_list->total_nr);
+			fclose(f);
+		}
 	}
 
 	return 0;
@@ -1955,11 +1970,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
-			if (item->command != TODO_COMMENT)
+			if (item->command != TODO_COMMENT) {
+				FILE *f = fopen(rebase_path_msgnum(), "w");
+
+				todo_list->done_nr++;
+
+				if (f) {
+					fprintf(f, "%d\n", todo_list->done_nr);
+					fclose(f);
+				}
 				fprintf(stderr, "Rebasing (%d/%d)%s",
-					++(todo_list->done_nr),
+					todo_list->done_nr,
 					todo_list->total_nr,
 					opts->verbose ? "\n" : "\r");
+			}
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:32 UTC · re: Johannes Schindelin · lore

[PATCH v2 34/34] sequencer (rebase -i): write out the final message

The shell script version of the interactive rebase has a very specific final message. Teach the sequencer to print the same.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)
Show changes to sequencer.c +3 −0
diff --git a/sequencer.c b/sequencer.c
index cb5e7f35fc..41be4cde16 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2118,6 +2118,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
+		fprintf(stderr, "Successfully rebased and updated %s.\n",
+			head_ref.buf);
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:32 UTC · re: Johannes Schindelin · lore

[PATCH v2 31/34] sequencer (rebase -i): suggest --edit-todo upon unknown command

This is the same behavior as known from `git rebase -i`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Show changes to sequencer.c +5 −1
diff --git a/sequencer.c b/sequencer.c
index edc213a2c8..498dd028d1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1355,8 +1355,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (res)
+	if (res) {
+		if (is_rebase_i(opts))
+			return error(_("please fix this using "
+				       "'git rebase --edit-todo'."));
 		return error(_("unusable instruction sheet: '%s'"), todo_file);
+	}
 
 	if (!todo_list->nr &&
 	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:29 UTC · re: Johannes Schindelin · lore

[PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

This patch introduces a new action for the sequencer. It really does not do a whole lot of its own right now, but lays the ground work for patches to come. The intention, of course, is to finally make the sequencer the work horse of the interactive rebase (the original idea behind the "sequencer" concept).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 +++++++++++++++++++++++++++++----
 sequencer.h |  3 ++-
 2 files changed, 31 insertions(+), 5 deletions(-)
Show changes to 2 files +31 −5

sequencer.c, sequencer.h

diff --git a/sequencer.c b/sequencer.c
index 30b10ba143..21cfdacd06 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -28,6 +28,14 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static GIT_PATH_FUNC(rebase_path, "rebase-merge")
+/*
+ * The file containing rebase commands, comments, and empty lines.
+ * This file is created by "git rebase -i" then edited by the user. As
+ * the lines are processed, they are removed from the front of this
+ * file and written to the tail of 'done'.
+ */
+static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
@@ -40,19 +48,22 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
-/* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
-	return 0;
+	return opts->action == REPLAY_INTERACTIVE_REBASE;
 }
 
 static const char *get_dir(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path();
 	return git_path_seq_dir();
 }
 
 static const char *get_todo_path(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path_todo();
 	return git_path_todo_file();
 }
 
@@ -168,7 +179,15 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
+	switch (opts->action) {
+	case REPLAY_REVERT:
+		return N_("revert");
+	case REPLAY_PICK:
+		return N_("cherry-pick");
+	case REPLAY_INTERACTIVE_REBASE:
+		return N_("rebase -i");
+	}
+	die(_("Unknown action: %d"), opts->action);
 }
 
 struct commit_message {
@@ -395,7 +414,10 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
-		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
+		/*
+		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
+		 * "rebase -i".
+		 */
 		return error(_("%s: Unable to write new index file"),
 			_(action_name(opts)));
 	rollback_lock_file(&index_lock);
@@ -1204,6 +1226,9 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	const char *todo_path = get_todo_path(opts);
 	int next = todo_list->current, offset, fd;
 
+	if (is_rebase_i(opts))
+		next++;
+
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("could not lock '%s'"), todo_path);
diff --git a/sequencer.h b/sequencer.h
index 7a513c576b..cb21cfddee 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -7,7 +7,8 @@ const char *git_path_seq_dir(void);
 
 enum replay_action {
 	REPLAY_REVERT,
-	REPLAY_PICK
+	REPLAY_PICK,
+	REPLAY_INTERACTIVE_REBASE
 };
 
 struct replay_opts {
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 13, 2016, 20:32 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 35 quoted lines
>  static inline int is_rebase_i(const struct replay_opts *opts)
>  {
> -	return 0;
> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
>  }
>  
>  static const char *get_dir(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path();
>  	return git_path_seq_dir();
>  }
>  
>  static const char *get_todo_path(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path_todo();
>  	return git_path_todo_file();
>  }
>  
> @@ -168,7 +179,15 @@ int sequencer_remove_state(struct replay_opts *opts)
>  
>  static const char *action_name(const struct replay_opts *opts)
>  {
> -	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
> +	switch (opts->action) {
> +	case REPLAY_REVERT:
> +		return N_("revert");
> +	case REPLAY_PICK:
> +		return N_("cherry-pick");
> +	case REPLAY_INTERACTIVE_REBASE:
> +		return N_("rebase -i");
> +	}
> +	die(_("Unknown action: %d"), opts->action);
>  }

This case statement which looks perfectly sensible---it says that there are three equal modes the subsystem operates in.

This is just a mental note and not a suggestion to change anything immediately, but it makes me wonder if git_dir/get_todo_path would also want to do so, moving towards retiring is_rebase_i() which is "everything else vs one oddball which is rebase-i" mindset.

Show 9 quoted lines
> @@ -395,7 +414,10 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
>  
>  	if (active_cache_changed &&
>  	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
> -		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
> +		/*
> +		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
> +		 * "rebase -i".
> +		 */

IIRC, the "TRANSLATORS:" comment has to deviate from our coding style due to tool limitation and has to be done like this:

> +		/* TRANSLATORS: %s will be "revert", "cherry-pick" or
> +		 * "rebase -i".
> +		 */
Show 7 quoted lines
> @@ -1204,6 +1226,9 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
>  	const char *todo_path = get_todo_path(opts);
>  	int next = todo_list->current, offset, fd;
>  
> +	if (is_rebase_i(opts))
> +		next++;
> +

This is because...? Everybody else counts 0-based while rebase-i counts from 1 or something?

>  	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
>  	if (fd < 0)
>  		return error_errno(_("could not lock '%s'"), todo_path);

Everything else in the patch is understandable. This bit isn't without explanation, at least to me.

Johannes Schindelin· Dec 19, 2016, 13:33 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

Hi Junio,
On Tue, 13 Dec 2016, Junio C Hamano wrote:
Show 18 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -395,7 +414,10 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
> >  
> >  	if (active_cache_changed &&
> >  	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
> > -		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
> > +		/*
> > +		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
> > +		 * "rebase -i".
> > +		 */
> 
> IIRC, the "TRANSLATORS:" comment has to deviate from our coding
> style due to tool limitation and has to be done like this:
> 
> > +		/* TRANSLATORS: %s will be "revert", "cherry-pick" or
> > +		 * "rebase -i".
> > +		 */

Aargh. I had fixed that in another patch series already, but failed to do so in this here patch series. Sorry.

Fixed.
Show 10 quoted lines
> > @@ -1204,6 +1226,9 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
> >  	const char *todo_path = get_todo_path(opts);
> >  	int next = todo_list->current, offset, fd;
> >  
> > +	if (is_rebase_i(opts))
> > +		next++;
> > +
> 
> This is because...?  Everybody else counts 0-based while rebase-i
> counts from 1 or something?

No. Since its conception, edit-patch-series.sh (the artist now known as rebase -i) processed each line of the "todo script" by writing a new "todo" file, skipping the first line, and appending it to the "done" file.

The sequencer chooses to write the new "insn sheet" after a successful operation.

This is not an option for rebase -i, as it has many error paths and it was simply impossible to express "save the todo script in case of failure" in shell script.

I added a comment to clarify why the current command is not written to git-rebase-todo.

Ciao, Dscho

Junio C Hamano· Dec 14, 2016, 19:29 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 13 quoted lines
> -/* We will introduce the 'interactive rebase' mode later */
>  static inline int is_rebase_i(const struct replay_opts *opts)
>  {
> -	return 0;
> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
>  }
>  
>  static const char *get_dir(const struct replay_opts *opts)
>  {
> +	if (is_rebase_i(opts))
> +		return rebase_path();
>  	return git_path_seq_dir();
>  }

This obviously makes the assumption made by 39784cd362 ("sequencer: remove useless get_dir() function", 2016-12-07) invalid, where the "remove useless" thought that the callers of this function wants a single answer that does not depend on opts.

I'll revert that commit from the sb/sequencer-abort-safety topic (as the topic is in 'next' already) to keep this one. Please holler if that is a mistaken decision.

Thanks.
Stephan Beyer· Dec 17, 2016, 20:23 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

Hi,
On 12/14/2016 08:29 PM, Junio C Hamano wrote:
Show 23 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> -/* We will introduce the 'interactive rebase' mode later */
>>  static inline int is_rebase_i(const struct replay_opts *opts)
>>  {
>> -	return 0;
>> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
>>  }
>>  
>>  static const char *get_dir(const struct replay_opts *opts)
>>  {
>> +	if (is_rebase_i(opts))
>> +		return rebase_path();
>>  	return git_path_seq_dir();
>>  }
> 
> This obviously makes the assumption made by 39784cd362 ("sequencer:
> remove useless get_dir() function", 2016-12-07) invalid, where the
> "remove useless" thought that the callers of this function wants a
> single answer that does not depend on opts.
> 
> I'll revert that commit from the sb/sequencer-abort-safety topic (as
> the topic is in 'next' already) to keep this one.  Please holler if
> that is a mistaken decision.

It seems to be the right decision if one wants to keep the internal-path backwards-compatibility of "git rebase -i" (and it seems that Dscho wants to keep it).

However, maintaining more than one directory (like "sequencer" for sequencer state and "rebase-merge" for rebase todo and log) can cause the necessity to be even more careful when hacking on sequencer... For example, the cleanup code must delete both of them, not only one of them.

Hence, I think that it's a wiser choice to keep one directory for sequencer state *and* additional files. If we (a) save everything in "sequencer", we break the internal-path backwards-compatbility but we can get rid of get_dir()... If we (b) save everything in "rebase-merge" when using rebase -i and save everything in "sequencer" for pick/revert, we are 100% backwards-compatible, but we have to change every occurrence of git_path_seq_dir() to get_dir() and the GIT_PATH_FUNC definitions of git_path_{todo,opts,head}_file must be replaced by definitions using get_dir().

Best
  Stephan
Johannes Schindelin· Dec 19, 2016, 14:25 UTC · re: Stephan Beyer · lore

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

Hi Stephan,
On Sat, 17 Dec 2016, Stephan Beyer wrote:
Show 28 quoted lines
> On 12/14/2016 08:29 PM, Junio C Hamano wrote:
> > Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> >> -/* We will introduce the 'interactive rebase' mode later */
> >>  static inline int is_rebase_i(const struct replay_opts *opts)
> >>  {
> >> -	return 0;
> >> +	return opts->action == REPLAY_INTERACTIVE_REBASE;
> >>  }
> >>  
> >>  static const char *get_dir(const struct replay_opts *opts)
> >>  {
> >> +	if (is_rebase_i(opts))
> >> +		return rebase_path();
> >>  	return git_path_seq_dir();
> >>  }
> > 
> > This obviously makes the assumption made by 39784cd362 ("sequencer:
> > remove useless get_dir() function", 2016-12-07) invalid, where the
> > "remove useless" thought that the callers of this function wants a
> > single answer that does not depend on opts.
> > 
> > I'll revert that commit from the sb/sequencer-abort-safety topic (as
> > the topic is in 'next' already) to keep this one.  Please holler if
> > that is a mistaken decision.
> 
> It seems to be the right decision if one wants to keep the internal-path
> backwards-compatibility of "git rebase -i" (and it seems that Dscho
> wants to keep it).

You make it sound as if I had any choice there. But you probably know as well as I do that scripted usage of rebase -i relies on the "internal-path backwards-compatibility", and as one of my stated goals was not to break anybody's existing rebase -i usage, well, you know.

> However, maintaining more than one directory (like "sequencer" for
> sequencer state and "rebase-merge" for rebase todo and log) can cause
> the necessity to be even more careful when hacking on sequencer... For
> example, the cleanup code must delete both of them, not only one of them.

That is incorrect. It depends on the options which directory is used. And it is that directory (and not both) that should be cleaned up in the end.

Otherwise you run into a ton of pain e.g. when running a rebase -i with an `exec git cherry-pick ...` line: all of a sudden, that little innocuous line would simply destroy the state directory of the current rebase -i.

That's a rather big no-no.
> Hence, I think that it's a wiser choice to keep one directory for
> sequencer state *and* additional files.
See above for a good reason not to choose that.
> If we (a) save everything in "sequencer", we break the internal-path
> backwards-compatbility but we can get rid of get_dir()...

... and we break power users' scripts that use rebase -i. Making those power users very angry.

Including me.
> If we (b) save everything in "rebase-merge" when using rebase -i and
> save everything in "sequencer" for pick/revert, we are 100%
> backwards-compatible, but we have to change every occurrence of
> git_path_seq_dir() to get_dir()

Yes, that is exactly what we have to do. And that is exactly what I did, too, in the patch series I labeled "prepare-sequencer".

The only three uses of git_path_seq_dir() I left intact are:
- in builtin/commit.c, where it is used to determine whether a cherry-pick
  is really still in progress, after testing that there is a
  CHERRY_PICK_HEAD file
- in get_dir() (obviously)
- in create_seq_dir(), where we could technically avoid calling the
  function a whopping three times and use a parameter instead (not my
  construction site, though, so I leave this alone)
> and the GIT_PATH_FUNC definitions of git_path_{todo,opts,head}_file must
> be replaced by definitions using get_dir().

That is incorrect. The git_path_todo(), git_path_opts() and git_path_head() functions need to be used in the cherry-pick/revert code path. So they need to be left as-are.

Ciao, Dscho

Stephan Beyer· Dec 19, 2016, 21:27 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 01/34] sequencer: support a new action: 'interactive rebase'

Hi Dscho,
Show 13 quoted lines
>> However, maintaining more than one directory (like "sequencer" for
>> sequencer state and "rebase-merge" for rebase todo and log) can cause
>> the necessity to be even more careful when hacking on sequencer... For
>> example, the cleanup code must delete both of them, not only one of them.
> 
> That is incorrect. It depends on the options which directory is used. And
> it is that directory (and not both) that should be cleaned up in the end.
> 
> Otherwise you run into a ton of pain e.g. when running a rebase -i with an
> `exec git cherry-pick ...` line: all of a sudden, that little innocuous
> line would simply destroy the state directory of the current rebase -i.
> 
> That's a rather big no-no.

Ahh, I see, there seems to be a misunderstanding on my side about how you did it (I did not look into the other patches (obviously)).

Thanks for clarifying!
Best
  Stephan
Johannes Schindelin· Dec 13, 2016, 15:29 UTC · re: Johannes Schindelin · lore

[PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

The 'noop' command is probably the most boring of all rebase -i commands to support in the sequencer.

Which makes it an excellent candidate for this first stab to add support for rebase -i's commands to the sequencer.

For the moment, let's also treat empty lines and commented-out lines as 'noop'; We will refine that handling later in this patch series.

To make it easier to identify "classes" of todo_commands (such as: determine whether a command is pick-like, i.e. handles a single commit), let's enforce a certain order of said commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)
Show changes to sequencer.c +32 −3
diff --git a/sequencer.c b/sequencer.c
index 21cfdacd06..1224799286 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -639,14 +639,23 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
+/*
+ * Note that ordering matters in this enum. Not only must it match the mapping
+ * below, it is also divided into several sections that matter.  When adding
+ * new commands, make sure you add it in the right section.
+ */
 enum todo_command {
+	/* commands that handle commits */
 	TODO_PICK = 0,
-	TODO_REVERT
+	TODO_REVERT,
+	/* commands that do nothing but are counted for reporting progress */
+	TODO_NOOP
 };
 
 static const char *todo_command_strings[] = {
 	"pick",
-	"revert"
+	"revert",
+	"noop"
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -916,6 +925,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
+	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
+		item->command = TODO_NOOP;
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
@@ -924,6 +941,13 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	if (i >= ARRAY_SIZE(todo_command_strings))
 		return -1;
 
+	if (item->command == TODO_NOOP) {
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
@@ -1292,7 +1316,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(item->command, item->commit, opts);
+		if (item->command <= TODO_REVERT)
+			res = do_pick_commit(item->command, item->commit,
+					opts);
+		else if (item->command != TODO_NOOP)
+			return error(_("unknown command %d"), item->command);
+
 		todo_list->current++;
 		if (res)
 			return res;
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 13, 2016, 20:38 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 5 quoted lines
> +/*
> + * Note that ordering matters in this enum. Not only must it match the mapping
> + * below, it is also divided into several sections that matter.  When adding
> + * new commands, make sure you add it in the right section.
> + */
Good thinking.  Makes me wish C were a better language, though ;-)
Show 15 quoted lines
>  enum todo_command {
> +	/* commands that handle commits */
>  	TODO_PICK = 0,
> -	TODO_REVERT
> +	TODO_REVERT,
> +	/* commands that do nothing but are counted for reporting progress */
> +	TODO_NOOP
>  };
>  
>  static const char *todo_command_strings[] = {
>  	"pick",
> -	"revert"
> +	"revert",
> +	"noop"
>  };
Show 10 quoted lines
> @@ -1292,7 +1316,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  		struct todo_item *item = todo_list->items + todo_list->current;
>  		if (save_todo(todo_list, opts))
>  			return -1;
> -		res = do_pick_commit(item->command, item->commit, opts);
> +		if (item->command <= TODO_REVERT)
> +			res = do_pick_commit(item->command, item->commit,
> +					opts);
> +		else if (item->command != TODO_NOOP)
> +			return error(_("unknown command %d"), item->command);

I wonder if making this a switch() statement is easier to read in the longer run. The only thing at this point we are gaining by "not only mapping and enum must match, the orders matter" is so that this codepath can do the same thing for PICK and REVERT, but these two would become more and more minority as we learn more words.

>  		todo_list->current++;
>  		if (res)
>  			return res;
Linus Torvalds· Dec 13, 2016, 20:48 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

On Tue, Dec 13, 2016 at 12:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 9 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>
>> +/*
>> + * Note that ordering matters in this enum. Not only must it match the mapping
>> + * below, it is also divided into several sections that matter.  When adding
>> + * new commands, make sure you add it in the right section.
>> + */
>
> Good thinking.  Makes me wish C were a better language, though ;-)
Do this:
  static const char *todo_command_strings[] = {
      [TODO_PICK] = "pick",
      [TODO_REVERT] = "revert",
      [TODO_NOOP] = "noop:,
  };

which makes the array be order-independent. You still need to make sure you fill in all the entries, of course, but it tends to avoid at least one gotcha, and it makes it more obvious how the two are tied together.

              Linus
Junio C Hamano· Dec 13, 2016, 21:54 UTC · re: Linus Torvalds · lore

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

Linus Torvalds <torvalds@linux-foundation.org> writes:
Show 25 quoted lines
> On Tue, Dec 13, 2016 at 12:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>>
>>> +/*
>>> + * Note that ordering matters in this enum. Not only must it match the mapping
>>> + * below, it is also divided into several sections that matter.  When adding
>>> + * new commands, make sure you add it in the right section.
>>> + */
>>
>> Good thinking.  Makes me wish C were a better language, though ;-)
>
> Do this:
>
>   static const char *todo_command_strings[] = {
>       [TODO_PICK] = "pick",
>       [TODO_REVERT] = "revert",
>       [TODO_NOOP] = "noop:,
>   };
>
> which makes the array be order-independent. You still need to make
> sure you fill in all the entries, of course, but it tends to avoid at
> least one gotcha, and it makes it more obvious how the two are tied
> together.
>
>               Linus

Yes, I know. But I do not think the variant of C we stick to is not new enough to have that.

Johannes Schindelin· Dec 19, 2016, 12:51 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

Hi,
On Tue, 13 Dec 2016, Junio C Hamano wrote:
Show 12 quoted lines
> Linus Torvalds <torvalds@linux-foundation.org> writes:
> 
> > On Tue, Dec 13, 2016 at 12:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> >>
> >>> +/*
> >>> + * Note that ordering matters in this enum. Not only must it match the mapping
> >>> + * below, it is also divided into several sections that matter.  When adding
> >>> + * new commands, make sure you add it in the right section.
> >>> + */
> >>
> >> Good thinking.

Not my thinking... This was in direct response to a suggestion by Dennis Kaarsemaker, I cannot take credit for the idea.

Show 17 quoted lines
> Makes me wish C were a better language, though ;-)
> >
> > Do this:
> >
> >   static const char *todo_command_strings[] = {
> >       [TODO_PICK] = "pick",
> >       [TODO_REVERT] = "revert",
> >       [TODO_NOOP] = "noop:,
> >   };
> >
> > which makes the array be order-independent. You still need to make
> > sure you fill in all the entries, of course, but it tends to avoid at
> > least one gotcha, and it makes it more obvious how the two are tied
> > together.
> 
> Yes, I know.  But I do not think the variant of C we stick to is not
> new enough to have that.

Let me try to express this without double negation: our coding guidelines state very explicitly that we do not use C99 initializers, and it also explains why: we want to support a broader range of compilers. For details, see:

https://github.com/git/git/blob/v2.11.0/Documentation/CodingGuidelines#L179-L181

TBH I briefly considered going the same route as I did for the fsck warnings (taking a lot of heat for the ugliness):

https://github.com/git/git/blob/v2.11.0/fsck.c#L17-L85
It would have looked somewhat like this:
	#define FOREACH_TODO(FUNC) \
		FUNC(PICK, 'p', "pick") \
		FUNC(REVERT, 0, "revert") \
		FUNC(EDIT, 'e', "edit") \
		...
		FUNC(COMMENT, 0, NULL)
	#define TODO(id, short, long) TODO_##id,
	enum todo_command {
		FOREACH_TODO(TODO)
		TODO_MAX
	};
	#undef TODO
	#define TODO(id, short, long) { short, long },
	static struct {
		char c;
		const char *str;
	} todo_command_info[] = {
		FOREACH_TODO(TODO)
		{ 0, NULL }
	};
	#undef TODO

I have to admit that even I prefer the version I provided in my contribution over the "fsck method" outlined above.

Ciao, Dscho

Junio C Hamano· Dec 19, 2016, 18:31 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 17 quoted lines
> On Tue, 13 Dec 2016, Junio C Hamano wrote:
>
>> Linus Torvalds <torvalds@linux-foundation.org> writes:
>> 
>> > On Tue, Dec 13, 2016 at 12:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> >> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> >>
>> >>> +/*
>> >>> + * Note that ordering matters in this enum. Not only must it match the mapping
>> >>> + * below, it is also divided into several sections that matter.  When adding
>> >>> + * new commands, make sure you add it in the right section.
>> >>> + */
>> >>
>> >> Good thinking.
>
> Not my thinking... This was in direct response to a suggestion by Dennis
> Kaarsemaker, I cannot take credit for the idea.

I now realize that I was unclear about what "thinking" I found good in my comment. I do not particularly like defining two parallel things and having to maintain them in sync. The "Good thinking" praise goes to whoever thought that this burdensome fact deserves a clear comment in front of these two things.

And ...
Show 11 quoted lines
>> Makes me wish C were a better language, though ;-)
>> >
>> > Do this:
>> >
>> >   static const char *todo_command_strings[] = {
>> >       [TODO_PICK] = "pick",
>> >       [TODO_REVERT] = "revert",
>> >       [TODO_NOOP] = "noop:,
>> >   };
>> >
>> > which makes the array be order-independent.

... solves only one-half of the problem with the language I had. The order of the entries in this array[] may become more flexible in the source, but you still have to define enum separately.

I guess if we really want to, we need to resort to something "ugly but workable" like what you did in fsck.c with FOREACH_MSG_ID(X). That approach may be the least ugly way if we have to maintain two or more parallel things in sync.

    ... and then realizes you wrote pretty much the same thing
    ... after writing all of the above ;-)
But it is way overkill for sequencer commands that are only handful.
Johannes Schindelin· Dec 19, 2016, 13:37 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 02/34] sequencer (rebase -i): implement the 'noop' command

Hi Junio,
On Tue, 13 Dec 2016, Junio C Hamano wrote:
Show 18 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -1292,7 +1316,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
> >  		struct todo_item *item = todo_list->items + todo_list->current;
> >  		if (save_todo(todo_list, opts))
> >  			return -1;
> > -		res = do_pick_commit(item->command, item->commit, opts);
> > +		if (item->command <= TODO_REVERT)
> > +			res = do_pick_commit(item->command, item->commit,
> > +					opts);
> > +		else if (item->command != TODO_NOOP)
> > +			return error(_("unknown command %d"), item->command);
> 
> I wonder if making this a switch() statement is easier to read in
> the longer run.  The only thing at this point we are gaining by "not
> only mapping and enum must match, the orders matter" is so that this
> codepath can do the same thing for PICK and REVERT, but these two
> would become more and more minority as we learn more words.

I doubt that this is easier to read. There are essentially three categories we are handling: exec, comments, and everything else. IMO the current code is the easiest to understand.

Ciao, Dscho

Johannes Schindelin· Dec 13, 2016, 15:30 UTC · re: Johannes Schindelin · lore

[PATCH v2 08/34] sequencer (rebase -i): implement the short commands

For users' convenience, most rebase commands can be abbreviated, e.g. 'p' instead of 'pick' and 'x' instead of 'exec'. Let's teach the sequencer to handle those abbreviated commands just fine.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 36 ++++++++++++++++++++++--------------
 1 file changed, 22 insertions(+), 14 deletions(-)
Show changes to sequencer.c +22 −14
diff --git a/sequencer.c b/sequencer.c
index 271c21581d..e443f4765d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -710,20 +710,23 @@ enum todo_command {
 	TODO_NOOP
 };
 
-static const char *todo_command_strings[] = {
-	"pick",
-	"revert",
-	"edit",
-	"fixup",
-	"squash",
-	"exec",
-	"noop"
+static struct {
+	char c;
+	const char *str;
+} todo_command_info[] = {
+	{ 'p', "pick" },
+	{ 0,   "revert" },
+	{ 'e', "edit" },
+	{ 'f', "fixup" },
+	{ 's', "squash" },
+	{ 'x', "exec" },
+	{ 0,   "noop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if ((size_t)command < ARRAY_SIZE(todo_command_strings))
-		return todo_command_strings[command];
+	if ((size_t)command < ARRAY_SIZE(todo_command_info))
+		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
 
@@ -1125,12 +1128,17 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
-		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_strings))
+		else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
+			bol++;
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_info))
 		return -1;
 
 	if (item->command == TODO_NOOP) {
@@ -1325,7 +1333,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 {
 	enum todo_command command = opts->action == REPLAY_PICK ?
 		TODO_PICK : TODO_REVERT;
-	const char *command_string = todo_command_strings[command];
+	const char *command_string = todo_command_info[command].str;
 	struct commit *commit;
 
 	if (prepare_revs(opts))
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 15, 2016, 18:43 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 08/34] sequencer (rebase -i): implement the short commands

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 6 quoted lines
> For users' convenience, most rebase commands can be abbreviated, e.g.
> 'p' instead of 'pick' and 'x' instead of 'exec'. Let's teach the
> sequencer to handle those abbreviated commands just fine.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
Sensible.
Johannes Schindelin· Dec 13, 2016, 15:30 UTC · re: Johannes Schindelin · lore

[PATCH v2 09/34] sequencer (rebase -i): write an author-script file

When the interactive rebase aborts, it writes out an author-script file to record the author information for the current commit. As we are about to teach the sequencer how to perform the actions behind an interactive rebase, it needs to write those author-script files, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 1 deletion(-)
Show changes to sequencer.c +50 −1
diff --git a/sequencer.c b/sequencer.c
index e443f4765d..80469b6954 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -515,6 +515,53 @@ static int is_index_unchanged(void)
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
 
+static int write_author_script(const char *message)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *eol;
+	int res;
+
+	for (;;)
+		if (!*message || starts_with(message, "\n")) {
+missing_author:
+			/* Missing 'author' line? */
+			unlink(rebase_path_author_script());
+			return 0;
+		}
+		else if (skip_prefix(message, "author ", &message))
+			break;
+		else if ((eol = strchr(message, '\n')))
+			message = eol + 1;
+		else
+			goto missing_author;
+
+	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, " <", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, "> ", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_DATE='@");
+	while (*message && *message != '\n' && *message != '\r')
+		if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	res = write_message(buf.buf, buf.len, rebase_path_author_script(), 1);
+	strbuf_release(&buf);
+	return res;
+}
+
 /*
  * Read the author-script file into an environment block, ready for use in
  * run_command(), that can be free()d afterwards.
@@ -974,7 +1021,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
+	if (is_rebase_i(opts) && write_author_script(msg.message) < 0)
+		res = -1;
+	else if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 15, 2016, 18:50 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 09/34] sequencer (rebase -i): write an author-script file

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 16 quoted lines
> +	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
> +	while (*message && *message != '\n' && *message != '\r')
> +		if (skip_prefix(message, " <", &message))
> +			break;
> +		else if (*message != '\'')
> +			strbuf_addch(&buf, *(message++));
> +		else
> +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
> +	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
> +	while (*message && *message != '\n' && *message != '\r')
> +		if (skip_prefix(message, "> ", &message))
> +			break;
> +		else if (*message != '\'')
> +			strbuf_addch(&buf, *(message++));
> +		else
> +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
Aren't these reading from an in-core commit object?  

If so, it should use split_ident_line() for consistency with other parts of the system to do this parsing. We should also already have a helper for simple shell-quoting in quote.c and you would want to use that instead of open coding like this.

Johannes Schindelin· Dec 19, 2016, 17:06 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 09/34] sequencer (rebase -i): write an author-script file

Hi Junio,
On Thu, 15 Dec 2016, Junio C Hamano wrote:
Show 25 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
> > +	while (*message && *message != '\n' && *message != '\r')
> > +		if (skip_prefix(message, " <", &message))
> > +			break;
> > +		else if (*message != '\'')
> > +			strbuf_addch(&buf, *(message++));
> > +		else
> > +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
> > +	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
> > +	while (*message && *message != '\n' && *message != '\r')
> > +		if (skip_prefix(message, "> ", &message))
> > +			break;
> > +		else if (*message != '\'')
> > +			strbuf_addch(&buf, *(message++));
> > +		else
> > +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
> 
> Aren't these reading from an in-core commit object?  
> 
> If so, it should use split_ident_line() for consistency with other
> parts of the system to do this parsing.  We should also already have
> a helper for simple shell-quoting in quote.c and you would want to
> use that instead of open coding like this.

We keep coming back to the same argument. You want this quoting/dequoting to be turned into a full-fledged parser. And I keep pointing out that the code here does not *need* to parse but only construct an environment block.

Hopefully the next iteration that integrates Peff's suggestions will find more of your approval.

Ciao, Dscho

Junio C Hamano· Dec 20, 2016, 01:32 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 09/34] sequencer (rebase -i): write an author-script file

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 34 quoted lines
> Hi Junio,
>
> On Thu, 15 Dec 2016, Junio C Hamano wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> > +	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
>> > +	while (*message && *message != '\n' && *message != '\r')
>> > +		if (skip_prefix(message, " <", &message))
>> > +			break;
>> > +		else if (*message != '\'')
>> > +			strbuf_addch(&buf, *(message++));
>> > +		else
>> > +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
>> > +	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
>> > +	while (*message && *message != '\n' && *message != '\r')
>> > +		if (skip_prefix(message, "> ", &message))
>> > +			break;
>> > +		else if (*message != '\'')
>> > +			strbuf_addch(&buf, *(message++));
>> > +		else
>> > +			strbuf_addf(&buf, "'\\\\%c'", *(message++));
>> 
>> Aren't these reading from an in-core commit object?  
>> 
>> If so, it should use split_ident_line() for consistency with other
>> parts of the system to do this parsing.  We should also already have
>> a helper for simple shell-quoting in quote.c and you would want to
>> use that instead of open coding like this.
>
> We keep coming back to the same argument. You want this quoting/dequoting
> to be turned into a full-fledged parser. And I keep pointing out that the
> code here does not *need* to parse but only construct an environment
> block.

I am afraid you are mis-reading me. I see a code that _READS_ some data format, for which we already have a parser, and then write out things based on what it read. I do not want you to make anything into a full-fledged parser---I just do not want to see an ad-hoc reader and instead the code to USE existing parser.

Junio C Hamano· Dec 20, 2016, 23:46 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 09/34] sequencer (rebase -i): write an author-script file

Junio C Hamano <gitster@pobox.com> writes:
Show 10 quoted lines
>> We keep coming back to the same argument. You want this quoting/dequoting
>> to be turned into a full-fledged parser. And I keep pointing out that the
>> code here does not *need* to parse but only construct an environment
>> block.
>
> I am afraid you are mis-reading me.  I see a code that _READS_ some
> data format, for which we already have a parser, and then write out
> things based on what it read.  I do not want you to make anything
> into a full-fledged parser---I just do not want to see an ad-hoc
> reader and instead the code to USE existing parser.
To extend this a bit.

I care rather deeply about not spreading the re-invention of parsers and formatters throughout the code, because that would introduce unnecesary maintenance burden.

Quoting a string so that it is acceptable inside a pair of single quotes, occasionally stepping outside of the sq context and using backquote to excape individual byte, for example, might feel simple enough that anybody can just write inline instead of learning how to call and actually calling sq_quote(). You open ', show each byte unless it is a ', in which case you close ' and give \' and immediately open '. That was what sq_quote() did originally and for a long time. If however we allowed everybody to reinvent the code to quote all over the place, with a lame "This is different and does not *need* to call shared code" excuse, it would have required a lot more effort to do a change like the one in 77d604c309 ("Enhanced sq_quote()", 2005-10-10) that changed the quoting rules slightly to make the output safer to accomodate different variants of shells.

The same thing can be said for split_ident() code. The "author" line has name, '<', email address, '>', timestring, '+' or '-', and timezone. Splitting them into NAME and TIME may be very simple, and "does not *need* to parse", right? Not really. If you look at how split_ident() evolved to accomodate the real world malformed lines like duplicated closing '>' and realize that what we have there may probably *NOT* the perfect one (i.e. there may be other malformed input we may have to accomodate by tweaking the implementation further), we really do not want a hand-rolled ad-hoc "splitter" that "does not *need* to parse".

The same thing can be said for the code that reads the author-script, too.

So please do not waste any more time arguing. I think you spent arguing more time than you would otherwise have had to spend if you just used existing helper functions, both in this thread and the older one about reading the author-script.

Johannes Schindelin· Dec 13, 2016, 15:30 UTC · re: Johannes Schindelin · lore

[PATCH v2 10/34] sequencer (rebase -i): allow continuing with staged changes

When an interactive rebase is interrupted, the user may stage changes before continuing, and we need to commit those changes in that case.

Please note that the nested "if" added to the sequencer_continue() is not combined into a single "if" because it will be extended with an "else" clause in a later patch in this patch series.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
Show changes to sequencer.c +40 −0
diff --git a/sequencer.c b/sequencer.c
index 80469b6954..855d3ba503 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1829,6 +1829,42 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
+static int commit_staged_changes(struct replay_opts *opts)
+{
+	int amend = 0;
+
+	if (has_unstaged_changes(1))
+		return error(_("cannot rebase: You have unstaged changes."));
+	if (!has_uncommitted_changes(0))
+		return 0;
+
+	if (file_exists(rebase_path_amend())) {
+		struct strbuf rev = STRBUF_INIT;
+		unsigned char head[20], to_amend[20];
+
+		if (get_sha1("HEAD", head))
+			return error(_("cannot amend non-existing commit"));
+		if (!read_oneliner(&rev, rebase_path_amend(), 0))
+			return error(_("invalid file: '%s'"), rebase_path_amend());
+		if (get_sha1_hex(rev.buf, to_amend))
+			return error(_("invalid contents: '%s'"),
+				rebase_path_amend());
+		if (hashcmp(head, to_amend))
+			return error(_("\nYou have uncommitted changes in your "
+				       "working tree. Please, commit them\n"
+				       "first and then run 'git rebase "
+				       "--continue' again."));
+
+		strbuf_release(&rev);
+		amend = 1;
+	}
+
+	if (run_git_commit(rebase_path_message(), opts, 1, 1, amend, 0))
+		return error(_("could not commit staged changes."));
+	unlink(rebase_path_amend());
+	return 0;
+}
+
 int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
@@ -1837,6 +1873,10 @@ int sequencer_continue(struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
+	if (is_rebase_i(opts)) {
+		if (commit_staged_changes(opts))
+			return -1;
+	}
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 15, 2016, 19:08 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 10/34] sequencer (rebase -i): allow continuing with staged changes

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 26 quoted lines
> When an interactive rebase is interrupted, the user may stage changes
> before continuing, and we need to commit those changes in that case.
>
> Please note that the nested "if" added to the sequencer_continue() is
> not combined into a single "if" because it will be extended with an
> "else" clause in a later patch in this patch series.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 40 ++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)
>
> diff --git a/sequencer.c b/sequencer.c
> index 80469b6954..855d3ba503 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1829,6 +1829,42 @@ static int continue_single_pick(void)
>  	return run_command_v_opt(argv, RUN_GIT_CMD);
>  }
>  
> +static int commit_staged_changes(struct replay_opts *opts)
> +{
> +	int amend = 0;
> +
> +	if (has_unstaged_changes(1))
> +		return error(_("cannot rebase: You have unstaged changes."));
The scripted one from 'master' seems to say
	$path_to_the_file: needs update
	You must edit all merge conflicts and then
	mark them as resolved using git add

when editing an existing commit in this case. The updated message looks more sensible for the situation, but I wonder if the control should even reach at this point.

One bad thing about reviewing this series is that all the comments are about codepaths that are not exercised, so they cannot be more than "they look good". A comment "If the caller does X, this will be better than the original" (or this will regress, for that matter) cannot be validated for its relevance because we won't know the what the caller does in the endgame while reviewing these earlier steps.

Johannes Schindelin· Dec 13, 2016, 15:30 UTC · re: Johannes Schindelin · lore

[PATCH v2 11/34] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

The scripted version of the interactive rebase already does that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
Show changes to sequencer.c +6 −1
diff --git a/sequencer.c b/sequencer.c
index 855d3ba503..abffaf3b40 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1835,8 +1835,13 @@ static int commit_staged_changes(struct replay_opts *opts)
 
 	if (has_unstaged_changes(1))
 		return error(_("cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0))
+	if (!has_uncommitted_changes(0)) {
+		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error(_("could not remove CHERRY_PICK_HEAD"));
 		return 0;
+	}
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 16, 2016, 19:13 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 11/34] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> The scripted version of the interactive rebase already does that.

Sensible. I was wondering why this wasn't there while reviewing 10/34, comparing the two (this is not a suggestion to squash this into the previous step).

Show 25 quoted lines
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 855d3ba503..abffaf3b40 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1835,8 +1835,13 @@ static int commit_staged_changes(struct replay_opts *opts)
>  
>  	if (has_unstaged_changes(1))
>  		return error(_("cannot rebase: You have unstaged changes."));
> -	if (!has_uncommitted_changes(0))
> +	if (!has_uncommitted_changes(0)) {
> +		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
> +
> +		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
> +			return error(_("could not remove CHERRY_PICK_HEAD"));
>  		return 0;
> +	}
>  
>  	if (file_exists(rebase_path_amend())) {
>  		struct strbuf rev = STRBUF_INIT;
Johannes Schindelin· Dec 19, 2016, 17:22 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 11/34] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

Hi Junio,
On Fri, 16 Dec 2016, Junio C Hamano wrote:
Show 7 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > The scripted version of the interactive rebase already does that.
> 
> Sensible.  I was wondering why this wasn't there while reviewing
> 10/34, comparing the two (this is not a suggestion to squash this
> into the previous step).

It's a bit of a historical wart, as I discovered test breakages only long after implementing the fixup/squash commands (as you may have guessed, I implemented the commands one after another, fixing things as discovered by the test suite; it took something like a month until I got the rebase--helper based rebase -i to pass t3404).

In the end, I decided not to squash this into 10/34 because it seemed to be a significant enough change to merit its own commit.

Ciao, Dscho

Johannes Schindelin· Dec 13, 2016, 15:32 UTC · re: Johannes Schindelin · lore

[PATCH v2 30/34] sequencer (rebase -i): show only failed cherry-picks' output

This is the behavior of the shell script version of the interactive rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)
Show changes to sequencer.c +4 −0
diff --git a/sequencer.c b/sequencer.c
index dfa4fab98b..edc213a2c8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -464,6 +464,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	o.ancestor = base ? base_label : "(empty tree)";
 	o.branch1 = "HEAD";
 	o.branch2 = next ? next_label : "(empty tree)";
+	if (is_rebase_i(opts))
+		o.buffer_output = 2;
 
 	head_tree = parse_tree_indirect(head);
 	next_tree = next ? next->tree : empty_tree();
@@ -475,6 +477,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (is_rebase_i(opts) && clean <= 0)
+		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Dec 13, 2016, 15:32 UTC · re: Johannes Schindelin · lore

[PATCH v2 29/34] sequencer (rebase -i): show only failed `git commit`'s output

This is the behavior of the shell script version of the interactive rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
Show changes to sequencer.c +7 −2
diff --git a/sequencer.c b/sequencer.c
index 63f6f25ced..dfa4fab98b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -647,10 +647,15 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 {
 	char **env = NULL;
 	struct argv_array array;
-	int rc;
+	int opt = RUN_GIT_CMD, rc;
 	const char *value;
 
 	if (is_rebase_i(opts)) {
+		if (!edit) {
+			opt |= RUN_COMMAND_STDOUT_TO_STDERR;
+			opt |= RUN_HIDE_STDERR_ON_SUCCESS;
+		}
+
 		env = read_author_script();
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
@@ -687,7 +692,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
 			(const char *const *)env);
 	argv_array_clear(&array);
 	free(env);
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Dec 13, 2016, 19:42 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> This marks the count down to '3': two more patch series after this
> (really tiny ones) and we have a faster rebase -i.
Nice.
Show 18 quoted lines
> Apart from mostly cosmetic patches (and the occasional odd bug that I
> fixed promptly), I used these patches since mid May to perform all of my
> interactive rebases. In mid June, I had the idea to teach rebase -i to
> run *both* scripted rebase and rebase--helper and to cross-validate the
> results. This slowed down all my interactive rebases since, but helped
> me catch three rather obscure bugs (e.g. that git commit --fixup unfolds
> long onelines and rebase -i still finds the correct original commit).
> ...
> Please note that the interdiff vs v1 is only of limited use: too many
> things changed in the meantime, in particular the prepare-sequencer
> branch that went through a couple of iterations before it found its way
> into git.git's master branch. So please take the interdiff with a
> mountain range of salt.
> ...
> Changes since v1:
> ...
> - removed the beautiful ordinal logic (to print out "1st", "2nd", "3rd"
>   etc) and made things consistent with the current `rebase -i`.

It was removed because it was too Anglo-centric and unusable in i18n context, no?

Judging from the list above, interdiff are pretty much all cosmetic and that is why you say it is only of limited use, I guess.

    ... goes and reads the remainder and finds that these were
    ... all minor changes, mostly cosmetic, with a helper function
    ... refactored out or two and things of that nature.

It is actually a good thing. We do not want to see it deviate too drastically from what you have been testing for some months.

Thanks.
Johannes Schindelin· Dec 19, 2016, 13:05 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

Hi Junio,
On Tue, 13 Dec 2016, Junio C Hamano wrote:
Show 23 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Apart from mostly cosmetic patches (and the occasional odd bug that I
> > fixed promptly), I used these patches since mid May to perform all of my
> > interactive rebases. In mid June, I had the idea to teach rebase -i to
> > run *both* scripted rebase and rebase--helper and to cross-validate the
> > results. This slowed down all my interactive rebases since, but helped
> > me catch three rather obscure bugs (e.g. that git commit --fixup unfolds
> > long onelines and rebase -i still finds the correct original commit).
> > ...
> > Please note that the interdiff vs v1 is only of limited use: too many
> > things changed in the meantime, in particular the prepare-sequencer
> > branch that went through a couple of iterations before it found its way
> > into git.git's master branch. So please take the interdiff with a
> > mountain range of salt.
> > ...
> > Changes since v1:
> > ...
> > - removed the beautiful ordinal logic (to print out "1st", "2nd", "3rd"
> >   etc) and made things consistent with the current `rebase -i`.
> 
> It was removed because it was too Anglo-centric and unusable in i18n
> context, no?

Yes, but I remember putting in a lot of time to try to come up with the most elegant way to convert a number into an English ordinal in a shell function. That's where all that wistfulness came from.

> Judging from the list above, interdiff are pretty much all cosmetic
> and that is why you say it is only of limited use, I guess.

No, I said that it is only of limited use because I had to fudge things to come up with an interdiff. I had to fudge things because there is no interdiff: it would require the previous iteration of the patch series to apply cleanly to the current `master`, which it does not. So I rebased the patches and left as much intact as possible, which means that the rebased changes do not even compile because they were based on a previous iteration of the prepare-sequencer patch series that did not make it into `master` without substantial changes.

Show 6 quoted lines
>     ... goes and reads the remainder and finds that these were
>     ... all minor changes, mostly cosmetic, with a helper function
>     ... refactored out or two and things of that nature.
> 
> It is actually a good thing.  We do not want to see it deviate too
> drastically from what you have been testing for some months.

Well, that ship has sailed. Neither of the patch series "require-clean-work-tree", "libify-sequencer" and "prepare-sequencer" made it into `master` without substantial deviations from the code I had been testing and improving for half a year. I did not expect the code to be accepted unchanged, anyways.

Ciao, Dscho

Johannes Sixt· Dec 14, 2016, 07:08 UTC · re: Johannes Schindelin · lore

Re: [PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

Am 13.12.2016 um 16:29 schrieb Johannes Schindelin:
> base-commit: 8d7a455ed52e2a96debc080dfc011b6bb00db5d2
> Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v2
> Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v2
Thank you so much!

I would appreciate if you could publish a branch that contains the end game so that I can test it, too. Currently I am still using

  git://github.com/dscho/git interactive-rebase (fca871a3cf4d)
-- Hannes
Johannes Schindelin· Dec 19, 2016, 14:09 UTC · re: Johannes Sixt · lore

Re: [PATCH v2 00/34] Teach the sequencer to act as rebase -i's backend

Hi Hannes,
On Wed, 14 Dec 2016, Johannes Sixt wrote:
Show 11 quoted lines
> Am 13.12.2016 um 16:29 schrieb Johannes Schindelin:
> > base-commit: 8d7a455ed52e2a96debc080dfc011b6bb00db5d2
> > Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v2
> > Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v2
> 
> Thank you so much!
> 
> I would appreciate if you could publish a branch that contains the end game so
> that I can test it, too. Currently I am still using
> 
>  git://github.com/dscho/git interactive-rebase (fca871a3cf4d)

That's the one. Unless I forget, I use the Git garden shears to update 'interactive-rebase' whenever I update 'sequencer-i', too.

Ciao, Dscho

Johannes Schindelin· Jan 2, 2017, 15:25 UTC · re: Johannes Schindelin · lore

[PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

This marks the count down to '3': two more patch series after this (really tiny ones) and we have a faster rebase -i.

The idea of this patch series is to teach the sequencer to understand all of the commands in `git-rebase-todo` scripts, to execute them and to behave pretty much very the same as `git rebase -i --continue` when called with the newly-introduced REPLAY_INTERACTIVE_REBASE mode.

Most of these patches should be pretty much straight-forward. When not, I tried to make a point of describing enough background in the commit message. Please feel free to point out where my explanations fall short.

Note that even after this patch series is applied, rebase -i is still unaffected. It will require the next patch series which introduces the rebase--helper that essentially implements `git rebase -i --continue` by calling the sequencer with the appropriate options.

The final patch series will move a couple of pre- and post-processing steps into the rebase--helper/sequencer (such as expanding/shrinking the SHA-1s, reordering the fixup!/squash! lines, etc). This might sound like a mere add-on, but it is essential for the speed improvements: those stupid little processing steps really dominated the execution time in my tests.

Apart from mostly cosmetic patches (and the occasional odd bug that I fixed promptly), I used these patches since mid May to perform all of my interactive rebases. In mid June, I had the idea to teach rebase -i to run *both* scripted rebase and rebase--helper and to cross-validate the results. This slowed down all my interactive rebases since, but helped me catch three rather obscure bugs (e.g. that git commit --fixup unfolds long onelines and rebase -i still finds the correct original commit).

This is all only to say that I am rather confident that the current code does the job.

Since sending out v1, I integrated all of these patch series into Git for Windows v2.10.0, where they have been live ever since, and used by myself (also in a Linux VM, as Git for Windows' master branch always builds also on Linux and passes the test suite, too).

Just to reiterate why I do all this: it speeds up the interactive rebase substantially. Even with a not yet fully builtin rebase -i, but just the part after the user edited the `git-rebase-todo` script.

The performance test I introduced to demonstrate this (p3404) shows a speed-up of +380% here (i.e. roughly 5x), from ~8.8 seconds to ~1.8 seconds. This is on Windows, where the performance impact of avoiding shell scripting is most noticable.

On MacOSX and on Linux, the speed-up is less pronounced, but still noticable, at least if you trust Travis CI, which I abused to perform that test for me. Check for yourself (searching for "3404.2") here: https://travis-ci.org/git/git/builds/156295227. According to those logs, p3404 is speeded up from ~0.45 seconds to ~0.12 seconds on Linux (read: about 3.5x) and from ~1.7 seconds to ~0.5 seconds on MacOSX (read: almost 4x).

Changes since v2:
- fixed a TRANSLATORS: comment
- added a comment to clarify why save_todo() does not write the current
  command to git-rebase-todo
- fixed the comment for "stopped-sha" that said that a long commit name
  is stored in that file, when it is in fact an abbreviated one
- consistently put the "else" keyword on the same line as the preceding
  closing brace, if any (this adds two patches to the already large
  patch series, sorry)
- completely revamped the update_squash_messages() function to make it
  more obvious why it does not overflow its buffer, and to fix it when
  using it with localised messages
- now the sequencer code uses find_commit_subject() consistently to find
  the commit message (it does not use the subject's length returned by
  the find_commit_subject() function, of course); this adds yet another
  patch to this already large patch series, sorry.
- introduced an is_noop() function (as opposed to testing <= TODO_NOOP
  manually) to make the code less magic.
- fixed two potential leaks of get_commit_buffer()'s returned buffer
- fixed leaks in the error code paths when trying to update the ref at
  the end
- renamed read_author_script() into read_env_script() to reflect that
  it would accept any environment setting, and made it much more
  readable
- completely reworked the strategy how to suppress the output of
  successful cherry-picks: instead of introducing
  RUN_HIDE_STDERR_ON_SUCCESS, the code now uses pipe_command() to catch
  the output itself, so that it can be shown in case of error
- replaced a spawned `diff-tree` command by code using the diff functions
  directly
Johannes Schindelin (38):
  sequencer: avoid unnecessary curly braces
  sequencer: move "else" keyword onto the same line as preceding brace
  sequencer: use a helper to find the commit message
  sequencer: support a new action: 'interactive rebase'
  sequencer (rebase -i): implement the 'noop' command
  sequencer (rebase -i): implement the 'edit' command
  sequencer (rebase -i): implement the 'exec' command
  sequencer (rebase -i): learn about the 'verbose' mode
  sequencer (rebase -i): write the 'done' file
  sequencer (rebase -i): add support for the 'fixup' and 'squash'
    commands
  sequencer (rebase -i): implement the short commands
  sequencer (rebase -i): write an author-script file
  sequencer (rebase -i): allow continuing with staged changes
  sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed
  sequencer (rebase -i): skip some revert/cherry-pick specific code path
  sequencer (rebase -i): the todo can be empty when continuing
  sequencer (rebase -i): update refs after a successful rebase
  sequencer (rebase -i): leave a patch upon error
  sequencer (rebase -i): implement the 'reword' command
  sequencer (rebase -i): allow fast-forwarding for edit/reword
  sequencer (rebase -i): refactor setting the reflog message
  sequencer (rebase -i): set the reflog message consistently
  sequencer (rebase -i): copy commit notes at end
  sequencer (rebase -i): record interrupted commits in rewritten, too
  sequencer (rebase -i): run the post-rewrite hook, if needed
  sequencer (rebase -i): respect the rebase.autostash setting
  sequencer (rebase -i): respect strategy/strategy_opts settings
  sequencer (rebase -i): allow rescheduling commands
  sequencer (rebase -i): implement the 'drop' command
  sequencer (rebase -i): differentiate between comments and 'noop'
  sequencer: make reading author-script more elegant
  sequencer: use run_command() directly
  sequencer (rebase -i): show only failed `git commit`'s output
  sequencer (rebase -i): show only failed cherry-picks' output
  sequencer (rebase -i): suggest --edit-todo upon unknown command
  sequencer (rebase -i): show the progress
  sequencer (rebase -i): write the progress into files
  sequencer (rebase -i): write out the final message
 sequencer.c | 1105 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 sequencer.h |    4 +-
 2 files changed, 1013 insertions(+), 96 deletions(-)
base-commit: e05806da9ec4aff8adfed142ab2a2b3b02e33c8c
Published-As: https://github.com/dscho/git/releases/tag/sequencer-i-v3
Fetch-It-Via: git fetch https://github.com/dscho/git sequencer-i-v3
Interdiff vs v2:
 diff --git a/run-command.c b/run-command.c
 index 5bb957afdd..ca905a9e80 100644
 --- a/run-command.c
 +++ b/run-command.c
 @@ -589,29 +589,6 @@ int run_command_v_opt_cd_env(const char **argv, int opt, const char *dir, const
  	cmd.clean_on_exit = opt & RUN_CLEAN_ON_EXIT ? 1 : 0;
  	cmd.dir = dir;
  	cmd.env = env;
 -
 -	if (opt & RUN_HIDE_STDERR_ON_SUCCESS) {
 -		struct strbuf buf = STRBUF_INIT;
 -		int res;
 -
 -		cmd.err = -1;
 -		if (start_command(&cmd) < 0)
 -			return -1;
 -
 -		if (strbuf_read(&buf, cmd.err, 0) < 0) {
 -			close(cmd.err);
 -			finish_command(&cmd); /* throw away exit code */
 -			return -1;
 -		}
 -
 -		close(cmd.err);
 -		res = finish_command(&cmd);
 -		if (res)
 -			fputs(buf.buf, stderr);
 -		strbuf_release(&buf);
 -		return res;
 -	}
 -
  	return run_command(&cmd);
  }
  
 diff --git a/run-command.h b/run-command.h
 index 65a21ddd4e..dd1c78c28d 100644
 --- a/run-command.h
 +++ b/run-command.h
 @@ -72,7 +72,6 @@ extern int run_hook_ve(const char *const *env, const char *name, va_list args);
  #define RUN_SILENT_EXEC_FAILURE 8
  #define RUN_USING_SHELL 16
  #define RUN_CLEAN_ON_EXIT 32
 -#define RUN_HIDE_STDERR_ON_SUCCESS 64
  int run_command_v_opt(const char **argv, int opt);
  
  /*
 diff --git a/sequencer.c b/sequencer.c
 index c0aeb4f5f7..0e7d2ca5c8 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -102,7 +102,7 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
  /*
   * When we stop at a given patch via the "edit" command, this file contains
 - * the long commit name of the corresponding patch.
 + * the abbreviated commit name of the corresponding patch.
   */
  static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  /*
 @@ -464,8 +464,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
  
  	if (active_cache_changed &&
  	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
 -		/*
 -		 * TRANSLATORS: %s will be "revert", "cherry-pick" or
 +		/* TRANSLATORS: %s will be "revert", "cherry-pick" or
  		 * "rebase -i".
  		 */
  		return error(_("%s: Unable to write new index file"),
 @@ -524,8 +523,7 @@ static int write_author_script(const char *message)
  			/* Missing 'author' line? */
  			unlink(rebase_path_author_script());
  			return 0;
 -		}
 -		else if (skip_prefix(message, "author ", &message))
 +		} else if (skip_prefix(message, "author ", &message))
  			break;
  		else if ((eol = strchr(message, '\n')))
  			message = eol + 1;
 @@ -560,18 +558,17 @@ static int write_author_script(const char *message)
  }
  
  /*
 - * Read the author-script file into an environment block, ready for use in
 - * run_command(), that can be free()d afterwards.
 + * Read a list of environment variable assignments (such as the author-script
 + * file) into an environment block. Returns -1 on error, 0 otherwise.
   */
 -static char **read_author_script(void)
 +static int read_env_script(struct argv_array *env)
  {
  	struct strbuf script = STRBUF_INIT;
  	int i, count = 0;
 -	char *p, *p2, **env;
 -	size_t env_size;
 +	char *p, *p2;
  
  	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
 -		return NULL;
 +		return -1;
  
  	for (p = script.buf; *p; p++)
  		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
 @@ -583,19 +580,12 @@ static char **read_author_script(void)
  			count++;
  		}
  
 -	env_size = (count + 1) * sizeof(*env);
 -	strbuf_grow(&script, env_size);
 -	memmove(script.buf + env_size, script.buf, script.len);
 -	p = script.buf + env_size;
 -	env = (char **)strbuf_detach(&script, NULL);
 -
  	for (i = 0; i < count; i++) {
 -		env[i] = p;
 +		argv_array_push(env, p);
  		p += strlen(p) + 1;
  	}
 -	env[count] = NULL;
  
 -	return env;
 +	return 0;
  }
  
  static const char staged_changes_advice[] =
 @@ -628,19 +618,18 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
  			  int allow_empty, int edit, int amend,
  			  int cleanup_commit_message)
  {
 -	char **env = NULL;
 -	struct argv_array array;
 -	int opt = RUN_GIT_CMD, rc;
 +	struct child_process cmd = CHILD_PROCESS_INIT;
  	const char *value;
  
 +	cmd.git_cmd = 1;
 +
  	if (is_rebase_i(opts)) {
  		if (!edit) {
 -			opt |= RUN_COMMAND_STDOUT_TO_STDERR;
 -			opt |= RUN_HIDE_STDERR_ON_SUCCESS;
 +			cmd.stdout_to_stderr = 1;
 +			cmd.err = -1;
  		}
  
 -		env = read_author_script();
 -		if (!env) {
 +		if (read_env_script(&cmd.env_array)) {
  			const char *gpg_opt = gpg_sign_opt_quoted(opts);
  
  			return error(_(staged_changes_advice),
 @@ -648,39 +637,47 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
  		}
  	}
  
 -	argv_array_init(&array);
 -	argv_array_push(&array, "commit");
 -	argv_array_push(&array, "-n");
 +	argv_array_push(&cmd.args, "commit");
 +	argv_array_push(&cmd.args, "-n");
  
  	if (amend)
 -		argv_array_push(&array, "--amend");
 +		argv_array_push(&cmd.args, "--amend");
  	if (opts->gpg_sign)
 -		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 +		argv_array_pushf(&cmd.args, "-S%s", opts->gpg_sign);
  	if (opts->signoff)
 -		argv_array_push(&array, "-s");
 +		argv_array_push(&cmd.args, "-s");
  	if (defmsg)
 -		argv_array_pushl(&array, "-F", defmsg, NULL);
 +		argv_array_pushl(&cmd.args, "-F", defmsg, NULL);
  	if (cleanup_commit_message)
 -		argv_array_push(&array, "--cleanup=strip");
 +		argv_array_push(&cmd.args, "--cleanup=strip");
  	if (edit)
 -		argv_array_push(&array, "-e");
 +		argv_array_push(&cmd.args, "-e");
  	else if (!cleanup_commit_message &&
  		 !opts->signoff && !opts->record_origin &&
  		 git_config_get_value("commit.cleanup", &value))
 -		argv_array_push(&array, "--cleanup=verbatim");
 +		argv_array_push(&cmd.args, "--cleanup=verbatim");
  
  	if (allow_empty)
 -		argv_array_push(&array, "--allow-empty");
 +		argv_array_push(&cmd.args, "--allow-empty");
  
  	if (opts->allow_empty_message)
 -		argv_array_push(&array, "--allow-empty-message");
 +		argv_array_push(&cmd.args, "--allow-empty-message");
  
 -	rc = run_command_v_opt_cd_env(array.argv, opt, NULL,
 -			(const char *const *)env);
 -	argv_array_clear(&array);
 -	free(env);
 +	if (cmd.err == -1) {
 +		/* hide stderr on success */
 +		struct strbuf buf = STRBUF_INIT;
 +		int rc = pipe_command(&cmd,
 +				      NULL, 0,
 +				      /* stdout is already redirected */
 +				      NULL, 0,
 +				      &buf, 0);
 +		if (rc)
 +			fputs(buf.buf, stderr);
 +		strbuf_release(&buf);
 +		return rc;
 +	}
  
 -	return rc;
 +	return run_command(&cmd);
  }
  
  static int is_original_commit_empty(struct commit *commit)
 @@ -786,6 +783,11 @@ static const char *command_to_string(const enum todo_command command)
  	die("Unknown command: %d", command);
  }
  
 +static int is_noop(const enum todo_command command)
 +{
 +	return TODO_NOOP <= command;
 +}
 +
  static int is_fixup(enum todo_command command)
  {
  	return command == TODO_FIXUP || command == TODO_SQUASH;
 @@ -799,36 +801,33 @@ static int update_squash_messages(enum todo_command command,
  	const char *message, *body;
  
  	if (file_exists(rebase_path_squash_msg())) {
 -		char *p, *p2;
 +		struct strbuf header = STRBUF_INIT;
 +		char *eol, *p;
  
  		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
  			return error(_("could not read '%s'"),
  				rebase_path_squash_msg());
  
 +		p = buf.buf + 1;
 +		eol = strchrnul(buf.buf, '\n');
  		if (buf.buf[0] != comment_line_char ||
 -		    !skip_prefix(buf.buf + 1, " This is a combination of ",
 -				 (const char **)&p))
 +		    (p += strcspn(p, "0123456789\n")) == eol)
  			return error(_("unexpected 1st line of squash message:"
  				       "\n\n\t%.*s"),
 -				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
 -				     buf.buf);
 -		count = strtol(p, &p2, 10);
 +				     (int)(eol - buf.buf), buf.buf);
 +		count = strtol(p, NULL, 10);
  
 -		if (count < 1 || *p2 != ' ')
 +		if (count < 1)
  			return error(_("invalid 1st line of squash message:\n"
  				       "\n\t%.*s"),
 -				     (int)(strchrnul(buf.buf, '\n') - buf.buf),
 -				     buf.buf);
 -
 -		sprintf((char *)p, "%d", ++count);
 -		if (!*p2)
 -			*p2 = ' ';
 -		else {
 -			*(++p2) = 'c';
 -			strbuf_insert(&buf, p2 - buf.buf, " ", 1);
 -		}
 -	}
 -	else {
 +				     (int)(eol - buf.buf), buf.buf);
 +
 +		strbuf_addf(&header, "%c ", comment_line_char);
 +		strbuf_addf(&header,
 +			    _("This is a combination of %d commits."), ++count);
 +		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
 +		strbuf_release(&header);
 +	} else {
  		unsigned char head[20];
  		struct commit *head_commit;
  		const char *head_message, *body;
 @@ -840,20 +839,22 @@ static int update_squash_messages(enum todo_command command,
  		if (!(head_message = get_commit_buffer(head_commit, NULL)))
  			return error(_("could not read HEAD's commit message"));
  
 -		body = strstr(head_message, "\n\n");
 -		if (!body)
 -			body = "";
 -		else
 -			body = skip_blank_lines(body + 2);
 +		find_commit_subject(head_message, &body);
  		if (write_message(body, strlen(body),
 -				  rebase_path_fixup_msg(), 0))
 +				  rebase_path_fixup_msg(), 0)) {
 +			unuse_commit_buffer(head_commit, head_message);
  			return error(_("cannot write '%s'"),
  				     rebase_path_fixup_msg());
 +		}
  
  		count = 2;
 -		strbuf_addf(&buf, _("%c This is a combination of 2 commits.\n"
 -				    "%c The first commit's message is:\n\n%s"),
 -			    comment_line_char, comment_line_char, body);
 +		strbuf_addf(&buf, "%c ", comment_line_char);
 +		strbuf_addf(&buf, _("This is a combination of %d commits."),
 +			    count);
 +		strbuf_addf(&buf, "\n%c ", comment_line_char);
 +		strbuf_addstr(&buf, _("This is the 1st commit message:"));
 +		strbuf_addstr(&buf, "\n\n");
 +		strbuf_addstr(&buf, body);
  
  		unuse_commit_buffer(head_commit, head_message);
  	}
 @@ -861,25 +862,21 @@ static int update_squash_messages(enum todo_command command,
  	if (!(message = get_commit_buffer(commit, NULL)))
  		return error(_("could not read commit message of %s"),
  			     oid_to_hex(&commit->object.oid));
 -	body = strstr(message, "\n\n");
 -	if (!body)
 -		body = "";
 -	else
 -		body = skip_blank_lines(body + 2);
 +	find_commit_subject(message, &body);
  
  	if (command == TODO_SQUASH) {
  		unlink(rebase_path_fixup_msg());
 -		strbuf_addf(&buf, _("\n%c This is the commit message #%d:\n"
 -				    "\n%s"),
 -			    comment_line_char, count, body);
 -	}
 -	else if (command == TODO_FIXUP) {
 -		strbuf_addf(&buf, _("\n%c The commit message #%d "
 -				    "will be skipped:\n\n"),
 -			    comment_line_char, count);
 +		strbuf_addf(&buf, "\n%c ", comment_line_char);
 +		strbuf_addf(&buf, _("This is the commit message #%d:"), count);
 +		strbuf_addstr(&buf, "\n\n");
 +		strbuf_addstr(&buf, body);
 +	} else if (command == TODO_FIXUP) {
 +		strbuf_addf(&buf, "\n%c ", comment_line_char);
 +		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
 +			    count);
 +		strbuf_addstr(&buf, "\n\n");
  		strbuf_add_commented_lines(&buf, body, strlen(body));
 -	}
 -	else
 +	} else
  		return error(_("unknown command: %d"), command);
  	unuse_commit_buffer(commit, message);
  
 @@ -956,9 +953,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	}
  	discard_cache();
  
 -	if (!commit->parents) {
 +	if (!commit->parents)
  		parent = NULL;
 -	}
  	else if (commit->parents->next) {
  		/* Reverting or cherry-picking a merge commit */
  		int cnt;
 @@ -1036,14 +1032,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		next = commit;
  		next_label = msg.label;
  
 -		/*
 -		 * Append the commit log message to msgbuf; it starts
 -		 * after the tree, parent, author, committer
 -		 * information followed by "\n\n".
 -		 */
 -		p = strstr(msg.message, "\n\n");
 -		if (p)
 -			strbuf_addstr(&msgbuf, skip_blank_lines(p + 2));
 +		/* Append the commit log message to msgbuf. */
 +		if (find_commit_subject(msg.message, &p))
 +			strbuf_addstr(&msgbuf, p);
  
  		if (opts->record_origin) {
  			if (!has_conforming_footer(&msgbuf, NULL, 0))
 @@ -1065,8 +1056,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		else if (file_exists(rebase_path_fixup_msg())) {
  			cleanup_commit_message = 1;
  			msg_file = rebase_path_fixup_msg();
 -		}
 -		else {
 +		} else {
  			const char *dest = git_path("SQUASH_MSG");
  			unlink(dest);
  			if (copy_file(dest, rebase_path_squash_msg(), 0666))
 @@ -1241,8 +1231,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
  			item->command = i;
  			break;
 -		}
 -		else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
 +		} else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
  			bol++;
  			item->command = i;
  			break;
 @@ -1312,7 +1301,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  		else if (is_fixup(item->command))
  			return error(_("cannot '%s' without a previous commit"),
  				command_to_string(item->command));
 -		else if (item->command < TODO_NOOP)
 +		else if (!is_noop(item->command))
  			fixup_okay = 1;
  	}
  
 @@ -1533,8 +1522,7 @@ static int create_seq_dir(void)
  		error(_("a cherry-pick or revert is already in progress"));
  		advise(_("try \"git cherry-pick (--continue | --quit | --abort)\""));
  		return -1;
 -	}
 -	else if (mkdir(git_path_seq_dir(), 0777) < 0)
 +	} else if (mkdir(git_path_seq_dir(), 0777) < 0)
  		return error_errno(_("could not create sequencer directory '%s'"),
  				   git_path_seq_dir());
  	return 0;
 @@ -1667,6 +1655,10 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
  	const char *todo_path = get_todo_path(opts);
  	int next = todo_list->current, offset, fd;
  
 +	/*
 +	 * rebase -i writes "git-rebase-todo" without the currently executing
 +	 * command, appending it to "done" instead.
 +	 */
  	if (is_rebase_i(opts))
  		next++;
  
 @@ -1739,7 +1731,7 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  {
  	struct strbuf buf = STRBUF_INIT;
  	struct rev_info log_tree_opt;
 -	const char *commit_buffer = get_commit_buffer(commit, NULL), *subject, *p;
 +	const char *subject, *p;
  	int res = 0;
  
  	p = short_commit_name(commit);
 @@ -1766,6 +1758,7 @@ static int make_patch(struct commit *commit, struct replay_opts *opts)
  
  	strbuf_addf(&buf, "%s/message", get_dir(opts));
  	if (!file_exists(buf.buf)) {
 +		const char *commit_buffer = get_commit_buffer(commit, NULL);
  		find_commit_subject(commit_buffer, &subject);
  		res |= write_message(subject, strlen(subject), buf.buf, 1);
  		unuse_commit_buffer(commit, commit_buffer);
 @@ -1805,8 +1798,7 @@ static int error_with_patch(struct commit *commit,
  			"Once you are satisfied with your changes, run\n"
  			"\n"
  			"  git rebase --continue\n", gpg_sign_opt_quoted(opts));
 -	}
 -	else if (exit_code)
 +	} else if (exit_code)
  		fprintf(stderr, "Could not apply %s... %.*s\n",
  			short_commit_name(commit), subject_len, subject);
  
 @@ -1854,8 +1846,7 @@ static int do_exec(const char *command_line)
  		if (status == 127)
  			/* command not found */
  			status = 1;
 -	}
 -	else if (dirty) {
 +	} else if (dirty) {
  		warning(_("execution succeeded: %s\nbut "
  			  "left changes to the index and/or the working tree\n"
  			  "Commit or stash your changes, and then run\n"
 @@ -1878,7 +1869,7 @@ static int is_final_fixup(struct todo_list *todo_list)
  	while (++i < todo_list->nr)
  		if (is_fixup(todo_list->items[i].command))
  			return 0;
 -		else if (todo_list->items[i].command < TODO_NOOP)
 +		else if (!is_noop(todo_list->items[i].command))
  			break;
  	return 1;
  }
 @@ -1888,7 +1879,7 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
  	int i;
  
  	for (i = todo_list->current + offset; i < todo_list->nr; i++)
 -		if (todo_list->items[i].command < TODO_NOOP)
 +		if (!is_noop(todo_list->items[i].command))
  			return todo_list->items[i].command;
  
  	return -1;
 @@ -2021,21 +2012,18 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  					intend_to_amend();
  				return error_failed_squash(item->commit, opts,
  					item->arg_len, item->arg);
 -			}
 -			else if (res && is_rebase_i(opts))
 +			} else if (res && is_rebase_i(opts))
  				return res | error_with_patch(item->commit,
  					item->arg, item->arg_len, opts, res,
  					item->command == TODO_REWORD);
 -		}
 -		else if (item->command == TODO_EXEC) {
 +		} else if (item->command == TODO_EXEC) {
  			char *end_of_arg = (char *)(item->arg + item->arg_len);
  			int saved = *end_of_arg;
  
  			*end_of_arg = '\0';
  			res = do_exec(item->arg);
  			*end_of_arg = saved;
 -		}
 -		else if (item->command < TODO_NOOP)
 +		} else if (!is_noop(item->command))
  			return error(_("unknown command %d"), item->command);
  
  		todo_list->current++;
 @@ -2055,40 +2043,60 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
  				starts_with(head_ref.buf, "refs/")) {
  			const char *msg;
  			unsigned char head[20], orig[20];
 -
 -			if (get_sha1("HEAD", head))
 -				return error(_("cannot read HEAD"));
 +			int res;
 +
 +			if (get_sha1("HEAD", head)) {
 +				res = error(_("cannot read HEAD"));
 +cleanup_head_ref:
 +				strbuf_release(&head_ref);
 +				strbuf_release(&buf);
 +				return res;
 +			}
  			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
 -					get_sha1_hex(buf.buf, orig))
 -				return error(_("could not read orig-head"));
 -			if (!read_oneliner(&buf, rebase_path_onto(), 0))
 -				return error(_("could not read 'onto'"));
 +					get_sha1_hex(buf.buf, orig)) {
 +				res = error(_("could not read orig-head"));
 +				goto cleanup_head_ref;
 +			}
 +			if (!read_oneliner(&buf, rebase_path_onto(), 0)) {
 +				res = error(_("could not read 'onto'"));
 +				goto cleanup_head_ref;
 +			}
  			msg = reflog_message(opts, "finish", "%s onto %s",
  				head_ref.buf, buf.buf);
  			if (update_ref(msg, head_ref.buf, head, orig,
 -					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 -				return error(_("could not update %s"),
 +					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR)) {
 +				res = error(_("could not update %s"),
  					head_ref.buf);
 +				goto cleanup_head_ref;
 +			}
  			msg = reflog_message(opts, "finish", "returning to %s",
  				head_ref.buf);
 -			if (create_symref("HEAD", head_ref.buf, msg))
 -				return error(_("could not update HEAD to %s"),
 +			if (create_symref("HEAD", head_ref.buf, msg)) {
 +				res = error(_("could not update HEAD to %s"),
  					head_ref.buf);
 +				goto cleanup_head_ref;
 +			}
  			strbuf_reset(&buf);
  		}
  
  		if (opts->verbose) {
 -			const char *argv[] = {
 -				"diff-tree", "--stat", NULL, NULL
 -			};
 -
 -			if (!read_oneliner(&buf, rebase_path_orig_head(), 0))
 -				return error(_("could not read '%s'"),
 -					rebase_path_orig_head());
 -			strbuf_addstr(&buf, "..HEAD");
 -			argv[2] = buf.buf;
 -			run_command_v_opt(argv, RUN_GIT_CMD);
 -			strbuf_reset(&buf);
 +			struct rev_info log_tree_opt;
 +			struct object_id orig, head;
 +
 +			memset(&log_tree_opt, 0, sizeof(log_tree_opt));
 +			init_revisions(&log_tree_opt, NULL);
 +			log_tree_opt.diff = 1;
 +			log_tree_opt.diffopt.output_format =
 +				DIFF_FORMAT_DIFFSTAT;
 +			log_tree_opt.disable_stdin = 1;
 +
 +			if (read_oneliner(&buf, rebase_path_orig_head(), 0) &&
 +			    !get_sha1(buf.buf, orig.hash) &&
 +			    !get_sha1("HEAD", head.hash)) {
 +				diff_tree_sha1(orig.hash, head.hash,
 +					       "", &log_tree_opt.diffopt);
 +				log_tree_diff_flush(&log_tree_opt);
 +			}
  		}
  		flush_rewritten_pending();
  		if (!stat(rebase_path_rewritten_list(), &st) &&
 @@ -2195,8 +2203,7 @@ int sequencer_continue(struct replay_opts *opts)
  	if (is_rebase_i(opts)) {
  		if (commit_staged_changes(opts))
  			return -1;
 -	}
 -	else if (!file_exists(get_todo_path(opts)))
 +	} else if (!file_exists(get_todo_path(opts)))
  		return continue_single_pick();
  	if (read_populate_opts(opts))
  		return -1;
 @@ -2216,8 +2223,7 @@ int sequencer_continue(struct replay_opts *opts)
  			goto release_todo_list;
  		}
  		todo_list.current++;
 -	}
 -	else if (file_exists(rebase_path_stopped_sha())) {
 +	} else if (file_exists(rebase_path_stopped_sha())) {
  		struct strbuf buf = STRBUF_INIT;
  		struct object_id oid;
  
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:26 UTC · re: Johannes Schindelin · lore

[PATCH v3 02/38] sequencer: move "else" keyword onto the same line as preceding brace

It is the current coding style of the Git project to write
	if (...) {
		...
	} else {
		...
	}

instead of putting the closing brace and the "else" keyword on separate lines.

Pointed out by Junio Hamano.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Show changes to sequencer.c +1 −2
diff --git a/sequencer.c b/sequencer.c
index 23793db08b..3eededcb98 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1070,8 +1070,7 @@ static int create_seq_dir(void)
 		error(_("a cherry-pick or revert is already in progress"));
 		advise(_("try \"git cherry-pick (--continue | --quit | --abort)\""));
 		return -1;
-	}
-	else if (mkdir(git_path_seq_dir(), 0777) < 0)
+	} else if (mkdir(git_path_seq_dir(), 0777) < 0)
 		return error_errno(_("could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Jan 12, 2017, 18:49 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 02/38] sequencer: move "else" keyword onto the same line as preceding brace

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 12 quoted lines
> It is the current coding style of the Git project to write
>
> 	if (...) {
> 		...
> 	} else {
> 		...
> 	}
>
> instead of putting the closing brace and the "else" keyword on separate
> lines.
>
> Pointed out by Junio Hamano.

For a small thing like this, the attribution is mostly irrelevant, but for the record, the credit should go to checkpatch.pl from the kernel project ;-).

I'll try to squash the part that was touched by 01/38 into this patch locally; even though I haven't finished going through the individual patches, I expect that my conclusion would be what I said in <xmqqinpnuaxl.fsf@gitster.mtv.corp.google.com>, i.e. the changes in the series are mostly good and improved relative to the previous round except for the parts reading and writing of author-script (from the maintainability's point of view).

As I do not think we want to see another reroll of three-dozen patches, I am leaning towards merging v3 (with minimum fixes like squashing 01/ and 02/ into one) to 'next' and cook it there and fix remaining issues incrementally while the series is in 'next'.

I may have to change my mind after I read through the remaining patches (I've done with the first dozen or so at this moment), but I do not expect that, judging from what I saw in the interdiff.

Thanks.
Show 19 quoted lines
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 23793db08b..3eededcb98 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1070,8 +1070,7 @@ static int create_seq_dir(void)
>  		error(_("a cherry-pick or revert is already in progress"));
>  		advise(_("try \"git cherry-pick (--continue | --quit | --abort)\""));
>  		return -1;
> -	}
> -	else if (mkdir(git_path_seq_dir(), 0777) < 0)
> +	} else if (mkdir(git_path_seq_dir(), 0777) < 0)
>  		return error_errno(_("could not create sequencer directory '%s'"),
>  				   git_path_seq_dir());
>  	return 0;
Johannes Schindelin· Jan 2, 2017, 15:26 UTC · re: Johannes Schindelin · lore

[PATCH v3 01/38] sequencer: avoid unnecessary curly braces

This was noticed while addressing Junio Hamano's concern that some "else" operators were on separate lines than the preceding closing brace.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Show changes to sequencer.c +1 −2
diff --git a/sequencer.c b/sequencer.c
index 9adb7bbf1d..23793db08b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -632,9 +632,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	discard_cache();
 
-	if (!commit->parents) {
+	if (!commit->parents)
 		parent = NULL;
-	}
 	else if (commit->parents->next) {
 		/* Reverting or cherry-picking a merge commit */
 		int cnt;
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Jan 12, 2017, 18:35 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 01/38] sequencer: avoid unnecessary curly braces

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 8 quoted lines
>  
> -	if (!commit->parents) {
> +	if (!commit->parents)
>  		parent = NULL;
> -	}
>  	else if (commit->parents->next) {
>  		/* Reverting or cherry-picking a merge commit */
>  		int cnt;
The result becomes
	if (...)
		single statement;
	else if (...) {
		multiple;
                statements;
        }
which is not quite an improvement.  

The preferred style is for all arms in if/elseif/else cascade to either use or not use brace pairs, so I think a fix toward that goal would be more like:

	if (!commit->parents) {
		parent = NULL;
-	}                
-	else if (commit->parents->next) {
+	} else if (commit->parents->next) {
		/* Reverting ...
Johannes Schindelin· Jan 14, 2017, 17:57 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 01/38] sequencer: avoid unnecessary curly braces

Hi Junio,
On Thu, 12 Jan 2017, Junio C Hamano wrote:
Show 21 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> >  
> > -	if (!commit->parents) {
> > +	if (!commit->parents)
> >  		parent = NULL;
> > -	}
> >  	else if (commit->parents->next) {
> >  		/* Reverting or cherry-picking a merge commit */
> >  		int cnt;
> 
> The result becomes
> 
> 	if (...)
> 		single statement;
> 	else if (...) {
> 		multiple;
>                 statements;
>         }
> 
> which is not quite an improvement.  

Yet, this used to be the coding style of Git, and your statement comes quite as a surprise to me.

Ciao, Johannes

Jeff King· Jan 14, 2017, 18:05 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 01/38] sequencer: avoid unnecessary curly braces

On Sat, Jan 14, 2017 at 06:57:13PM +0100, Johannes Schindelin wrote:
Show 26 quoted lines
> On Thu, 12 Jan 2017, Junio C Hamano wrote:
> 
> > Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> > 
> > >  
> > > -	if (!commit->parents) {
> > > +	if (!commit->parents)
> > >  		parent = NULL;
> > > -	}
> > >  	else if (commit->parents->next) {
> > >  		/* Reverting or cherry-picking a merge commit */
> > >  		int cnt;
> > 
> > The result becomes
> > 
> > 	if (...)
> > 		single statement;
> > 	else if (...) {
> > 		multiple;
> >                 statements;
> >         }
> > 
> > which is not quite an improvement.  
> 
> Yet, this used to be the coding style of Git, and your statement comes
> quite as a surprise to me.
Yeah, I thought we were OK with:
  if (cond)
	single statement;
  else {
	multiple;
	statements;
  }
but not the other way around:
  if (cond) {
	multiple;
	statements;
  } else
	single statement;

I don't know if the "else if" changes that or not, but I certainly have written things like your patch does.

-Peff
Jacob Keller· Jan 16, 2017, 04:10 UTC · re: Jeff King · lore

Re: [PATCH v3 01/38] sequencer: avoid unnecessary curly braces

On Sat, Jan 14, 2017 at 10:05 AM, Jeff King <peff@peff.net> wrote:
Show 50 quoted lines
> On Sat, Jan 14, 2017 at 06:57:13PM +0100, Johannes Schindelin wrote:
>
>> On Thu, 12 Jan 2017, Junio C Hamano wrote:
>>
>> > Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> >
>> > >
>> > > - if (!commit->parents) {
>> > > + if (!commit->parents)
>> > >           parent = NULL;
>> > > - }
>> > >   else if (commit->parents->next) {
>> > >           /* Reverting or cherry-picking a merge commit */
>> > >           int cnt;
>> >
>> > The result becomes
>> >
>> >     if (...)
>> >             single statement;
>> >     else if (...) {
>> >             multiple;
>> >                 statements;
>> >         }
>> >
>> > which is not quite an improvement.
>>
>> Yet, this used to be the coding style of Git, and your statement comes
>> quite as a surprise to me.
>
> Yeah, I thought we were OK with:
>
>   if (cond)
>         single statement;
>   else {
>         multiple;
>         statements;
>   }
>
> but not the other way around:
>
>   if (cond) {
>         multiple;
>         statements;
>   } else
>         single statement;
>
> I don't know if the "else if" changes that or not, but I certainly have
> written things like your patch does.
>
> -Peff

Personally, I am of the faith that if any branch of the if-then-else-if-then-else blocks needs braces, then all branches should use braces. However, I think that

if (condition)
  <line>
else {
  <block>
}

is reasonably close to this, as the main part can still clearly see the if condition pretty well.

Thanks, Jake

Johannes Schindelin· Jan 2, 2017, 15:26 UTC · re: Johannes Schindelin · lore

[PATCH v3 03/38] sequencer: use a helper to find the commit message

It is actually not safe to look for a commit message by looking for the first empty line and skipping it.

The find_commit_subject() function looks more carefully, so let's use it. Since we are interested in the entire commit message, we re-compute the string length after verifying that the commit subject is not empty (in which case the entire commit message would be empty, something that should not happen but that we want to handle gracefully).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
Show changes to sequencer.c +3 −8
diff --git a/sequencer.c b/sequencer.c
index 3eededcb98..720857beda 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -703,14 +703,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		next = commit;
 		next_label = msg.label;
 
-		/*
-		 * Append the commit log message to msgbuf; it starts
-		 * after the tree, parent, author, committer
-		 * information followed by "\n\n".
-		 */
-		p = strstr(msg.message, "\n\n");
-		if (p)
-			strbuf_addstr(&msgbuf, skip_blank_lines(p + 2));
+		/* Append the commit log message to msgbuf. */
+		if (find_commit_subject(msg.message, &p))
+			strbuf_addstr(&msgbuf, p);
 
 		if (opts->record_origin) {
 			if (!has_conforming_footer(&msgbuf, NULL, 0))
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Jan 12, 2017, 18:36 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 03/38] sequencer: use a helper to find the commit message

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 11 quoted lines
> It is actually not safe to look for a commit message by looking for the
> first empty line and skipping it.
>
> The find_commit_subject() function looks more carefully, so let's use
> it. Since we are interested in the entire commit message, we re-compute
> the string length after verifying that the commit subject is not empty
> (in which case the entire commit message would be empty, something that
> should not happen but that we want to handle gracefully).
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
Very sensible.
Show 25 quoted lines
>  sequencer.c | 11 +++--------
>  1 file changed, 3 insertions(+), 8 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 3eededcb98..720857beda 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -703,14 +703,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		next = commit;
>  		next_label = msg.label;
>  
> -		/*
> -		 * Append the commit log message to msgbuf; it starts
> -		 * after the tree, parent, author, committer
> -		 * information followed by "\n\n".
> -		 */
> -		p = strstr(msg.message, "\n\n");
> -		if (p)
> -			strbuf_addstr(&msgbuf, skip_blank_lines(p + 2));
> +		/* Append the commit log message to msgbuf. */
> +		if (find_commit_subject(msg.message, &p))
> +			strbuf_addstr(&msgbuf, p);
>  
>  		if (opts->record_origin) {
>  			if (!has_conforming_footer(&msgbuf, NULL, 0))
Johannes Schindelin· Jan 2, 2017, 15:26 UTC · re: Johannes Schindelin · lore

[PATCH v3 04/38] sequencer: support a new action: 'interactive rebase'

This patch introduces a new action for the sequencer. It really does not do a whole lot of its own right now, but lays the ground work for patches to come. The intention, of course, is to finally make the sequencer the work horse of the interactive rebase (the original idea behind the "sequencer" concept).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 36 ++++++++++++++++++++++++++++++++----
 sequencer.h |  3 ++-
 2 files changed, 34 insertions(+), 5 deletions(-)
Show changes to 2 files +34 −5

sequencer.c, sequencer.h

diff --git a/sequencer.c b/sequencer.c
index 720857beda..690460bc67 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -30,6 +30,14 @@ static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 static GIT_PATH_FUNC(git_path_abort_safety_file, "sequencer/abort-safety")
 
+static GIT_PATH_FUNC(rebase_path, "rebase-merge")
+/*
+ * The file containing rebase commands, comments, and empty lines.
+ * This file is created by "git rebase -i" then edited by the user. As
+ * the lines are processed, they are removed from the front of this
+ * file and written to the tail of 'done'.
+ */
+static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
@@ -42,19 +50,22 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
-/* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
-	return 0;
+	return opts->action == REPLAY_INTERACTIVE_REBASE;
 }
 
 static const char *get_dir(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path();
 	return git_path_seq_dir();
 }
 
 static const char *get_todo_path(const struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return rebase_path_todo();
 	return git_path_todo_file();
 }
 
@@ -122,7 +133,15 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
+	switch (opts->action) {
+	case REPLAY_REVERT:
+		return N_("revert");
+	case REPLAY_PICK:
+		return N_("cherry-pick");
+	case REPLAY_INTERACTIVE_REBASE:
+		return N_("rebase -i");
+	}
+	die(_("Unknown action: %d"), opts->action);
 }
 
 struct commit_message {
@@ -364,7 +383,9 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 
 	if (active_cache_changed &&
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
-		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
+		/* TRANSLATORS: %s will be "revert", "cherry-pick" or
+		 * "rebase -i".
+		 */
 		return error(_("%s: Unable to write new index file"),
 			_(action_name(opts)));
 	rollback_lock_file(&index_lock);
@@ -1198,6 +1219,13 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 	const char *todo_path = get_todo_path(opts);
 	int next = todo_list->current, offset, fd;
 
+	/*
+	 * rebase -i writes "git-rebase-todo" without the currently executing
+	 * command, appending it to "done" instead.
+	 */
+	if (is_rebase_i(opts))
+		next++;
+
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("could not lock '%s'"), todo_path);
diff --git a/sequencer.h b/sequencer.h
index 7a513c576b..cb21cfddee 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -7,7 +7,8 @@ const char *git_path_seq_dir(void);
 
 enum replay_action {
 	REPLAY_REVERT,
-	REPLAY_PICK
+	REPLAY_PICK,
+	REPLAY_INTERACTIVE_REBASE
 };
 
 struct replay_opts {
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:26 UTC · re: Johannes Schindelin · lore

[PATCH v3 05/38] sequencer (rebase -i): implement the 'noop' command

The 'noop' command is probably the most boring of all rebase -i commands to support in the sequencer.

Which makes it an excellent candidate for this first stab to add support for rebase -i's commands to the sequencer.

For the moment, let's also treat empty lines and commented-out lines as 'noop'; We will refine that handling later in this patch series.

To make it easier to identify "classes" of todo_commands (such as: determine whether a command is pick-like, i.e. handles a single commit), let's enforce a certain order of said commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 39 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
Show changes to sequencer.c +36 −3
diff --git a/sequencer.c b/sequencer.c
index 690460bc67..84f18e64e9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -607,14 +607,23 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
+/*
+ * Note that ordering matters in this enum. Not only must it match the mapping
+ * below, it is also divided into several sections that matter.  When adding
+ * new commands, make sure you add it in the right section.
+ */
 enum todo_command {
+	/* commands that handle commits */
 	TODO_PICK = 0,
-	TODO_REVERT
+	TODO_REVERT,
+	/* commands that do nothing but are counted for reporting progress */
+	TODO_NOOP
 };
 
 static const char *todo_command_strings[] = {
 	"pick",
-	"revert"
+	"revert",
+	"noop"
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -624,6 +633,10 @@ static const char *command_to_string(const enum todo_command command)
 	die("Unknown command: %d", command);
 }
 
+static int is_noop(const enum todo_command command)
+{
+	return TODO_NOOP <= (size_t)command;
+}
 
 static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct replay_opts *opts)
@@ -879,6 +892,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	/* left-trim */
 	bol += strspn(bol, " \t");
 
+	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
+		item->command = TODO_NOOP;
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
@@ -887,6 +908,13 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	if (i >= ARRAY_SIZE(todo_command_strings))
 		return -1;
 
+	if (item->command == TODO_NOOP) {
+		item->commit = NULL;
+		item->arg = bol;
+		item->arg_len = eol - bol;
+		return 0;
+	}
+
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
@@ -1289,7 +1317,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(item->command, item->commit, opts);
+		if (item->command <= TODO_REVERT)
+			res = do_pick_commit(item->command, item->commit,
+					opts);
+		else if (!is_noop(item->command))
+			return error(_("unknown command %d"), item->command);
+
 		todo_list->current++;
 		if (res)
 			return res;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:26 UTC · re: Johannes Schindelin · lore

[PATCH v3 06/38] sequencer (rebase -i): implement the 'edit' command

This patch is a straight-forward reimplementation of the `edit` operation of the interactive rebase command.

Well, not *quite* straight-forward: when stopping, the `edit` command wants to write the `patch` file (which is not only the patch, but includes the commit message and author information). To that end, this patch requires the earlier work that taught the log-tree machinery to respect the `file` setting of rev_info->diffopt to write to a file stream different than stdout.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 114 insertions(+), 3 deletions(-)
Show changes to sequencer.c +114 −3
diff --git a/sequencer.c b/sequencer.c
index 84f18e64e9..b138a3906c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -17,6 +17,7 @@
 #include "argv-array.h"
 #include "quote.h"
 #include "trailer.h"
+#include "log-tree.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -45,6 +46,20 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
 /*
+ * When an "edit" rebase command is being processed, the SHA1 of the
+ * commit to be edited is recorded in this file.  When "git rebase
+ * --continue" is executed, if there are any staged changes then they
+ * will be amended to the HEAD commit, but only provided the HEAD
+ * commit is still the commit to be edited.  When any other rebase
+ * command is processed, this file is deleted.
+ */
+static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
+/*
+ * When we stop at a given patch via the "edit" command, this file contains
+ * the abbreviated commit name of the corresponding patch.
+ */
+static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -616,6 +631,7 @@ enum todo_command {
 	/* commands that handle commits */
 	TODO_PICK = 0,
 	TODO_REVERT,
+	TODO_EDIT,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP
 };
@@ -623,6 +639,7 @@ enum todo_command {
 static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
+	"edit",
 	"noop"
 };
 
@@ -1302,9 +1319,87 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
+static int make_patch(struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	struct rev_info log_tree_opt;
+	const char *subject, *p;
+	int res = 0;
+
+	p = short_commit_name(commit);
+	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
+		return -1;
+
+	strbuf_addf(&buf, "%s/patch", get_dir(opts));
+	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
+	init_revisions(&log_tree_opt, NULL);
+	log_tree_opt.abbrev = 0;
+	log_tree_opt.diff = 1;
+	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
+	log_tree_opt.disable_stdin = 1;
+	log_tree_opt.no_commit_id = 1;
+	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
+	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
+	if (!log_tree_opt.diffopt.file)
+		res |= error_errno(_("could not open '%s'"), buf.buf);
+	else {
+		res |= log_tree_commit(&log_tree_opt, commit);
+		fclose(log_tree_opt.diffopt.file);
+	}
+	strbuf_reset(&buf);
+
+	strbuf_addf(&buf, "%s/message", get_dir(opts));
+	if (!file_exists(buf.buf)) {
+		const char *commit_buffer = get_commit_buffer(commit, NULL);
+		find_commit_subject(commit_buffer, &subject);
+		res |= write_message(subject, strlen(subject), buf.buf, 1);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
+	strbuf_release(&buf);
+
+	return res;
+}
+
+static int intend_to_amend(void)
+{
+	unsigned char head[20];
+	char *p;
+
+	if (get_sha1("HEAD", head))
+		return error(_("cannot read HEAD"));
+
+	p = sha1_to_hex(head);
+	return write_message(p, strlen(p), rebase_path_amend(), 1);
+}
+
+static int error_with_patch(struct commit *commit,
+	const char *subject, int subject_len,
+	struct replay_opts *opts, int exit_code, int to_amend)
+{
+	if (make_patch(commit, opts))
+		return -1;
+
+	if (to_amend) {
+		if (intend_to_amend())
+			return -1;
+
+		fprintf(stderr, "You can amend the commit now, with\n"
+			"\n"
+			"  git commit --amend %s\n"
+			"\n"
+			"Once you are satisfied with your changes, run\n"
+			"\n"
+			"  git rebase --continue\n", gpg_sign_opt_quoted(opts));
+	} else if (exit_code)
+		fprintf(stderr, "Could not apply %s... %.*s\n",
+			short_commit_name(commit), subject_len, subject);
+
+	return exit_code;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	int res;
+	int res = 0;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	if (opts->allow_ff)
@@ -1317,10 +1412,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_REVERT)
+		if (item->command <= TODO_EDIT) {
 			res = do_pick_commit(item->command, item->commit,
 					opts);
-		else if (!is_noop(item->command))
+			if (item->command == TODO_EDIT) {
+				struct commit *commit = item->commit;
+				if (!res)
+					warning(_("stopped at %s... %.*s"),
+						short_commit_name(commit),
+						item->arg_len, item->arg);
+				return error_with_patch(commit,
+					item->arg, item->arg_len, opts, res,
+					!res);
+			}
+		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
 		todo_list->current++;
@@ -1328,6 +1433,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			return res;
 	}
 
+	if (is_rebase_i(opts)) {
+		/* Stopped in the middle, as planned? */
+		if (todo_list->current < todo_list->nr)
+			return 0;
+	}
+
 	/*
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Jan 12, 2017, 19:00 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 06/38] sequencer (rebase -i): implement the 'edit' command

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 40 quoted lines
> +static int make_patch(struct commit *commit, struct replay_opts *opts)
> +{
> +	struct strbuf buf = STRBUF_INIT;
> +	struct rev_info log_tree_opt;
> +	const char *subject, *p;
> +	int res = 0;
> +
> +	p = short_commit_name(commit);
> +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
> +		return -1;
> +
> +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
> +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
> +	init_revisions(&log_tree_opt, NULL);
> +	log_tree_opt.abbrev = 0;
> +	log_tree_opt.diff = 1;
> +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
> +	log_tree_opt.disable_stdin = 1;
> +	log_tree_opt.no_commit_id = 1;
> +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
> +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
> +	if (!log_tree_opt.diffopt.file)
> +		res |= error_errno(_("could not open '%s'"), buf.buf);
> +	else {
> +		res |= log_tree_commit(&log_tree_opt, commit);
> +		fclose(log_tree_opt.diffopt.file);
> +	}
> +	strbuf_reset(&buf);
> +
> +	strbuf_addf(&buf, "%s/message", get_dir(opts));
> +	if (!file_exists(buf.buf)) {
> +		const char *commit_buffer = get_commit_buffer(commit, NULL);
> +		find_commit_subject(commit_buffer, &subject);
> +		res |= write_message(subject, strlen(subject), buf.buf, 1);
> +		unuse_commit_buffer(commit, commit_buffer);
> +	}
> +	strbuf_release(&buf);
> +
> +	return res;
> +}

Unlike the scripted version, where a merge is shown with "diff --cc" and a root commit is shown as "Root commit", this only deals with a single-parent commit. Is this because this helper, at least in its current form, will not be used by "rebase -m" and not with "--root"?

If that is the case, that is perfectly fine, perhaps that deserves a mention in the log message and in-code comment before the function.

Johannes Schindelin· Jan 14, 2017, 17:56 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 06/38] sequencer (rebase -i): implement the 'edit' command

Hi Junio,
On Thu, 12 Jan 2017, Junio C Hamano wrote:
Show 46 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +static int make_patch(struct commit *commit, struct replay_opts *opts)
> > +{
> > +	struct strbuf buf = STRBUF_INIT;
> > +	struct rev_info log_tree_opt;
> > +	const char *subject, *p;
> > +	int res = 0;
> > +
> > +	p = short_commit_name(commit);
> > +	if (write_message(p, strlen(p), rebase_path_stopped_sha(), 1) < 0)
> > +		return -1;
> > +
> > +	strbuf_addf(&buf, "%s/patch", get_dir(opts));
> > +	memset(&log_tree_opt, 0, sizeof(log_tree_opt));
> > +	init_revisions(&log_tree_opt, NULL);
> > +	log_tree_opt.abbrev = 0;
> > +	log_tree_opt.diff = 1;
> > +	log_tree_opt.diffopt.output_format = DIFF_FORMAT_PATCH;
> > +	log_tree_opt.disable_stdin = 1;
> > +	log_tree_opt.no_commit_id = 1;
> > +	log_tree_opt.diffopt.file = fopen(buf.buf, "w");
> > +	log_tree_opt.diffopt.use_color = GIT_COLOR_NEVER;
> > +	if (!log_tree_opt.diffopt.file)
> > +		res |= error_errno(_("could not open '%s'"), buf.buf);
> > +	else {
> > +		res |= log_tree_commit(&log_tree_opt, commit);
> > +		fclose(log_tree_opt.diffopt.file);
> > +	}
> > +	strbuf_reset(&buf);
> > +
> > +	strbuf_addf(&buf, "%s/message", get_dir(opts));
> > +	if (!file_exists(buf.buf)) {
> > +		const char *commit_buffer = get_commit_buffer(commit, NULL);
> > +		find_commit_subject(commit_buffer, &subject);
> > +		res |= write_message(subject, strlen(subject), buf.buf, 1);
> > +		unuse_commit_buffer(commit, commit_buffer);
> > +	}
> > +	strbuf_release(&buf);
> > +
> > +	return res;
> > +}
> 
> Unlike the scripted version, where a merge is shown with "diff --cc"
> and a root commit is shown as "Root commit", this only deals with a
> single-parent commit.

Indeed. The reason is that we never encounter a merge commit (as we explicitly do not handle --preserve-merges) nor root commits (as we explicitly do not handle --root)

Ciao, Johannes

Johannes Schindelin· Jan 2, 2017, 15:26 UTC · re: Johannes Schindelin · lore

[PATCH v3 07/38] sequencer (rebase -i): implement the 'exec' command

The 'exec' command is a little special among rebase -i's commands, as it does *not* have a SHA-1 as first parameter. Instead, everything after the `exec` command is treated as command-line to execute.

Let's reuse the arg/arg_len fields of the todo_item structure (which hold the oneline for pick/edit commands) to point to the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
Show changes to sequencer.c +57 −0
diff --git a/sequencer.c b/sequencer.c
index b138a3906c..e9c10d7fe5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -18,6 +18,7 @@
 #include "quote.h"
 #include "trailer.h"
 #include "log-tree.h"
+#include "wt-status.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -632,6 +633,8 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	/* commands that do something else than handling a single commit */
+	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP
 };
@@ -640,6 +643,7 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"exec",
 	"noop"
 };
 
@@ -938,6 +942,12 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return -1;
 	bol += padding;
 
+	if (item->command == TODO_EXEC) {
+		item->arg = bol;
+		item->arg_len = (int)(eol - bol);
+		return 0;
+	}
+
 	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
@@ -1397,6 +1407,46 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int do_exec(const char *command_line)
+{
+	const char *child_argv[] = { NULL, NULL };
+	int dirty, status;
+
+	fprintf(stderr, "Executing: %s\n", command_line);
+	child_argv[0] = command_line;
+	status = run_command_v_opt(child_argv, RUN_USING_SHELL);
+
+	/* force re-reading of the cache */
+	if (discard_cache() < 0 || read_cache() < 0)
+		return error(_("could not read index"));
+
+	dirty = require_clean_work_tree("rebase", NULL, 1, 1);
+
+	if (status) {
+		warning(_("execution failed: %s\n%s"
+			  "You can fix the problem, and then run\n"
+			  "\n"
+			  "  git rebase --continue\n"
+			  "\n"),
+			command_line,
+			dirty ? N_("and made changes to the index and/or the "
+				"working tree\n") : "");
+		if (status == 127)
+			/* command not found */
+			status = 1;
+	} else if (dirty) {
+		warning(_("execution succeeded: %s\nbut "
+			  "left changes to the index and/or the working tree\n"
+			  "Commit or stash your changes, and then run\n"
+			  "\n"
+			  "  git rebase --continue\n"
+			  "\n"), command_line);
+		status = 1;
+	}
+
+	return status;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1425,6 +1475,13 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+		} else if (item->command == TODO_EXEC) {
+			char *end_of_arg = (char *)(item->arg + item->arg_len);
+			int saved = *end_of_arg;
+
+			*end_of_arg = '\0';
+			res = do_exec(item->arg);
+			*end_of_arg = saved;
 		} else if (!is_noop(item->command))
 			return error(_("unknown command %d"), item->command);
 
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:26 UTC · re: Johannes Schindelin · lore

[PATCH v3 08/38] sequencer (rebase -i): learn about the 'verbose' mode

When calling `git rebase -i -v`, the user wants to see some statistics after the commits were rebased. Let's show some.

The strbuf we use to perform that task will be used for other things in subsequent commits, hence it is declared and initialized in a wider scope than strictly needed here.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 28 ++++++++++++++++++++++++++++
 sequencer.h |  1 +
 2 files changed, 29 insertions(+)
Show changes to 2 files +29 −0

sequencer.c, sequencer.h

diff --git a/sequencer.c b/sequencer.c
index e9c10d7fe5..ddc4d144d7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -65,6 +65,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
  * command-line (and are only consumed, not modified, by the sequencer).
  */
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
+static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
+static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1088,6 +1090,9 @@ static int read_populate_opts(struct replay_opts *opts)
 		}
 		strbuf_release(&buf);
 
+		if (file_exists(rebase_path_verbose()))
+			opts->verbose = 1;
+
 		return 0;
 	}
 
@@ -1491,9 +1496,32 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
+
+		if (opts->verbose) {
+			struct rev_info log_tree_opt;
+			struct object_id orig, head;
+
+			memset(&log_tree_opt, 0, sizeof(log_tree_opt));
+			init_revisions(&log_tree_opt, NULL);
+			log_tree_opt.diff = 1;
+			log_tree_opt.diffopt.output_format =
+				DIFF_FORMAT_DIFFSTAT;
+			log_tree_opt.disable_stdin = 1;
+
+			if (read_oneliner(&buf, rebase_path_orig_head(), 0) &&
+			    !get_sha1(buf.buf, orig.hash) &&
+			    !get_sha1("HEAD", head.hash)) {
+				diff_tree_sha1(orig.hash, head.hash,
+					       "", &log_tree_opt.diffopt);
+				log_tree_diff_flush(&log_tree_opt);
+			}
+		}
+		strbuf_release(&buf);
 	}
 
 	/*
diff --git a/sequencer.h b/sequencer.h
index cb21cfddee..f885b68395 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -24,6 +24,7 @@ struct replay_opts {
 	int allow_empty;
 	int allow_empty_message;
 	int keep_redundant_commits;
+	int verbose;
 
 	int mainline;
 
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 09/38] sequencer (rebase -i): write the 'done' file

In the interactive rebase, commands that were successfully processed are not simply discarded, but appended to the 'done' file instead. This is used e.g. to display the current state to the user in the output of `git status` or the progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
Show changes to sequencer.c +23 −0
diff --git a/sequencer.c b/sequencer.c
index ddc4d144d7..8ea3d6aa94 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -41,6 +41,12 @@ static GIT_PATH_FUNC(rebase_path, "rebase-merge")
  */
 static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
 /*
+ * The rebase command lines that have already been processed. A line
+ * is moved here when it is first handled, before any associated user
+ * actions.
+ */
+static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -1296,6 +1302,23 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
 		return error(_("failed to finalize '%s'."), todo_path);
+
+	if (is_rebase_i(opts)) {
+		const char *done_path = rebase_path_done();
+		int fd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);
+		int prev_offset = !next ? 0 :
+			todo_list->items[next - 1].offset_in_buf;
+
+		if (fd >= 0 && offset > prev_offset &&
+		    write_in_full(fd, todo_list->buf.buf + prev_offset,
+				  offset - prev_offset) < 0) {
+			close(fd);
+			return error_errno(_("could not write to '%s'"),
+					   done_path);
+		}
+		if (fd >= 0)
+			close(fd);
+	}
 	return 0;
 }
 
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 11/38] sequencer (rebase -i): implement the short commands

For users' convenience, most rebase commands can be abbreviated, e.g. 'p' instead of 'pick' and 'x' instead of 'exec'. Let's teach the sequencer to handle those abbreviated commands just fine.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)
Show changes to sequencer.c +21 −14
diff --git a/sequencer.c b/sequencer.c
index 6a939a10bd..29b944d724 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -678,20 +678,23 @@ enum todo_command {
 	TODO_NOOP
 };
 
-static const char *todo_command_strings[] = {
-	"pick",
-	"revert",
-	"edit",
-	"fixup",
-	"squash",
-	"exec",
-	"noop"
+static struct {
+	char c;
+	const char *str;
+} todo_command_info[] = {
+	{ 'p', "pick" },
+	{ 0,   "revert" },
+	{ 'e', "edit" },
+	{ 'f', "fixup" },
+	{ 's', "squash" },
+	{ 'x', "exec" },
+	{ 0,   "noop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if ((size_t)command < ARRAY_SIZE(todo_command_strings))
-		return todo_command_strings[command];
+	if ((size_t)command < ARRAY_SIZE(todo_command_info))
+		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
 
@@ -1087,12 +1090,16 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
-		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
+			item->command = i;
+			break;
+		} else if (bol[1] == ' ' && *bol == todo_command_info[i].c) {
+			bol++;
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_strings))
+	if (i >= ARRAY_SIZE(todo_command_info))
 		return -1;
 
 	if (item->command == TODO_NOOP) {
@@ -1287,7 +1294,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 {
 	enum todo_command command = opts->action == REPLAY_PICK ?
 		TODO_PICK : TODO_REVERT;
-	const char *command_string = todo_command_strings[command];
+	const char *command_string = todo_command_info[command].str;
 	struct commit *commit;
 
 	if (prepare_revs(opts))
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 10/38] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands

This is a huge patch, and at the same time a huge step forward to execute the performance-critical parts of the interactive rebase in a builtin command.

Since 'fixup' and 'squash' are not only similar, but also need to know about each other (we want to reduce a series of fixups/squashes into a single, final commit message edit, from the user's point of view), we really have to implement them both at the same time.

Most of the actual work is done by the existing code path that already handles the "pick" and the "edit" commands; We added support for other features (e.g. to amend the commit message) in the patches leading up to this one, yet there are still quite a few bits in this patch that simply would not make sense as individual patches (such as: determining whether there was anything to "fix up" in the "todo" script, etc).

In theory, it would be possible to reuse the fast-forward code path also for the fixup and the squash code paths, but in practice this would make the code less readable. The end result cannot be fast-forwarded anyway, therefore let's just extend the cherry-picking code path for now.

Since the sequencer parses the entire `git-rebase-todo` script in one go, fixup or squash commands without a preceding pick can be reported early (in git-rebase--interactive, we could only report such errors just before executing the fixup/squash).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 227 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 217 insertions(+), 10 deletions(-)
Show changes to sequencer.c +217 −10
diff --git a/sequencer.c b/sequencer.c
index 8ea3d6aa94..6a939a10bd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -47,6 +47,35 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The commit message that is planned to be used for any changes that
+ * need to be committed following a user interaction.
+ */
+static GIT_PATH_FUNC(rebase_path_message, "rebase-merge/message")
+/*
+ * The file into which is accumulated the suggested commit message for
+ * squash/fixup commands. When the first of a series of squash/fixups
+ * is seen, the file is created and the commit message from the
+ * previous commit and from the first squash/fixup commit are written
+ * to it. The commit message for each subsequent squash/fixup commit
+ * is appended to the file as it is processed.
+ *
+ * The first line of the file is of the form
+ *     # This is a combination of $count commits.
+ * where $count is the number of commits whose messages have been
+ * written to the file so far (including the initial "pick" commit).
+ * Each time that a commit message is processed, this line is read and
+ * updated. It is deleted just before the combined commit is made.
+ */
+static GIT_PATH_FUNC(rebase_path_squash_msg, "rebase-merge/message-squash")
+/*
+ * If the current series of squash/fixups has not yet included a squash
+ * command, then this file exists and holds the commit message of the
+ * original "pick" commit.  (If the series ends without a "squash"
+ * command, then this can be used as the commit message of the combined
+ * commit without opening the editor.)
+ */
+static GIT_PATH_FUNC(rebase_path_fixup_msg, "rebase-merge/message-fixup")
+/*
  * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
  * GIT_AUTHOR_DATE that will be used for the commit that is currently
  * being rebased.
@@ -641,6 +670,8 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_FIXUP,
+	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
@@ -651,6 +682,8 @@ static const char *todo_command_strings[] = {
 	"pick",
 	"revert",
 	"edit",
+	"fixup",
+	"squash",
 	"exec",
 	"noop"
 };
@@ -667,15 +700,114 @@ static int is_noop(const enum todo_command command)
 	return TODO_NOOP <= (size_t)command;
 }
 
+static int is_fixup(enum todo_command command)
+{
+	return command == TODO_FIXUP || command == TODO_SQUASH;
+}
+
+static int update_squash_messages(enum todo_command command,
+		struct commit *commit, struct replay_opts *opts)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int count, res;
+	const char *message, *body;
+
+	if (file_exists(rebase_path_squash_msg())) {
+		struct strbuf header = STRBUF_INIT;
+		char *eol, *p;
+
+		if (strbuf_read_file(&buf, rebase_path_squash_msg(), 2048) <= 0)
+			return error(_("could not read '%s'"),
+				rebase_path_squash_msg());
+
+		p = buf.buf + 1;
+		eol = strchrnul(buf.buf, '\n');
+		if (buf.buf[0] != comment_line_char ||
+		    (p += strcspn(p, "0123456789\n")) == eol)
+			return error(_("unexpected 1st line of squash message:"
+				       "\n\n\t%.*s"),
+				     (int)(eol - buf.buf), buf.buf);
+		count = strtol(p, NULL, 10);
+
+		if (count < 1)
+			return error(_("invalid 1st line of squash message:\n"
+				       "\n\t%.*s"),
+				     (int)(eol - buf.buf), buf.buf);
+
+		strbuf_addf(&header, "%c ", comment_line_char);
+		strbuf_addf(&header,
+			    _("This is a combination of %d commits."), ++count);
+		strbuf_splice(&buf, 0, eol - buf.buf, header.buf, header.len);
+		strbuf_release(&header);
+	} else {
+		unsigned char head[20];
+		struct commit *head_commit;
+		const char *head_message, *body;
+
+		if (get_sha1("HEAD", head))
+			return error(_("need a HEAD to fixup"));
+		if (!(head_commit = lookup_commit_reference(head)))
+			return error(_("could not read HEAD"));
+		if (!(head_message = get_commit_buffer(head_commit, NULL)))
+			return error(_("could not read HEAD's commit message"));
+
+		find_commit_subject(head_message, &body);
+		if (write_message(body, strlen(body),
+				  rebase_path_fixup_msg(), 0)) {
+			unuse_commit_buffer(head_commit, head_message);
+			return error(_("cannot write '%s'"),
+				     rebase_path_fixup_msg());
+		}
+
+		count = 2;
+		strbuf_addf(&buf, "%c ", comment_line_char);
+		strbuf_addf(&buf, _("This is a combination of %d commits."),
+			    count);
+		strbuf_addf(&buf, "\n%c ", comment_line_char);
+		strbuf_addstr(&buf, _("This is the 1st commit message:"));
+		strbuf_addstr(&buf, "\n\n");
+		strbuf_addstr(&buf, body);
+
+		unuse_commit_buffer(head_commit, head_message);
+	}
+
+	if (!(message = get_commit_buffer(commit, NULL)))
+		return error(_("could not read commit message of %s"),
+			     oid_to_hex(&commit->object.oid));
+	find_commit_subject(message, &body);
+
+	if (command == TODO_SQUASH) {
+		unlink(rebase_path_fixup_msg());
+		strbuf_addf(&buf, "\n%c ", comment_line_char);
+		strbuf_addf(&buf, _("This is the commit message #%d:"), count);
+		strbuf_addstr(&buf, "\n\n");
+		strbuf_addstr(&buf, body);
+	} else if (command == TODO_FIXUP) {
+		strbuf_addf(&buf, "\n%c ", comment_line_char);
+		strbuf_addf(&buf, _("The commit message #%d will be skipped:"),
+			    count);
+		strbuf_addstr(&buf, "\n\n");
+		strbuf_add_commented_lines(&buf, body, strlen(body));
+	} else
+		return error(_("unknown command: %d"), command);
+	unuse_commit_buffer(commit, message);
+
+	res = write_message(buf.buf, buf.len, rebase_path_squash_msg(), 0);
+	strbuf_release(&buf);
+	return res;
+}
+
 static int do_pick_commit(enum todo_command command, struct commit *commit,
-		struct replay_opts *opts)
+		struct replay_opts *opts, int final_fixup)
 {
+	int edit = opts->edit, cleanup_commit_message = 0;
+	const char *msg_file = edit ? NULL : git_path_merge_msg();
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, allow;
+	int res, unborn = 0, amend = 0, allow;
 
 	if (opts->no_commit) {
 		/*
@@ -720,7 +852,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
-	if (opts->allow_ff &&
+	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
 	     (!parent && unborn)))
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
@@ -779,6 +911,27 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
+	if (is_fixup(command)) {
+		if (update_squash_messages(command, commit, opts))
+			return -1;
+		amend = 1;
+		if (!final_fixup)
+			msg_file = rebase_path_squash_msg();
+		else if (file_exists(rebase_path_fixup_msg())) {
+			cleanup_commit_message = 1;
+			msg_file = rebase_path_fixup_msg();
+		} else {
+			const char *dest = git_path("SQUASH_MSG");
+			unlink(dest);
+			if (copy_file(dest, rebase_path_squash_msg(), 0666))
+				return error(_("could not rename '%s' to '%s'"),
+					     rebase_path_squash_msg(), dest);
+			unlink(git_path("MERGE_MSG"));
+			msg_file = dest;
+			edit = 1;
+		}
+	}
+
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
@@ -834,8 +987,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit, 0, 0);
+		res = run_git_commit(msg_file, opts, allow, edit, amend,
+				     cleanup_commit_message);
+
+	if (!res && final_fixup) {
+		unlink(rebase_path_fixup_msg());
+		unlink(rebase_path_squash_msg());
+	}
 
 leave:
 	free_message(commit, &msg);
@@ -976,7 +1134,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
 	struct todo_item *item;
 	char *p = buf, *next_p;
-	int i, res = 0;
+	int i, res = 0, fixup_okay = file_exists(rebase_path_done());
 
 	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
@@ -991,8 +1149,16 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		if (parse_insn_line(item, p, eol)) {
 			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
-			item->command = -1;
+			item->command = TODO_NOOP;
 		}
+
+		if (fixup_okay)
+			; /* do nothing */
+		else if (is_fixup(item->command))
+			return error(_("cannot '%s' without a previous commit"),
+				command_to_string(item->command));
+		else if (!is_noop(item->command))
+			fixup_okay = 1;
 	}
 	if (!todo_list->nr)
 		return error(_("no commits parsed."));
@@ -1435,6 +1601,20 @@ static int error_with_patch(struct commit *commit,
 	return exit_code;
 }
 
+static int error_failed_squash(struct commit *commit,
+	struct replay_opts *opts, int subject_len, const char *subject)
+{
+	if (rename(rebase_path_squash_msg(), rebase_path_message()))
+		return error(_("could not rename '%s' to '%s'"),
+			rebase_path_squash_msg(), rebase_path_message());
+	unlink(rebase_path_fixup_msg());
+	unlink(git_path("MERGE_MSG"));
+	if (copy_file(git_path("MERGE_MSG"), rebase_path_message(), 0666))
+		return error(_("could not copy '%s' to '%s'"),
+			     rebase_path_message(), git_path("MERGE_MSG"));
+	return error_with_patch(commit, subject, subject_len, opts, 1, 0);
+}
+
 static int do_exec(const char *command_line)
 {
 	const char *child_argv[] = { NULL, NULL };
@@ -1475,6 +1655,21 @@ static int do_exec(const char *command_line)
 	return status;
 }
 
+static int is_final_fixup(struct todo_list *todo_list)
+{
+	int i = todo_list->current;
+
+	if (!is_fixup(todo_list->items[i].command))
+		return 0;
+
+	while (++i < todo_list->nr)
+		if (is_fixup(todo_list->items[i].command))
+			return 0;
+		else if (!is_noop(todo_list->items[i].command))
+			break;
+	return 1;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1490,9 +1685,15 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		struct todo_item *item = todo_list->items + todo_list->current;
 		if (save_todo(todo_list, opts))
 			return -1;
-		if (item->command <= TODO_EDIT) {
+		if (is_rebase_i(opts)) {
+			unlink(rebase_path_message());
+			unlink(rebase_path_author_script());
+			unlink(rebase_path_stopped_sha());
+			unlink(rebase_path_amend());
+		}
+		if (item->command <= TODO_SQUASH) {
 			res = do_pick_commit(item->command, item->commit,
-					opts);
+					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
@@ -1503,6 +1704,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (res && is_fixup(item->command)) {
+				if (res == 1)
+					intend_to_amend();
+				return error_failed_squash(item->commit, opts,
+					item->arg_len, item->arg);
+			}
 		} else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
 			int saved = *end_of_arg;
@@ -1601,7 +1808,7 @@ static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
 	return do_pick_commit(opts->action == REPLAY_PICK ?
-		TODO_PICK : TODO_REVERT, cmit, opts);
+		TODO_PICK : TODO_REVERT, cmit, opts, 0);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 12/38] sequencer (rebase -i): write an author-script file

When the interactive rebase aborts, it writes out an author-script file to record the author information for the current commit. As we are about to teach the sequencer how to perform the actions behind an interactive rebase, it needs to write those author-script files, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)
Show changes to sequencer.c +49 −1
diff --git a/sequencer.c b/sequencer.c
index 29b944d724..9913882603 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -483,6 +483,52 @@ static int is_index_unchanged(void)
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
 
+static int write_author_script(const char *message)
+{
+	struct strbuf buf = STRBUF_INIT;
+	const char *eol;
+	int res;
+
+	for (;;)
+		if (!*message || starts_with(message, "\n")) {
+missing_author:
+			/* Missing 'author' line? */
+			unlink(rebase_path_author_script());
+			return 0;
+		} else if (skip_prefix(message, "author ", &message))
+			break;
+		else if ((eol = strchr(message, '\n')))
+			message = eol + 1;
+		else
+			goto missing_author;
+
+	strbuf_addstr(&buf, "GIT_AUTHOR_NAME='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, " <", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_EMAIL='");
+	while (*message && *message != '\n' && *message != '\r')
+		if (skip_prefix(message, "> ", &message))
+			break;
+		else if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	strbuf_addstr(&buf, "'\nGIT_AUTHOR_DATE='@");
+	while (*message && *message != '\n' && *message != '\r')
+		if (*message != '\'')
+			strbuf_addch(&buf, *(message++));
+		else
+			strbuf_addf(&buf, "'\\\\%c'", *(message++));
+	res = write_message(buf.buf, buf.len, rebase_path_author_script(), 1);
+	strbuf_release(&buf);
+	return res;
+}
+
 /*
  * Read the author-script file into an environment block, ready for use in
  * run_command(), that can be free()d afterwards.
@@ -935,7 +981,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
+	if (is_rebase_i(opts) && write_author_script(msg.message) < 0)
+		res = -1;
+	else if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 15/38] sequencer (rebase -i): skip some revert/cherry-pick specific code path

When a cherry-pick continues without a "todo script", the intention is simply to pick a single commit.

However, when an interactive rebase is continued without a "todo script", it means that the last command has been completed and that we now need to clean up.

This commit guards the revert/cherry-pick specific steps so that they are not executed in rebase -i mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)
Show changes to sequencer.c +14 −12
diff --git a/sequencer.c b/sequencer.c
index 52e17c8887..a7b9ee0d04 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1878,26 +1878,28 @@ int sequencer_continue(struct replay_opts *opts)
 	if (is_rebase_i(opts)) {
 		if (commit_staged_changes(opts))
 			return -1;
-	}
-	if (!file_exists(get_todo_path(opts)))
+	} else if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
 		return -1;
 	if ((res = read_populate_todo(&todo_list, opts)))
 		goto release_todo_list;
 
-	/* Verify that the conflict has been resolved */
-	if (file_exists(git_path_cherry_pick_head()) ||
-	    file_exists(git_path_revert_head())) {
-		res = continue_single_pick();
-		if (res)
+	if (!is_rebase_i(opts)) {
+		/* Verify that the conflict has been resolved */
+		if (file_exists(git_path_cherry_pick_head()) ||
+		    file_exists(git_path_revert_head())) {
+			res = continue_single_pick();
+			if (res)
+				goto release_todo_list;
+		}
+		if (index_differs_from("HEAD", 0, 0)) {
+			res = error_dirty_index(opts);
 			goto release_todo_list;
+		}
+		todo_list.current++;
 	}
-	if (index_differs_from("HEAD", 0, 0)) {
-		res = error_dirty_index(opts);
-		goto release_todo_list;
-	}
-	todo_list.current++;
+
 	res = pick_commits(&todo_list, opts);
 release_todo_list:
 	todo_list_release(&todo_list);
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 14/38] sequencer (rebase -i): remove CHERRY_PICK_HEAD when no longer needed

The scripted version of the interactive rebase already does that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
Show changes to sequencer.c +6 −1
diff --git a/sequencer.c b/sequencer.c
index 69301fecc6..52e17c8887 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1832,8 +1832,13 @@ static int commit_staged_changes(struct replay_opts *opts)
 
 	if (has_unstaged_changes(1))
 		return error(_("cannot rebase: You have unstaged changes."));
-	if (!has_uncommitted_changes(0))
+	if (!has_uncommitted_changes(0)) {
+		const char *cherry_pick_head = git_path("CHERRY_PICK_HEAD");
+
+		if (file_exists(cherry_pick_head) && unlink(cherry_pick_head))
+			return error(_("could not remove CHERRY_PICK_HEAD"));
 		return 0;
+	}
 
 	if (file_exists(rebase_path_amend())) {
 		struct strbuf rev = STRBUF_INIT;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 13/38] sequencer (rebase -i): allow continuing with staged changes

When an interactive rebase is interrupted, the user may stage changes before continuing, and we need to commit those changes in that case.

Please note that the nested "if" added to the sequencer_continue() is not combined into a single "if" because it will be extended with an "else" clause in a later patch in this patch series.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
Show changes to sequencer.c +40 −0
diff --git a/sequencer.c b/sequencer.c
index 9913882603..69301fecc6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1826,6 +1826,42 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
+static int commit_staged_changes(struct replay_opts *opts)
+{
+	int amend = 0;
+
+	if (has_unstaged_changes(1))
+		return error(_("cannot rebase: You have unstaged changes."));
+	if (!has_uncommitted_changes(0))
+		return 0;
+
+	if (file_exists(rebase_path_amend())) {
+		struct strbuf rev = STRBUF_INIT;
+		unsigned char head[20], to_amend[20];
+
+		if (get_sha1("HEAD", head))
+			return error(_("cannot amend non-existing commit"));
+		if (!read_oneliner(&rev, rebase_path_amend(), 0))
+			return error(_("invalid file: '%s'"), rebase_path_amend());
+		if (get_sha1_hex(rev.buf, to_amend))
+			return error(_("invalid contents: '%s'"),
+				rebase_path_amend());
+		if (hashcmp(head, to_amend))
+			return error(_("\nYou have uncommitted changes in your "
+				       "working tree. Please, commit them\n"
+				       "first and then run 'git rebase "
+				       "--continue' again."));
+
+		strbuf_release(&rev);
+		amend = 1;
+	}
+
+	if (run_git_commit(rebase_path_message(), opts, 1, 1, amend, 0))
+		return error(_("could not commit staged changes."));
+	unlink(rebase_path_amend());
+	return 0;
+}
+
 int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
@@ -1834,6 +1870,10 @@ int sequencer_continue(struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
+	if (is_rebase_i(opts)) {
+		if (commit_staged_changes(opts))
+			return -1;
+	}
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 16/38] sequencer (rebase -i): the todo can be empty when continuing

When the last command of an interactive rebase fails, the user needs to resolve the problem and then continue the interactive rebase. Naturally, the todo script is empty by then. So let's not complain about that!

To that end, let's move that test out of the function that parses the todo script, and into the more high-level function read_populate_todo(). This is also necessary by now because the lower-level parse_insn_buffer() has no idea whether we are performing an interactive rebase or not.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
Show changes to sequencer.c +5 −2
diff --git a/sequencer.c b/sequencer.c
index a7b9ee0d04..6a840216b1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1215,8 +1215,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		else if (!is_noop(item->command))
 			fixup_okay = 1;
 	}
-	if (!todo_list->nr)
-		return error(_("no commits parsed."));
+
 	return res;
 }
 
@@ -1240,6 +1239,10 @@ static int read_populate_todo(struct todo_list *todo_list,
 	if (res)
 		return error(_("unusable instruction sheet: '%s'"), todo_file);
 
+	if (!todo_list->nr &&
+	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
+		return error(_("no commits parsed."));
+
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
 			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 18/38] sequencer (rebase -i): leave a patch upon error

When doing an interactive rebase, we want to leave a 'patch' file for further inspection by the user (even if we never tried to actually apply that patch, since we're cherry-picking instead).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Show changes to sequencer.c +3 −1
diff --git a/sequencer.c b/sequencer.c
index 80b2b2a975..a2002f1c12 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1769,7 +1769,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					intend_to_amend();
 				return error_failed_squash(item->commit, opts,
 					item->arg_len, item->arg);
-			}
+			} else if (res && is_rebase_i(opts))
+				return res | error_with_patch(item->commit,
+					item->arg, item->arg_len, opts, res, 0);
 		} else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
 			int saved = *end_of_arg;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:27 UTC · re: Johannes Schindelin · lore

[PATCH v3 17/38] sequencer (rebase -i): update refs after a successful rebase

An interactive rebase operates on a detached HEAD (to keep the reflog of the original branch relatively clean), and updates the branch only at the end.

Now that the sequencer learns to perform interactive rebases, it also needs to learn the trick to update the branch before removing the directory containing the state of the interactive rebase.

We introduce a new head_ref variable in a wider scope than necessary at the moment, to allow for a later patch that prints out "Successfully rebased and updated <ref>".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 46 +++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 45 insertions(+), 1 deletion(-)
Show changes to sequencer.c +45 −1
diff --git a/sequencer.c b/sequencer.c
index 6a840216b1..80b2b2a975 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -102,6 +102,8 @@ static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
+static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
+static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1784,12 +1786,53 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	}
 
 	if (is_rebase_i(opts)) {
-		struct strbuf buf = STRBUF_INIT;
+		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
 			return 0;
 
+		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
+				starts_with(head_ref.buf, "refs/")) {
+			unsigned char head[20], orig[20];
+			int res;
+
+			if (get_sha1("HEAD", head)) {
+				res = error(_("cannot read HEAD"));
+cleanup_head_ref:
+				strbuf_release(&head_ref);
+				strbuf_release(&buf);
+				return res;
+			}
+			if (!read_oneliner(&buf, rebase_path_orig_head(), 0) ||
+					get_sha1_hex(buf.buf, orig)) {
+				res = error(_("could not read orig-head"));
+				goto cleanup_head_ref;
+			}
+			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
+				head_ref.buf);
+			if (!read_oneliner(&buf, rebase_path_onto(), 0)) {
+				res = error(_("could not read 'onto'"));
+				goto cleanup_head_ref;
+			}
+			if (update_ref(buf.buf, head_ref.buf, head, orig,
+					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR)) {
+				res = error(_("could not update %s"),
+					head_ref.buf);
+				goto cleanup_head_ref;
+			}
+			strbuf_reset(&buf);
+			strbuf_addf(&buf,
+				"rebase -i (finish): returning to %s",
+				head_ref.buf);
+			if (create_symref("HEAD", head_ref.buf, buf.buf)) {
+				res = error(_("could not update HEAD to %s"),
+					head_ref.buf);
+				goto cleanup_head_ref;
+			}
+			strbuf_reset(&buf);
+		}
+
 		if (opts->verbose) {
 			struct rev_info log_tree_opt;
 			struct object_id orig, head;
@@ -1810,6 +1853,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			}
 		}
 		strbuf_release(&buf);
+		strbuf_release(&head_ref);
 	}
 
 	/*
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 19/38] sequencer (rebase -i): implement the 'reword' command

This is now trivial, as all the building blocks are in place: all we need to do is to flip the "edit" switch when committing.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
Show changes to sequencer.c +7 −2
diff --git a/sequencer.c b/sequencer.c
index a2002f1c12..50e998acc4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -718,6 +718,7 @@ enum todo_command {
 	TODO_PICK = 0,
 	TODO_REVERT,
 	TODO_EDIT,
+	TODO_REWORD,
 	TODO_FIXUP,
 	TODO_SQUASH,
 	/* commands that do something else than handling a single commit */
@@ -733,6 +734,7 @@ static struct {
 	{ 'p', "pick" },
 	{ 0,   "revert" },
 	{ 'e', "edit" },
+	{ 'r', "reword" },
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
@@ -962,7 +964,9 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		}
 	}
 
-	if (is_fixup(command)) {
+	if (command == TODO_REWORD)
+		edit = 1;
+	else if (is_fixup(command)) {
 		if (update_squash_messages(command, commit, opts))
 			return -1;
 		amend = 1;
@@ -1771,7 +1775,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg_len, item->arg);
 			} else if (res && is_rebase_i(opts))
 				return res | error_with_patch(item->commit,
-					item->arg, item->arg_len, opts, res, 0);
+					item->arg, item->arg_len, opts, res,
+					item->command == TODO_REWORD);
 		} else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
 			int saved = *end_of_arg;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 21/38] sequencer (rebase -i): refactor setting the reflog message

This makes the code DRYer, with the obvious benefit that we can enhance the code further in a single place.

We can also reuse the functionality elsewhere by calling this new function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)
Show changes to sequencer.c +26 −7
diff --git a/sequencer.c b/sequencer.c
index 23161f593e..0d8e11f580 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1743,6 +1743,26 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static const char *reflog_message(struct replay_opts *opts,
+	const char *sub_action, const char *fmt, ...)
+{
+	va_list ap;
+	static struct strbuf buf = STRBUF_INIT;
+
+	va_start(ap, fmt);
+	strbuf_reset(&buf);
+	strbuf_addstr(&buf, action_name(opts));
+	if (sub_action)
+		strbuf_addf(&buf, " (%s)", sub_action);
+	if (fmt) {
+		strbuf_addstr(&buf, ": ");
+		strbuf_vaddf(&buf, fmt, ap);
+	}
+	va_end(ap);
+
+	return buf.buf;
+}
+
 static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	int res = 0;
@@ -1810,6 +1830,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 		if (read_oneliner(&head_ref, rebase_path_head_name(), 0) &&
 				starts_with(head_ref.buf, "refs/")) {
+			const char *msg;
 			unsigned char head[20], orig[20];
 			int res;
 
@@ -1825,23 +1846,21 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				res = error(_("could not read orig-head"));
 				goto cleanup_head_ref;
 			}
-			strbuf_addf(&buf, "rebase -i (finish): %s onto ",
-				head_ref.buf);
 			if (!read_oneliner(&buf, rebase_path_onto(), 0)) {
 				res = error(_("could not read 'onto'"));
 				goto cleanup_head_ref;
 			}
-			if (update_ref(buf.buf, head_ref.buf, head, orig,
+			msg = reflog_message(opts, "finish", "%s onto %s",
+				head_ref.buf, buf.buf);
+			if (update_ref(msg, head_ref.buf, head, orig,
 					REF_NODEREF, UPDATE_REFS_MSG_ON_ERR)) {
 				res = error(_("could not update %s"),
 					head_ref.buf);
 				goto cleanup_head_ref;
 			}
-			strbuf_reset(&buf);
-			strbuf_addf(&buf,
-				"rebase -i (finish): returning to %s",
+			msg = reflog_message(opts, "finish", "returning to %s",
 				head_ref.buf);
-			if (create_symref("HEAD", head_ref.buf, buf.buf)) {
+			if (create_symref("HEAD", head_ref.buf, msg)) {
 				res = error(_("could not update HEAD to %s"),
 					head_ref.buf);
 				goto cleanup_head_ref;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 23/38] sequencer (rebase -i): copy commit notes at end

When rebasing commits that have commit notes attached, the interactive rebase rewrites those notes faithfully at the end. The sequencer must do this, too, if it wishes to do interactive rebase's job.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)
Show changes to sequencer.c +76 −0
diff --git a/sequencer.c b/sequencer.c
index 95ae4bcd1e..50380a15b8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -96,6 +96,15 @@ static GIT_PATH_FUNC(rebase_path_amend, "rebase-merge/amend")
  */
 static GIT_PATH_FUNC(rebase_path_stopped_sha, "rebase-merge/stopped-sha")
 /*
+ * For the post-rewrite hook, we make a list of rewritten commits and
+ * their new sha1s.  The rewritten-pending list keeps the sha1s of
+ * commits that have been processed, but not committed yet,
+ * e.g. because they are waiting for a 'squash' command.
+ */
+static GIT_PATH_FUNC(rebase_path_rewritten_list, "rebase-merge/rewritten-list")
+static GIT_PATH_FUNC(rebase_path_rewritten_pending,
+	"rebase-merge/rewritten-pending")
+/*
  * The following files are written by git-rebase just after parsing the
  * command-line (and are only consumed, not modified, by the sequencer).
  */
@@ -850,6 +859,44 @@ static int update_squash_messages(enum todo_command command,
 	return res;
 }
 
+static void flush_rewritten_pending(void) {
+	struct strbuf buf = STRBUF_INIT;
+	unsigned char newsha1[20];
+	FILE *out;
+
+	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
+			!get_sha1("HEAD", newsha1) &&
+			(out = fopen(rebase_path_rewritten_list(), "a"))) {
+		char *bol = buf.buf, *eol;
+
+		while (*bol) {
+			eol = strchrnul(bol, '\n');
+			fprintf(out, "%.*s %s\n", (int)(eol - bol),
+					bol, sha1_to_hex(newsha1));
+			if (!*eol)
+				break;
+			bol = eol + 1;
+		}
+		fclose(out);
+		unlink(rebase_path_rewritten_pending());
+	}
+	strbuf_release(&buf);
+}
+
+static void record_in_rewritten(struct object_id *oid,
+		enum todo_command next_command) {
+	FILE *out = fopen(rebase_path_rewritten_pending(), "a");
+
+	if (!out)
+		return;
+
+	fprintf(out, "%s\n", oid_to_hex(oid));
+	fclose(out);
+
+	if (!is_fixup(next_command))
+		flush_rewritten_pending();
+}
+
 static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct replay_opts *opts, int final_fixup)
 {
@@ -1743,6 +1790,17 @@ static int is_final_fixup(struct todo_list *todo_list)
 	return 1;
 }
 
+static enum todo_command peek_command(struct todo_list *todo_list, int offset)
+{
+	int i;
+
+	for (i = todo_list->current + offset; i < todo_list->nr; i++)
+		if (!is_noop(todo_list->items[i].command))
+			return todo_list->items[i].command;
+
+	return -1;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1801,6 +1859,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					item->arg, item->arg_len, opts, res,
 					!res);
 			}
+			if (is_rebase_i(opts) && !res)
+				record_in_rewritten(&item->commit->object.oid,
+					peek_command(todo_list, 1));
 			if (res && is_fixup(item->command)) {
 				if (res == 1)
 					intend_to_amend();
@@ -1827,6 +1888,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 
 	if (is_rebase_i(opts)) {
 		struct strbuf head_ref = STRBUF_INIT, buf = STRBUF_INIT;
+		struct stat st;
 
 		/* Stopped in the middle, as planned? */
 		if (todo_list->current < todo_list->nr)
@@ -1891,6 +1953,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				log_tree_diff_flush(&log_tree_opt);
 			}
 		}
+		flush_rewritten_pending();
+		if (!stat(rebase_path_rewritten_list(), &st) &&
+				st.st_size > 0) {
+			struct child_process child = CHILD_PROCESS_INIT;
+
+			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			child.git_cmd = 1;
+			argv_array_push(&child.args, "notes");
+			argv_array_push(&child.args, "copy");
+			argv_array_push(&child.args, "--for-rewrite=rebase");
+			/* we don't care if this copying failed */
+			run_command(&child);
+		}
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 20/38] sequencer (rebase -i): allow fast-forwarding for edit/reword

The sequencer already knew how to fast-forward instead of cherry-picking, if possible.

We want to continue to do this, of course, but in case of the 'reword' command, we will need to call `git commit` after fast-forwarding.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
Show changes to sequencer.c +17 −8
diff --git a/sequencer.c b/sequencer.c
index 50e998acc4..23161f593e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -860,7 +860,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, amend = 0, allow;
+	int res, unborn = 0, amend = 0, allow = 0;
 
 	if (opts->no_commit) {
 		/*
@@ -905,11 +905,23 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	else
 		parent = commit->parents->item;
 
+	if (get_message(commit, &msg) != 0)
+		return error(_("cannot get commit message for %s"),
+			oid_to_hex(&commit->object.oid));
+
 	if (opts->allow_ff && !is_fixup(command) &&
 	    ((parent && !hashcmp(parent->object.oid.hash, head)) ||
-	     (!parent && unborn)))
-		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
-
+	     (!parent && unborn))) {
+		if (is_rebase_i(opts))
+			write_author_script(msg.message);
+		res = fast_forward_to(commit->object.oid.hash, head, unborn,
+			opts);
+		if (res || command != TODO_REWORD)
+			goto leave;
+		edit = amend = 1;
+		msg_file = NULL;
+		goto fast_forward_edit;
+	}
 	if (parent && parse_commit(parent) < 0)
 		/* TRANSLATORS: The first %s will be a "todo" command like
 		   "revert" or "pick", the second %s a SHA1. */
@@ -917,10 +929,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			command_to_string(command),
 			oid_to_hex(&parent->object.oid));
 
-	if (get_message(commit, &msg) != 0)
-		return error(_("cannot get commit message for %s"),
-			oid_to_hex(&commit->object.oid));
-
 	/*
 	 * "commit" is an existing commit.  We would want to apply
 	 * the difference it introduces since its first parent "prev"
@@ -1044,6 +1052,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
+fast_forward_edit:
 		res = run_git_commit(msg_file, opts, allow, edit, amend,
 				     cleanup_commit_message);
 
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 24/38] sequencer (rebase -i): record interrupted commits in rewritten, too

When continuing after a `pick` command failed, we want that commit to show up in the rewritten-list (and its notes to be rewritten), too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 8 ++++++++
 1 file changed, 8 insertions(+)
Show changes to sequencer.c +8 −0
diff --git a/sequencer.c b/sequencer.c
index 50380a15b8..d7273fd1b3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2060,6 +2060,14 @@ int sequencer_continue(struct replay_opts *opts)
 			goto release_todo_list;
 		}
 		todo_list.current++;
+	} else if (file_exists(rebase_path_stopped_sha())) {
+		struct strbuf buf = STRBUF_INIT;
+		struct object_id oid;
+
+		if (read_oneliner(&buf, rebase_path_stopped_sha(), 1) &&
+		    !get_sha1_committish(buf.buf, oid.hash))
+			record_in_rewritten(&oid, peek_command(&todo_list, 0));
+		strbuf_release(&buf);
 	}
 
 	res = pick_commits(&todo_list, opts);
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 25/38] sequencer (rebase -i): run the post-rewrite hook, if needed

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
Show changes to sequencer.c +14 −0
diff --git a/sequencer.c b/sequencer.c
index d7273fd1b3..43ced8db31 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1957,6 +1957,8 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process child = CHILD_PROCESS_INIT;
+			const char *post_rewrite_hook =
+				find_hook("post-rewrite");
 
 			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			child.git_cmd = 1;
@@ -1965,6 +1967,18 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			argv_array_push(&child.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
 			run_command(&child);
+
+			if (post_rewrite_hook) {
+				struct child_process hook = CHILD_PROCESS_INIT;
+
+				hook.in = open(rebase_path_rewritten_list(),
+					O_RDONLY);
+				hook.stdout_to_stderr = 1;
+				argv_array_push(&hook.args, post_rewrite_hook);
+				argv_array_push(&hook.args, "rebase");
+				/* we don't care if this hook failed */
+				run_command(&hook);
+			}
 		}
 
 		strbuf_release(&buf);
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 22/38] sequencer (rebase -i): set the reflog message consistently

We already used the same reflog message as the scripted version of rebase -i when finishing. With this commit, we do that also for all the commands before that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)
Show changes to sequencer.c +4 −0
diff --git a/sequencer.c b/sequencer.c
index 0d8e11f580..95ae4bcd1e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1785,6 +1785,10 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 			unlink(rebase_path_amend());
 		}
 		if (item->command <= TODO_SQUASH) {
+			if (is_rebase_i(opts))
+				setenv("GIT_REFLOG_ACTION", reflog_message(opts,
+					command_to_string(item->command), NULL),
+					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
 			if (item->command == TODO_EDIT) {
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 26/38] sequencer (rebase -i): respect the rebase.autostash setting

Git's `rebase` command inspects the `rebase.autostash` config setting to determine whether it should stash any uncommitted changes before rebasing and re-apply them afterwards.

As we introduce more bits and pieces to let the sequencer act as interactive rebase's backend, here is the part that adds support for the autostash feature.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
Show changes to sequencer.c +43 −0
diff --git a/sequencer.c b/sequencer.c
index 43ced8db31..06f7cebe48 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -113,6 +113,7 @@ static GIT_PATH_FUNC(rebase_path_orig_head, "rebase-merge/orig-head")
 static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
+static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1801,6 +1802,47 @@ static enum todo_command peek_command(struct todo_list *todo_list, int offset)
 	return -1;
 }
 
+static int apply_autostash(struct replay_opts *opts)
+{
+	struct strbuf stash_sha1 = STRBUF_INIT;
+	struct child_process child = CHILD_PROCESS_INIT;
+	int ret = 0;
+
+	if (!read_oneliner(&stash_sha1, rebase_path_autostash(), 1)) {
+		strbuf_release(&stash_sha1);
+		return 0;
+	}
+	strbuf_trim(&stash_sha1);
+
+	child.git_cmd = 1;
+	argv_array_push(&child.args, "stash");
+	argv_array_push(&child.args, "apply");
+	argv_array_push(&child.args, stash_sha1.buf);
+	if (!run_command(&child))
+		printf(_("Applied autostash."));
+	else {
+		struct child_process store = CHILD_PROCESS_INIT;
+
+		store.git_cmd = 1;
+		argv_array_push(&store.args, "stash");
+		argv_array_push(&store.args, "store");
+		argv_array_push(&store.args, "-m");
+		argv_array_push(&store.args, "autostash");
+		argv_array_push(&store.args, "-q");
+		argv_array_push(&store.args, stash_sha1.buf);
+		if (run_command(&store))
+			ret = error(_("cannot store %s"), stash_sha1.buf);
+		else
+			printf(_("Applying autostash resulted in conflicts.\n"
+				"Your changes are safe in the stash.\n"
+				"You can run \"git stash pop\" or"
+				" \"git stash drop\" at any time.\n"));
+	}
+
+	strbuf_release(&stash_sha1);
+	return ret;
+}
+
 static const char *reflog_message(struct replay_opts *opts,
 	const char *sub_action, const char *fmt, ...)
 {
@@ -1980,6 +2022,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 				run_command(&hook);
 			}
 		}
+		apply_autostash(opts);
 
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 27/38] sequencer (rebase -i): respect strategy/strategy_opts settings

The sequencer already has an idea about using different merge strategies. We just piggy-back on top of that, using rebase -i's own settings, when running the sequencer in interactive rebase mode.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)
Show changes to sequencer.c +25 −1
diff --git a/sequencer.c b/sequencer.c
index 06f7cebe48..04a64cf0dc 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -114,6 +114,8 @@ static GIT_PATH_FUNC(rebase_path_verbose, "rebase-merge/verbose")
 static GIT_PATH_FUNC(rebase_path_head_name, "rebase-merge/head-name")
 static GIT_PATH_FUNC(rebase_path_onto, "rebase-merge/onto")
 static GIT_PATH_FUNC(rebase_path_autostash, "rebase-merge/autostash")
+static GIT_PATH_FUNC(rebase_path_strategy, "rebase-merge/strategy")
+static GIT_PATH_FUNC(rebase_path_strategy_opts, "rebase-merge/strategy_opts")
 
 static inline int is_rebase_i(const struct replay_opts *opts)
 {
@@ -1368,6 +1370,26 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
+static void read_strategy_opts(struct replay_opts *opts, struct strbuf *buf)
+{
+	int i;
+
+	strbuf_reset(buf);
+	if (!read_oneliner(buf, rebase_path_strategy(), 0))
+		return;
+	opts->strategy = strbuf_detach(buf, NULL);
+	if (!read_oneliner(buf, rebase_path_strategy_opts(), 0))
+		return;
+
+	opts->xopts_nr = split_cmdline(buf->buf, (const char ***)&opts->xopts);
+	for (i = 0; i < opts->xopts_nr; i++) {
+		const char *arg = opts->xopts[i];
+
+		skip_prefix(arg, "--", &arg);
+		opts->xopts[i] = xstrdup(arg);
+	}
+}
+
 static int read_populate_opts(struct replay_opts *opts)
 {
 	if (is_rebase_i(opts)) {
@@ -1381,11 +1403,13 @@ static int read_populate_opts(struct replay_opts *opts)
 				opts->gpg_sign = xstrdup(buf.buf + 2);
 			}
 		}
-		strbuf_release(&buf);
 
 		if (file_exists(rebase_path_verbose()))
 			opts->verbose = 1;
 
+		read_strategy_opts(opts, &buf);
+		strbuf_release(&buf);
+
 		return 0;
 	}
 
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:28 UTC · re: Johannes Schindelin · lore

[PATCH v3 28/38] sequencer (rebase -i): allow rescheduling commands

The interactive rebase has the very special magic that a cherry-pick that exits with a status different from 0 and 1 signifies a failure to even record that a cherry-pick was started.

This can happen e.g. when a fast-forward fails because it would overwrite untracked files.

In that case, we must reschedule the command that we thought we already had at least started successfully.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++++
 1 file changed, 6 insertions(+)
Show changes to sequencer.c +6 −0
diff --git a/sequencer.c b/sequencer.c
index 04a64cf0dc..dd5b843a84 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1915,6 +1915,12 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 					1);
 			res = do_pick_commit(item->command, item->commit,
 					opts, is_final_fixup(todo_list));
+			if (is_rebase_i(opts) && res < 0) {
+				/* Reschedule */
+				todo_list->current--;
+				if (save_todo(todo_list, opts))
+					return -1;
+			}
 			if (item->command == TODO_EDIT) {
 				struct commit *commit = item->commit;
 				if (!res)
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:34 UTC · re: Johannes Schindelin · lore

[PATCH v3 29/38] sequencer (rebase -i): implement the 'drop' command

The parsing part of a 'drop' command is almost identical to parsing a 'pick', while the operation is the same as that of a 'noop'.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
Show changes to sequencer.c +5 −3
diff --git a/sequencer.c b/sequencer.c
index dd5b843a84..6e92f186ae 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -736,7 +736,8 @@ enum todo_command {
 	/* commands that do something else than handling a single commit */
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
-	TODO_NOOP
+	TODO_NOOP,
+	TODO_DROP
 };
 
 static struct {
@@ -750,7 +751,8 @@ static struct {
 	{ 'f', "fixup" },
 	{ 's', "squash" },
 	{ 'x', "exec" },
-	{ 0,   "noop" }
+	{ 0,   "noop" },
+	{ 'd', "drop" }
 };
 
 static const char *command_to_string(const enum todo_command command)
@@ -762,7 +764,7 @@ static const char *command_to_string(const enum todo_command command)
 
 static int is_noop(const enum todo_command command)
 {
-	return TODO_NOOP <= (size_t)command;
+	return TODO_NOOP <= command;
 }
 
 static int is_fixup(enum todo_command command)
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:34 UTC · re: Johannes Schindelin · lore

[PATCH v3 30/38] sequencer (rebase -i): differentiate between comments and 'noop'

In the upcoming patch, we will support rebase -i's progress reporting. The progress skips comments but counts 'noop's.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
Show changes to sequencer.c +9 −6
diff --git a/sequencer.c b/sequencer.c
index 6e92f186ae..41f80ea2c4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -737,7 +737,9 @@ enum todo_command {
 	TODO_EXEC,
 	/* commands that do nothing but are counted for reporting progress */
 	TODO_NOOP,
-	TODO_DROP
+	TODO_DROP,
+	/* comments (not counted for reporting progress) */
+	TODO_COMMENT
 };
 
 static struct {
@@ -752,12 +754,13 @@ static struct {
 	{ 's', "squash" },
 	{ 'x', "exec" },
 	{ 0,   "noop" },
-	{ 'd', "drop" }
+	{ 'd', "drop" },
+	{ 0,   NULL }
 };
 
 static const char *command_to_string(const enum todo_command command)
 {
-	if ((size_t)command < ARRAY_SIZE(todo_command_info))
+	if (command < TODO_COMMENT)
 		return todo_command_info[command].str;
 	die("Unknown command: %d", command);
 }
@@ -1198,14 +1201,14 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	bol += strspn(bol, " \t");
 
 	if (bol == eol || *bol == '\r' || *bol == comment_line_char) {
-		item->command = TODO_NOOP;
+		item->command = TODO_COMMENT;
 		item->commit = NULL;
 		item->arg = bol;
 		item->arg_len = eol - bol;
 		return 0;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(todo_command_info); i++)
+	for (i = 0; i < TODO_COMMENT; i++)
 		if (skip_prefix(bol, todo_command_info[i].str, &bol)) {
 			item->command = i;
 			break;
@@ -1214,7 +1217,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 			item->command = i;
 			break;
 		}
-	if (i >= ARRAY_SIZE(todo_command_info))
+	if (i >= TODO_COMMENT)
 		return -1;
 
 	if (item->command == TODO_NOOP) {
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 32/38] sequencer: use run_command() directly

Instead of using the convenience function run_command_v_opt_cd_env(), we now use the run_command() function. The former function is simply a wrapper of the latter, trying to make it more convenient to use.

However, we already have to construct the argv and the env parameters, and we will need even finer control e.g. over the output of the command, so let's just stop using the convenience function.

Based on patches and suggestions by Johannes Sixt and Jeff King.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 36 ++++++++++++++++--------------------
 1 file changed, 16 insertions(+), 20 deletions(-)
Show changes to sequencer.c +16 −20
diff --git a/sequencer.c b/sequencer.c
index a0d0aaeaf8..c7dc5a2ad4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -604,12 +604,13 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty, int edit, int amend,
 			  int cleanup_commit_message)
 {
-	struct argv_array env = ARGV_ARRAY_INIT, array;
-	int rc;
+	struct child_process cmd = CHILD_PROCESS_INIT;
 	const char *value;
 
+	cmd.git_cmd = 1;
+
 	if (is_rebase_i(opts)) {
-		if (!read_env_script(&env)) {
+		if (read_env_script(&cmd.env_array)) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
 			return error(_(staged_changes_advice),
@@ -617,39 +618,34 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		}
 	}
 
-	argv_array_init(&array);
-	argv_array_push(&array, "commit");
-	argv_array_push(&array, "-n");
+	argv_array_push(&cmd.args, "commit");
+	argv_array_push(&cmd.args, "-n");
 
 	if (amend)
-		argv_array_push(&array, "--amend");
+		argv_array_push(&cmd.args, "--amend");
 	if (opts->gpg_sign)
-		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
+		argv_array_pushf(&cmd.args, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
-		argv_array_push(&array, "-s");
+		argv_array_push(&cmd.args, "-s");
 	if (defmsg)
-		argv_array_pushl(&array, "-F", defmsg, NULL);
+		argv_array_pushl(&cmd.args, "-F", defmsg, NULL);
 	if (cleanup_commit_message)
-		argv_array_push(&array, "--cleanup=strip");
+		argv_array_push(&cmd.args, "--cleanup=strip");
 	if (edit)
-		argv_array_push(&array, "-e");
+		argv_array_push(&cmd.args, "-e");
 	else if (!cleanup_commit_message &&
 		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
-		argv_array_push(&array, "--cleanup=verbatim");
+		argv_array_push(&cmd.args, "--cleanup=verbatim");
 
 	if (allow_empty)
-		argv_array_push(&array, "--allow-empty");
+		argv_array_push(&cmd.args, "--allow-empty");
 
 	if (opts->allow_empty_message)
-		argv_array_push(&array, "--allow-empty-message");
+		argv_array_push(&cmd.args, "--allow-empty-message");
 
-	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
-			(const char *const *)env.argv);
-	argv_array_clear(&array);
-	argv_array_clear(&env);
 
-	return rc;
+	return run_command(&cmd);
 }
 
 static int is_original_commit_empty(struct commit *commit)
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 33/38] sequencer (rebase -i): show only failed `git commit`'s output

This is the behavior of the shell script version of the interactive rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
Show changes to sequencer.c +18 −0
diff --git a/sequencer.c b/sequencer.c
index c7dc5a2ad4..a501dfce38 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -610,6 +610,11 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	cmd.git_cmd = 1;
 
 	if (is_rebase_i(opts)) {
+		if (!edit) {
+			cmd.stdout_to_stderr = 1;
+			cmd.err = -1;
+		}
+
 		if (read_env_script(&cmd.env_array)) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
@@ -644,6 +649,19 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&cmd.args, "--allow-empty-message");
 
+	if (cmd.err == -1) {
+		/* hide stderr on success */
+		struct strbuf buf = STRBUF_INIT;
+		int rc = pipe_command(&cmd,
+				      NULL, 0,
+				      /* stdout is already redirected */
+				      NULL, 0,
+				      &buf, 0);
+		if (rc)
+			fputs(buf.buf, stderr);
+		strbuf_release(&buf);
+		return rc;
+	}
 
 	return run_command(&cmd);
 }
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 31/38] sequencer: make reading author-script more elegant

Rather than abusing a strbuf to come up with an environment block, let's just use the argv_array structure which serves the same purpose much better.

While at it, rename the function to reflect the fact that it does not really care exactly what environment variables are defined in said file.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 +++++++++++---------------------
 1 file changed, 11 insertions(+), 21 deletions(-)
Show changes to sequencer.c +11 −21
diff --git a/sequencer.c b/sequencer.c
index 41f80ea2c4..a0d0aaeaf8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -544,18 +544,17 @@ static int write_author_script(const char *message)
 }
 
 /*
- * Read the author-script file into an environment block, ready for use in
- * run_command(), that can be free()d afterwards.
+ * Read a list of environment variable assignments (such as the author-script
+ * file) into an environment block. Returns -1 on error, 0 otherwise.
  */
-static char **read_author_script(void)
+static int read_env_script(struct argv_array *env)
 {
 	struct strbuf script = STRBUF_INIT;
 	int i, count = 0;
-	char *p, *p2, **env;
-	size_t env_size;
+	char *p, *p2;
 
 	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
-		return NULL;
+		return -1;
 
 	for (p = script.buf; *p; p++)
 		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
@@ -567,19 +566,12 @@ static char **read_author_script(void)
 			count++;
 		}
 
-	env_size = (count + 1) * sizeof(*env);
-	strbuf_grow(&script, env_size);
-	memmove(script.buf + env_size, script.buf, script.len);
-	p = script.buf + env_size;
-	env = (char **)strbuf_detach(&script, NULL);
-
 	for (i = 0; i < count; i++) {
-		env[i] = p;
+		argv_array_push(env, p);
 		p += strlen(p) + 1;
 	}
-	env[count] = NULL;
 
-	return env;
+	return 0;
 }
 
 static const char staged_changes_advice[] =
@@ -612,14 +604,12 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty, int edit, int amend,
 			  int cleanup_commit_message)
 {
-	char **env = NULL;
-	struct argv_array array;
+	struct argv_array env = ARGV_ARRAY_INIT, array;
 	int rc;
 	const char *value;
 
 	if (is_rebase_i(opts)) {
-		env = read_author_script();
-		if (!env) {
+		if (!read_env_script(&env)) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
 			return error(_(staged_changes_advice),
@@ -655,9 +645,9 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "--allow-empty-message");
 
 	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
-			(const char *const *)env);
+			(const char *const *)env.argv);
 	argv_array_clear(&array);
-	free(env);
+	argv_array_clear(&env);
 
 	return rc;
 }
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 34/38] sequencer (rebase -i): show only failed cherry-picks' output

This is the behavior of the shell script version of the interactive rebase, by using the `output` function defined in `git-rebase.sh`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 4 ++++
 1 file changed, 4 insertions(+)
Show changes to sequencer.c +4 −0
diff --git a/sequencer.c b/sequencer.c
index a501dfce38..4f37ba8d33 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -433,6 +433,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	o.ancestor = base ? base_label : "(empty tree)";
 	o.branch1 = "HEAD";
 	o.branch2 = next ? next_label : "(empty tree)";
+	if (is_rebase_i(opts))
+		o.buffer_output = 2;
 
 	head_tree = parse_tree_indirect(head);
 	next_tree = next ? next->tree : empty_tree();
@@ -444,6 +446,8 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	clean = merge_trees(&o,
 			    head_tree,
 			    next_tree, base_tree, &result);
+	if (is_rebase_i(opts) && clean <= 0)
+		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
 	if (clean < 0)
 		return clean;
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 35/38] sequencer (rebase -i): suggest --edit-todo upon unknown command

This is the same behavior as known from `git rebase -i`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Show changes to sequencer.c +5 −1
diff --git a/sequencer.c b/sequencer.c
index 4f37ba8d33..4792a3de3b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1314,8 +1314,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (res)
+	if (res) {
+		if (is_rebase_i(opts))
+			return error(_("please fix this using "
+				       "'git rebase --edit-todo'."));
 		return error(_("unusable instruction sheet: '%s'"), todo_file);
+	}
 
 	if (!todo_list->nr &&
 	    (!is_rebase_i(opts) || !file_exists(rebase_path_done())))
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:35 UTC · re: Johannes Schindelin · lore

[PATCH v3 36/38] sequencer (rebase -i): show the progress

The interactive rebase keeps the user informed about its progress. If the sequencer wants to do the grunt work of the interactive rebase, it also needs to show that progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
Show changes to sequencer.c +32 −0
diff --git a/sequencer.c b/sequencer.c
index 4792a3de3b..2c9c555ab6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1181,6 +1181,7 @@ struct todo_list {
 	struct strbuf buf;
 	struct todo_item *items;
 	int nr, alloc, current;
+	int done_nr, total_nr;
 };
 
 #define TODO_LIST_INIT { STRBUF_INIT }
@@ -1297,6 +1298,17 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 	return res;
 }
 
+static int count_commands(struct todo_list *todo_list)
+{
+	int count = 0, i;
+
+	for (i = 0; i < todo_list->nr; i++)
+		if (todo_list->items[i].command != TODO_COMMENT)
+			count++;
+
+	return count;
+}
+
 static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
@@ -1339,6 +1351,21 @@ static int read_populate_todo(struct todo_list *todo_list,
 				return error(_("cannot revert during a cherry-pick."));
 	}
 
+	if (is_rebase_i(opts)) {
+		struct todo_list done = TODO_LIST_INIT;
+
+		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
+				!parse_insn_buffer(done.buf.buf, &done))
+			todo_list->done_nr = count_commands(&done);
+		else
+			todo_list->done_nr = 0;
+
+		todo_list->total_nr = todo_list->done_nr
+			+ count_commands(todo_list);
+
+		todo_list_release(&done);
+	}
+
 	return 0;
 }
 
@@ -1920,6 +1947,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
+			if (item->command != TODO_COMMENT)
+				fprintf(stderr, "Rebasing (%d/%d)%s",
+					++(todo_list->done_nr),
+					todo_list->total_nr,
+					opts->verbose ? "\n" : "\r");
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:36 UTC · re: Johannes Schindelin · lore

[PATCH v3 37/38] sequencer (rebase -i): write the progress into files

For the benefit of e.g. the shell prompt, the interactive rebase not only displays the progress for the user to see, but also writes it into the msgnum/end files in the state directory.

Teach the sequencer this new trick.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
Show changes to sequencer.c +27 −3
diff --git a/sequencer.c b/sequencer.c
index 2c9c555ab6..b39cd21e03 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -47,6 +47,16 @@ static GIT_PATH_FUNC(rebase_path_todo, "rebase-merge/git-rebase-todo")
  */
 static GIT_PATH_FUNC(rebase_path_done, "rebase-merge/done")
 /*
+ * The file to keep track of how many commands were already processed (e.g.
+ * for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgnum, "rebase-merge/msgnum");
+/*
+ * The file to keep track of how many commands are to be processed in total
+ * (e.g. for the prompt).
+ */
+static GIT_PATH_FUNC(rebase_path_msgtotal, "rebase-merge/end");
+/*
  * The commit message that is planned to be used for any changes that
  * need to be committed following a user interaction.
  */
@@ -1353,6 +1363,7 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 	if (is_rebase_i(opts)) {
 		struct todo_list done = TODO_LIST_INIT;
+		FILE *f = fopen(rebase_path_msgtotal(), "w");
 
 		if (strbuf_read_file(&done.buf, rebase_path_done(), 0) > 0 &&
 				!parse_insn_buffer(done.buf.buf, &done))
@@ -1362,8 +1373,12 @@ static int read_populate_todo(struct todo_list *todo_list,
 
 		todo_list->total_nr = todo_list->done_nr
 			+ count_commands(todo_list);
-
 		todo_list_release(&done);
+
+		if (f) {
+			fprintf(f, "%d\n", todo_list->total_nr);
+			fclose(f);
+		}
 	}
 
 	return 0;
@@ -1947,11 +1962,20 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		if (save_todo(todo_list, opts))
 			return -1;
 		if (is_rebase_i(opts)) {
-			if (item->command != TODO_COMMENT)
+			if (item->command != TODO_COMMENT) {
+				FILE *f = fopen(rebase_path_msgnum(), "w");
+
+				todo_list->done_nr++;
+
+				if (f) {
+					fprintf(f, "%d\n", todo_list->done_nr);
+					fclose(f);
+				}
 				fprintf(stderr, "Rebasing (%d/%d)%s",
-					++(todo_list->done_nr),
+					todo_list->done_nr,
 					todo_list->total_nr,
 					opts->verbose ? "\n" : "\r");
+			}
 			unlink(rebase_path_message());
 			unlink(rebase_path_author_script());
 			unlink(rebase_path_stopped_sha());
-- 
2.11.0.rc3.windows.1
Johannes Schindelin· Jan 2, 2017, 15:36 UTC · re: Johannes Schindelin · lore

[PATCH v3 38/38] sequencer (rebase -i): write out the final message

The shell script version of the interactive rebase has a very specific final message. Teach the sequencer to print the same.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)
Show changes to sequencer.c +3 −0
diff --git a/sequencer.c b/sequencer.c
index b39cd21e03..0e7d2ca5c8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2127,6 +2127,9 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		apply_autostash(opts);
 
+		fprintf(stderr, "Successfully rebased and updated %s.\n",
+			head_ref.buf);
+
 		strbuf_release(&buf);
 		strbuf_release(&head_ref);
 	}
-- 
2.11.0.rc3.windows.1
Junio C Hamano· Jan 9, 2017, 22:26 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

Johannes Schindelin <johannes.schindelin@gmx.de> writes:
Show 6 quoted lines
> Changes since v2:
>
> - fixed a TRANSLATORS: comment
> ...
> - replaced a spawned `diff-tree` command by code using the diff functions
>   directly

I just finished skimming the interdiff (the difference between the result of merging the v2 into 'master' and the result of applying this series on 'master'). I didn't cross reference it with all the review comments that I saw on the list for the previous round, I wouldn't have noticed even if there are issues that still haven't been addressed. But what I saw in the interdiff were mostly good changes.

I however think that the renaming of read_author_script() is totally backwards from maintainability's point of view.

Both "am" and "rebase" use an external file whose name is the same "author-script" (because they are meant to serve the same purpose), and introduction of new code that is different from what "am" uses to read it in v2 was bad enough. If the rename of the function in this round really means "author-script of 'am' can hold only author information, but that of 'rebase' can hold anything the end user desires", it makes things even worse. If 'rebase' side extends the file to hold other environment variables (in its own code), users would wonder why the same extension will not be available in 'am' side (and vice versa).

I ran out of time for the day and haven't looked at individual patches yet. I may find other issues, but from the interdiff, that was the only thing that I found even worse than the previous round, and everything else was either the same or has improved.

Thanks.
Junio C Hamano· Jan 9, 2017, 23:02 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

Junio C Hamano <gitster@pobox.com> writes:
> I ran out of time for the day and haven't looked at individual
> patches yet.  I may find other issues, but from the interdiff, that
> was the only thing that I found even worse than the previous round,
> and everything else was either the same or has improved.
... I forgot to say a few things:
 * In the meantime, I think this is overall better than what we had
   for the past few weeks; I'll try to replace the copy in my tree
   with this round, even though this may not be the final one.
 * There were a few style issues [*1*] pointed out by the
   checkpatch.pl script borrowed from the kernel land.
 * And finally, you wrote "This marks the count down to '3': two
   more patch series after this (really tiny ones) and we have a
   faster rebase -i."  This is seriously an exciting news.  Thanks.

I probably am still forgetting to say a few more things, but I'd send this message out in this shape for now. Thanks.

[Footnote]
*1*
Subject: [PATCH v3 10/38] sequencer (rebase -i): add support for the 'fixup' and 'squash' commands
ERROR: do not use assignment in if condition
#118: FILE: sequencer.c:725:
+		if (buf.buf[0] != comment_line_char ||
ERROR: do not use assignment in if condition
#142: FILE: sequencer.c:749:
+		if (!(head_commit = lookup_commit_reference(head)))
ERROR: do not use assignment in if condition
#144: FILE: sequencer.c:751:
+		if (!(head_message = get_commit_buffer(head_commit, NULL)))
ERROR: do not use assignment in if condition
#167: FILE: sequencer.c:774:
+	if (!(message = get_commit_buffer(commit, NULL)))
total: 4 errors, 1 warnings, 320 lines checked
------------------------------------------------
Subject: [PATCH v3 12/38] sequencer (rebase -i): write an author-script file
ERROR: do not use assignment in if condition
#35: FILE: sequencer.c:500:
+		else if ((eol = strchr(message, '\n')))
total: 1 errors, 0 warnings, 62 lines checked
------------------------------------------------
Subject: [PATCH v3 23/38] sequencer (rebase -i): copy commit notes at end
ERROR: open brace '{' following function declarations go on the next line
#36: FILE: sequencer.c:862:
+static void flush_rewritten_pending(void) {
ERROR: do not use assignment in if condition
#41: FILE: sequencer.c:867:
+	if (strbuf_read_file(&buf, rebase_path_rewritten_pending(), 82) > 0 &&
total: 2 errors, 0 warnings, 112 lines checked
Johannes Schindelin· Jan 14, 2017, 18:04 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

Hi Junio,
On Mon, 9 Jan 2017, Junio C Hamano wrote:
Show 12 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Changes since v2:
> >
> > - fixed a TRANSLATORS: comment
> > ...
> > - replaced a spawned `diff-tree` command by code using the diff functions
> >   directly
> 
> I just finished skimming the interdiff (the difference between the
> result of merging the v2 into 'master' and the result of applying
> this series on 'master').

I wish you would not have skimmed it, but provided a thorough review. There was a rather serious bug in this (not the first problem introduced into one of my patch series *because of* code review, unhidden-git and mmap-regexec are also very recent examples, I really should learn to resist the prodding to replace well-tested code with code of unknown correctness).

The problem in this instance was that the authorship is no longer retained when continuing after resolving a conflict. Let me stress again that this has not been a problem with v1 of sequencer-i, nor with v2. The regression was caused by changes required by the code review.

In case you wonder: Yes, I am upset by this.
The required fixup patch is:
-- snipsnap --
Subject: [PATCH] fixup! sequencer: make reading author-script more elegant

An unfortunate regression of formerly battle-tested code sadly crept into Git for Windows v2.11.0(2): authorship was not retained in case of conflicts during picks.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                   |  2 +-
 t/t3404-rebase-interactive.sh | 16 ++++++++++++++++
 2 files changed, 17 insertions(+), 1 deletion(-)
Show changes to 2 files +17 −1

sequencer.c, t/t3404-rebase-interactive.sh

diff --git a/sequencer.c b/sequencer.c
index 73b2ec6894..8ecab02291 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -612,7 +612,7 @@ static int read_env_script(struct argv_array *env)
 			count++;
 		}
 
-	for (i = 0; i < count; i++) {
+	for (i = 0, p = script.buf; i < count; i++) {
 		argv_array_push(env, p);
 		p += strlen(p) + 1;
 	}
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 71b9c8ef8b..61113be08a 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -237,6 +237,22 @@ test_expect_success 'retain authorship' '
 	git show HEAD | grep "^Author: Twerp Snog"
 '
 
+test_expect_success 'retain authorship w/ conflicts' '
+	git reset --hard twerp &&
+	test_commit a conflict a conflict-a &&
+	git reset --hard twerp &&
+	GIT_AUTHOR_NAME=AttributeMe \
+	test_commit b conflict b conflict-b &&
+	set_fake_editor &&
+	test_must_fail git rebase -i conflict-a &&
+	echo resolved >conflict &&
+	git add conflict &&
+	git rebase --continue &&
+	test $(git rev-parse conflict-a^0) = $(git rev-parse HEAD^) &&
+	git show >out &&
+	grep AttributeMe out
+'
+
 test_expect_success 'squash' '
 	git reset --hard twerp &&
 	echo B > file7 &&
-- 
2.11.0.windows.3
Junio C Hamano· Jan 15, 2017, 02:50 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 9 quoted lines
> The problem in this instance was that the authorship is no longer retained
> when continuing after resolving a conflict. Let me stress again that this
> has not been a problem with v1 of sequencer-i, nor with v2. The regression
> was caused by changes required by the code review.
>
> In case you wonder: Yes, I am upset by this.
>
> -- snipsnap --
> Subject: [PATCH] fixup! sequencer: make reading author-script more elegant

I do not think anybody asked to make the code "more elegant". Quite frankly, I do not expect elegance in your code (or any of the code in our codebase, for that matter). What we want is readable code that does not make the overall codebase less maintainable that is correct. Not reinveting a new codepath when there is already code that does the thing is one of the things that we may need to do, but that was not done between these rerolls.

Of course, when trying to share code, the existing code we have that the new codepath needs to borrow would have to be refactored and extended, and a new bug can sneak in during the process. If that were what happened, I would be a bit more sympathetic, but I suspect that this "more elegant" thing that needed fix-up is not that.

You may be upset, but I cannot quite bring myself to feel sympathy in this particular case.

Johannes Schindelin· Jan 14, 2017, 18:08 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

Hi Junio,
On Mon, 9 Jan 2017, Junio C Hamano wrote:
> I however think that the renaming of read_author_script() is totally
> backwards from maintainability's point of view.  

You stated elsewhere that converting a script into a builtin should focus on a faithful conversion.

The original code is:
	. "$author_script"

Granted, this *cannot* be converted faithfully without reimplementing a shell interpreter. So I did the next best thing: I converted it into code that reads a block of environment variable settings.

What you asked for is totally unreasonable: you ask me to make this conversion *even less faithful*.

What is worse: you argue from the "maintainability point of view", when it is pretty obvious that *adding validation that was not there before* can in no way make the code more maintainable, as it *adds new logic*.

And what is the worst: over all these discussions about a nothingburger (you simply cannot convince me that I should introduce validating code that has not been there before in the same patch series that simply tries to recreate existing functionality), the most important part of a code review was forgotten: to make sure that the changes are correct.

The worst direction a code review can take is to introduce regressions, and that is exactly what happened.

Ciao, Johannes

Junio C Hamano· Jan 15, 2017, 07:47 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 10 quoted lines
> You stated elsewhere that converting a script into a builtin should focus
> on a faithful conversion.
>
> The original code is:
>
> 	. "$author_script"
>
> Granted, this *cannot* be converted faithfully without reimplementing a
> shell interpreter. So I did the next best thing: I converted it into code
> that reads a block of environment variable settings.

It is unfortunate that you took "faithful" too literally. While I do appreciate it if your conversion aims to faithfully replicate the original, even to be bug-to-bug compatible, but obviously we cannot replicate everything the above original _could_ do.

By "a faithful conversion", I meant that the behaviour of the reimplementation should be as faithful to the original's intent as possible. Nothing more.

The intent of the above original is to read back what we wrote to preserve the author identity we learned earlier when we wrote the file. We read the "author" line from the commit object header, and write assignments to GIT_AUTHOR_{NAME,EMAIL,DATE} variables. Nothing more is intended.

The end-users COULD abuse the original code to cause it to do a lot more than that, e.g. by adding "export FOO=BAR" at the end and have the value of the new environment variable propagate throughout the code and even down to subprocesses. They can even assign to some variables we use internally for bookkeeping and break "rebase -i" machinery. But that is outside the intent of the original code--we do not need to or want to replicate that faithfully.

I also need to react to the "environment variable settings" at the end of the quoted paragraph.

If the code in the sequencer.c reads things other than the three variables we ourselves set, and make them into environment variables and propagate to subprocesses (hooks and editors), it would be a bug. The original did not intend to do that (the dot-sourcing is overly loose than reading three known variables and nothing else, but is OK because we do not support the case where end users muck with the file). Also, writing FOO=BAR alone (not "export FOO=BAR" or "FOO=BAR; export FOO") to the file wouldn't have exported FOO to subprocesses anyway.

Johannes Schindelin· Jan 16, 2017, 10:45 UTC · re: Junio C Hamano · lore

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

Hi Junio,
On Sat, 14 Jan 2017, Junio C Hamano wrote:
Show 20 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > You stated elsewhere that converting a script into a builtin should focus
> > on a faithful conversion.
> >
> > The original code is:
> >
> > 	. "$author_script"
> 
> [...]
>
> If the code in the sequencer.c reads things other than the three
> variables we ourselves set, and make them into environment variables
> and propagate to subprocesses (hooks and editors), it would be a
> bug.  The original did not intend to do that (the dot-sourcing is
> overly loose than reading three known variables and nothing else,
> but is OK because we do not support the case where end users muck
> with the file).  Also, writing FOO=BAR alone (not "export FOO=BAR"
> or "FOO=BAR; export FOO") to the file wouldn't have exported FOO to
> subprocesses anyway.

That analysis cannot be completely correct, as the GIT_AUTHOR_* variables *are* used by the `git commit` subprocess.

In any case, it is clear that we (I include all reviewers here) messed this patch series up quite a bit. Hence I will be more careful from now on to only act on suggestions that do, in fact, improve the patch series from my point of view.

Ciao, Johannes

Junio C Hamano· Jan 17, 2017, 19:50 UTC · re: Johannes Schindelin · lore

Re: [PATCH v3 00/38] Teach the sequencer to act as rebase -i's backend

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 18 quoted lines
>> > The original code is:
>> >
>> > 	. "$author_script"
>> 
>> [...]
>>
>> If the code in the sequencer.c reads things other than the three
>> variables we ourselves set, and make them into environment variables
>> and propagate to subprocesses (hooks and editors), it would be a
>> bug.  The original did not intend to do that (the dot-sourcing is
>> overly loose than reading three known variables and nothing else,
>> but is OK because we do not support the case where end users muck
>> with the file).  Also, writing FOO=BAR alone (not "export FOO=BAR"
>> or "FOO=BAR; export FOO") to the file wouldn't have exported FOO to
>> subprocesses anyway.
>
> That analysis cannot be completely correct, as the GIT_AUTHOR_* variables
> *are* used by the `git commit` subprocess.

In the scripted version, do_with_author shell function explicitly exports GIT_AUTHOR_* variables because they are the only ones we care about that are read from existing commit and carried forward via the author-script mechanism. We do not care about, and in fact we do not intend to support, any other variables or shell commands that appear in the author-script.

← back to recent threads