{"thread":{"id":"43983","subject":"[PATCH 1/9] rebase -i: generate the script via rebase--helper","startedAt":"2016-09-02T16:23:14Z","lastAt":"2017-06-01T06:48:31Z","messageCount":100,"participants":["Johannes Schindelin","Dennis Kaarsemaker","Josh Triplett","Junio C Hamano","Jeff King","Phillip Wood","Liam Beguin","Stefan Beller","liam Beguin","René Scharfe","Ævar Arnfjörð Bjarmason"],"isPatch":true,"patchVersion":1,"patchTotal":9},"messages":[{"id":"300888","messageId":"be9c724f1d664d1c85dd1df71e352e3ac1ccc0f9.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:01Z","receivedAt":"2016-09-02T16:23:14Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The first step of an interactive rebase is to generate the so-called \"todo\nscript\", to be stored in the state directory as \"git-rebase-todo\" and to\nbe edited by the user.\n\nOriginally, we adjusted the output of `git log ` using a simple\nsed script. Over the course of the years, the code became more\ncomplicated. We now use shell scripting to edit the output of `git log`\nconditionally, depending whether to keep \"empty\" commits (i.e. commits\nthat do not change any files).\n\nOn platforms where shell scripting is not native, this can be a serious\ndrag. And it opens the door for incompatibilities between platforms when\nit comes to shell scripting or to Unix-y commands.\n\nLet's just re-implement the todo script generation in plain C, using the\nrevision machinery directly.\n\nThis is substantially faster, improving the speed relative to the\nshell script version of the interactive rebase from 2x to 3x on Windows.\n\nNote that the rearrange_squash() function in git-rebase--interactive\nrelied on the fact that we set the \"format\" variable to the config setting\nrebase.instructionFormat. Relying on a side effect like this is no good,\nhence we explicitly perform that assignment (possibly again) in\nrearrange_squash().\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 8 +++++++-\n git-rebase--interactive.sh | 44 +++++++++++++++++++++++---------------------\n sequencer.c | 44 ++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 2 ++\n 4 files changed, 76 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex ca1ebb2..821058d 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -11,15 +11,19 @@ static const char * const builtin_rebase_helper_usage[] = {\n int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n {\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n+\tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n+\t\tOPT_BOOL(0, \"keep-empty\", &keep_empty, N_(\"keep empty commits\")),\n \t\tOPT_CMDMODE(0, \"continue\", &command, N_(\"continue rebase\"),\n \t\t\t\tCONTINUE),\n \t\tOPT_CMDMODE(0, \"abort\", &command, N_(\"abort rebase\"),\n \t\t\t\tABORT),\n+\t\tOPT_CMDMODE(0, \"make-script\", &command,\n+\t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n \t\tOPT_END()\n \t};\n \n@@ -36,5 +40,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!sequencer_continue(&opts);\n \tif (command == ABORT && argc == 1)\n \t\treturn !!sequencer_remove_state(&opts);\n+\tif (command == MAKE_SCRIPT && argc > 1)\n+\t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 022766b..01c9fec 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -775,6 +775,7 @@ collapse_todo_ids() {\n # each log message will be re-retrieved in order to normalize the\n # autosquash arrangement\n rearrange_squash () {\n+\tformat=$(git config --get rebase.instructionFormat)\n \t# extract fixup!/squash! lines and resolve any referenced sha1's\n \twhile read -r pick sha1 message\n \tdo\n@@ -1203,26 +1204,27 @@ else\n \trevisions=$onto...$orig_head\n \tshortrevisions=$shorthead\n fi\n-format=$(git config --get rebase.instructionFormat)\n-# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n-git rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n-\t--reverse --left-right --topo-order \\\n-\t$revisions ${restrict_revision+^$restrict_revision} | \\\n-\tsed -n \"s/^>//p\" |\n-while read -r sha1 rest\n-do\n-\n-\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n-\tthen\n-\t\tcomment_out=\"$comment_char \"\n-\telse\n-\t\tcomment_out=\n-\tfi\n+if test t != \"$preserve_merges\"\n+then\n+\tgit rebase--helper --make-script ${keep_empty:+--keep-empty} \\\n+\t\t$revisions ${restrict_revision+^$restrict_revision} >\"$todo\"\n+else\n+\tformat=$(git config --get rebase.instructionFormat)\n+\t# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n+\tgit rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n+\t\t--reverse --left-right --topo-order \\\n+\t\t$revisions ${restrict_revision+^$restrict_revision} | \\\n+\t\tsed -n \"s/^>//p\" |\n+\twhile read -r sha1 rest\n+\tdo\n+\n+\t\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n+\t\tthen\n+\t\t\tcomment_out=\"$comment_char \"\n+\t\telse\n+\t\t\tcomment_out=\n+\t\tfi\n \n-\tif test t != \"$preserve_merges\"\n-\tthen\n-\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n-\telse\n \t\tif test -z \"$rebase_root\"\n \t\tthen\n \t\t\tpreserve=t\n@@ -1241,8 +1243,8 @@ do\n \t\t\ttouch \"$rewritten\"/$sha1\n \t\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n \t\tfi\n-\tfi\n-done\n+\tdone\n+fi\n \n # Watch for commits that been dropped by --cherry-pick\n if test t = \"$preserve_merges\"\ndiff --git a/sequencer.c b/sequencer.c\nindex c0c6661..43e078a 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2347,3 +2347,47 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n \n \tstrbuf_release(&sob);\n }\n+\n+int sequencer_make_script(int keep_empty, FILE *out,\n+\t\tint argc, const char **argv)\n+{\n+\tchar *format = \"%s\";\n+\tstruct pretty_print_context pp = {0};\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct rev_info revs;\n+\tstruct commit *commit;\n+\n+\tinit_revisions(&revs, NULL);\n+\trevs.verbose_header = 1;\n+\trevs.max_parents = 1;\n+\trevs.cherry_pick = 1;\n+\trevs.limited = 1;\n+\trevs.reverse = 1;\n+\trevs.right_only = 1;\n+\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n+\trevs.topo_order = 1;\n+\n+\trevs.pretty_given = 1;\n+\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n+\tget_commit_format(format, &revs);\n+\tpp.fmt = revs.commit_format;\n+\tpp.output_encoding = get_log_output_encoding();\n+\n+\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n+\t\treturn error(\"make_script: unhandled options\");\n+\n+\tif (prepare_revision_walk(&revs) < 0)\n+\t\treturn error(\"make_script: error preparing revisions\");\n+\n+\twhile ((commit = get_revision(&revs))) {\n+\t\tstrbuf_reset(&buf);\n+\t\tif (!keep_empty && is_original_commit_empty(commit))\n+\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n+\t\tstrbuf_addf(&buf, \"pick %s \", oid_to_hex(&commit->object.oid));\n+\t\tpretty_print_commit(&pp, commit, &buf);\n+\t\tstrbuf_addch(&buf, '\\n');\n+\t\tfputs(buf.buf, out);\n+\t}\n+\tstrbuf_release(&buf);\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex fd2a719..bc524be 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -58,6 +58,8 @@ int sequencer_remove_state(struct replay_opts *opts);\n int sequencer_commit(const char *defmsg, struct replay_opts *opts,\n \t\t\t int allow_empty, int edit, int amend,\n \t\t\t int cleanup_commit_message);\n+int sequencer_make_script(int keep_empty, FILE *out,\n+\t\tint argc, const char **argv);\n \n extern const char sign_off_header[];\n \n-- \n2.9.3.windows.3\n\n\n"},{"id":"300889","messageId":"675e4059876782b7283913dd53488ed814eed137.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 2/9] rebase -i: remove useless indentation","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:04Z","receivedAt":"2016-09-02T16:23:17Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The commands used to be indented, and it is nice to look at, but when we\ntransform the SHA-1s, the indentation is removed. So let's do away with it.\n\nFor the moment, at least: when we will use the upcoming rebase--helper\nto transform the SHA-1s, we *will* keep the indentation and can\nreintroduce it. Yet, to be able to validate the rebase--helper against\nthe output of the current shell script version, we need to remove the\nextra indentation.\n\nSigned-off-by: Johannes Schindelin \n---\n git-rebase--interactive.sh | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 01c9fec..5df5850 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -146,13 +146,13 @@ reschedule_last_action () {\n append_todo_help () {\n \tgettext \"\n Commands:\n- p, pick = use commit\n- r, reword = use commit, but edit the commit message\n- e, edit = use commit, but stop for amending\n- s, squash = use commit, but meld into previous commit\n- f, fixup = like \\\"squash\\\", but discard this commit's log message\n- x, exec = run command (the rest of the line) using shell\n- d, drop = remove commit\n+p, pick = use commit\n+r, reword = use commit, but edit the commit message\n+e, edit = use commit, but stop for amending\n+s, squash = use commit, but meld into previous commit\n+f, fixup = like \\\"squash\\\", but discard this commit's log message\n+x, exec = run command (the rest of the line) using shell\n+d, drop = remove commit\n \n These lines can be re-ordered; they are executed from top to bottom.\n \" | git stripspace --comment-lines >>\"$todo\"\n-- \n2.9.3.windows.3\n\n\n"},{"id":"300890","messageId":"cover.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":null,"subject":"[PATCH 0/9] The final building block for a faster rebase -i","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:22:56Z","receivedAt":"2016-09-02T16:23:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This patch series reimplements the expensive pre- and post-processing of\nthe todo script in C.\n\nAnd it concludes the work I did to accelerate rebase -i.\n\n\nJohannes Schindelin (9):\n rebase -i: generate the script via rebase--helper\n rebase -i: remove useless indentation\n rebase -i: do not invent onelines when expanding/collapsing SHA-1s\n rebase -i: also expand/collapse the SHA-1s via the rebase--helper\n t3404: relax rebase.missingCommitsCheck tests\n rebase -i: check for missing commits in the rebase--helper\n rebase -i: skip unnecessary picks using the rebase--helper\n t3415: test fixup with wrapped oneline\n rebase -i: rearrange fixup/squash lines using the rebase--helper\n\n builtin/rebase--helper.c | 29 ++-\n git-rebase--interactive.sh | 362 ++++-------------------------\n sequencer.c | 514 ++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 7 +\n t/t3404-rebase-interactive.sh | 22 +-\n t/t3415-rebase-autosquash.sh | 16 +-\n 6 files changed, 614 insertions(+), 336 deletions(-)\n\nBased-On: rebase--helper at https://github.com/dscho/git\nFetch-Base-Via: git fetch https://github.com/dscho/git rebase--helper\nPublished-As: https://github.com/dscho/git/releases/tag/rebase-i-extra-v1\nFetch-It-Via: git fetch https://github.com/dscho/git rebase-i-extra-v1\n\n-- \n2.9.3.windows.3\n\nbase-commit: 4c39918f42eb8228ea4241073f86f2ac851f4636\n"},{"id":"300891","messageId":"943801411097f3f68004423bc8a78725c8023dc2.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 3/9] rebase -i: do not invent onelines when expanding/collapsing SHA-1s","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:08Z","receivedAt":"2016-09-02T16:23:23Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To avoid problems with short SHA-1s that become non-unique during the\nrebase, we rewrite the todo script with short/long SHA-1s before and\nafter letting the user edit the script. Since SHA-1s are not intuitive\nfor humans, rebase -i also provides the onelines (commit message\nsubjects) in the script, purely for the user's convenience.\n\nIt is very possible to generate a todo script via different means than\nrebase -i and then to let rebase -i run with it; In this case, these\nonelines are not required.\n\nAnd this is where the expand/collapse machinery has a bug: it *expects*\nthat oneline, and failing to find one reuses the previous SHA-1 as\n\"oneline\".\n\nIt was most likely an oversight, and made implementation in the (quite\nlimiting) shell script language less convoluted. However, we are about\nto reimplement performance-critical parts in C (and due to spawning a\ngit.exe process for every single line of the todo script, the\nexpansion/collapsing of the SHA-1s *is* performance-hampering on\nWindows), therefore let's fix this bug to make cross-validation with the\nC version of that functionality possible.\n\nSigned-off-by: Johannes Schindelin \n---\n git-rebase--interactive.sh | 7 ++++++-\n 1 file changed, 6 insertions(+), 1 deletion(-)\n\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 5df5850..0eb5583 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -750,7 +750,12 @@ transform_todo_ids () {\n \t\t\t;;\n \t\t*)\n \t\t\tsha1=$(git rev-parse --verify --quiet \"$@\" ${rest%%[\t ]*}) &&\n-\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n+\t\t\tif test \"a$rest\" = \"a${rest#*[\t ]}\"\n+\t\t\tthen\n+\t\t\t\trest=$sha1\n+\t\t\telse\n+\t\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n+\t\t\tfi\n \t\t\t;;\n \t\tesac\n \t\tprintf '%s\\n' \"$command${rest:+ }$rest\"\n-- \n2.9.3.windows.3\n\n\n"},{"id":"300892","messageId":"fe53c109c757ede9cd1b0d2944d2b5e16575c49b.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:10Z","receivedAt":"2016-09-02T16:23:26Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This is crucial to improve performance on Windows, as the speed is now\nmostly dominated by the SHA-1 transformation (because it spawns a new\nrev-parse process for *every* line, and spawning processes is pretty\nslow from Git for Windows' MSYS2 Bash).\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 10 +++++++-\n git-rebase--interactive.sh | 4 ++--\n sequencer.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 2 ++\n 4 files changed, 72 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex 821058d..9444c8d 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n \tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -24,6 +24,10 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\t\tABORT),\n \t\tOPT_CMDMODE(0, \"make-script\", &command,\n \t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n+\t\tOPT_CMDMODE(0, \"shorten-sha1s\", &command,\n+\t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n+\t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n+\t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n \t\tOPT_END()\n \t};\n \n@@ -42,5 +46,9 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!sequencer_remove_state(&opts);\n \tif (command == MAKE_SCRIPT && argc > 1)\n \t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n+\tif (command == SHORTEN_SHA1S && argc == 1)\n+\t\treturn !!transform_todo_ids(1);\n+\tif (command == EXPAND_SHA1S && argc == 1)\n+\t\treturn !!transform_todo_ids(0);\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 0eb5583..f642ec2 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -764,11 +764,11 @@ transform_todo_ids () {\n }\n \n expand_todo_ids() {\n-\ttransform_todo_ids\n+\tgit rebase--helper --expand-sha1s\n }\n \n collapse_todo_ids() {\n-\ttransform_todo_ids --short\n+\tgit rebase--helper --shorten-sha1s\n }\n \n # Rearrange the todo list that has both \"pick sha1 msg\" and\ndiff --git a/sequencer.c b/sequencer.c\nindex 43e078a..ee4fdb0 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2391,3 +2391,62 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \tstrbuf_release(&buf);\n \treturn 0;\n }\n+\n+\n+int transform_todo_ids(int shorten_sha1s)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tint fd, res, i;\n+\tFILE *out;\n+\n+\tstrbuf_reset(&todo_list.buf);\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0)\n+\t\treturn error_errno(_(\"Could not open '%s'\"), todo_file);\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"Could not read %s.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\n+\tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\tif (res) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn error(_(\"Unusable instruction sheet: %s\"), todo_file);\n+\t}\n+\n+\tout = fopen(todo_file, \"w\");\n+\tif (!out) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn error(_(\"Unable to open '%s' for writing\"), todo_file);\n+\t}\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tint bol = item->offset_in_buf;\n+\t\tconst char *p = todo_list.buf.buf + bol;\n+\t\tint eol = i + 1 < todo_list.nr ?\n+\t\t\ttodo_list.items[i + 1].offset_in_buf :\n+\t\t\ttodo_list.buf.len;\n+\n+\t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n+\t\t\tfwrite(p, eol - bol, 1, out);\n+\t\telse {\n+\t\t\tint eoc = strcspn(p, \" \\t\");\n+\t\t\tconst char *sha1 = shorten_sha1s ?\n+\t\t\t\tshort_commit_name(item->commit) :\n+\t\t\t\toid_to_hex(&item->commit->object.oid);\n+\n+\t\t\tif (!eoc) {\n+\t\t\t\tp += strspn(p, \" \\t\");\n+\t\t\t\teoc = strcspn(p, \" \\t\");\n+\t\t\t}\n+\n+\t\t\tfprintf(out, \"%.*s %s %.*s\\n\",\n+\t\t\t\teoc, p, sha1, item->arg_len, item->arg);\n+\t\t}\n+\t}\n+\tfclose(out);\n+\ttodo_list_release(&todo_list);\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex bc524be..5feb525 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -61,6 +61,8 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,\n int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv);\n \n+int transform_todo_ids(int shorten_sha1s);\n+\n extern const char sign_off_header[];\n \n void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n-- \n2.9.3.windows.3\n\n\n"},{"id":"300893","messageId":"b2f3188f323b70f935956256f037820125b16568.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 5/9] t3404: relax rebase.missingCommitsCheck tests","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:14Z","receivedAt":"2016-09-02T16:23:38Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"These tests were a bit anal about the *exact* warning/error message\nprinted by git rebase. But those messages are intended for the *end\nuser*, therefore it does not make sense to test so rigidly for the\n*exact* wording.\n\nIn the following, we will reimplement the missing commits check in\nthe sequencer, with slightly different words.\n\nSo let's just test for the parts in the warning/error message that\nwe *really* care about, nothing more, nothing less.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3404-rebase-interactive.sh | 22 ++++------------------\n 1 file changed, 4 insertions(+), 18 deletions(-)\n\ndiff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh\nindex 597e94e..a18759e 100755\n--- a/t/t3404-rebase-interactive.sh\n+++ b/t/t3404-rebase-interactive.sh\n@@ -1215,20 +1215,13 @@ test_expect_success 'rebase -i respects rebase.missingCommitsCheck = error' '\n \ttest B = $(git cat-file commit HEAD^ | sed -ne \\$p)\n '\n \n-cat >expect <actual &&\n-\ttest_i18ncmp expect actual &&\n+\ttest_i18ngrep \"badcmd $(git rev-list --oneline -1 master~1)\" actual &&\n+\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n \tFAKE_LINES=\"1 2 3 drop 4 5\" git rebase --edit-todo &&\n \tgit rebase --continue &&\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p) &&\n@@ -1250,20 +1243,13 @@ test_expect_success 'tabs and spaces are accepted in the todolist' '\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n '\n \n-cat >expect <actual &&\n-\ttest_i18ncmp expect actual &&\n+\ttest_i18ngrep \"edit XXXXXXX False commit\" actual &&\n+\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n \tFAKE_LINES=\"1 2 4 5 6\" git rebase --edit-todo &&\n \tgit rebase --continue &&\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n-- \n2.9.3.windows.3\n\n\n"},{"id":"300894","messageId":"6d28acb59d7c217f1f8d6d364e5a52c8bde35085.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 6/9] rebase -i: check for missing commits in the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:32Z","receivedAt":"2016-09-02T16:23:42Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"In particular on Windows, where shell scripts are even more expensive\nthan on MacOSX or Linux, it makes sense to move a loop that forks\nGit at least once for every line in the todo list into a builtin.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 7 +-\n git-rebase--interactive.sh | 164 ++-------------------------------------------\n sequencer.c | 124 ++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n 4 files changed, 136 insertions(+), 160 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex 9444c8d..e706eac 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -13,7 +13,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n \tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n+\t\tCHECK_TODO_LIST\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -28,6 +29,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n \t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n \t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n+\t\tOPT_CMDMODE(0, \"check-todo-list\", &command,\n+\t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n \t\tOPT_END()\n \t};\n \n@@ -50,5 +53,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!transform_todo_ids(1);\n \tif (command == EXPAND_SHA1S && argc == 1)\n \t\treturn !!transform_todo_ids(0);\n+\tif (command == CHECK_TODO_LIST && argc == 1)\n+\t\treturn !!check_todo_list();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex f642ec2..02a7698 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -880,96 +880,6 @@ add_exec_commands () {\n \tmv \"$1.new\" \"$1\"\n }\n \n-# Check if the SHA-1 passed as an argument is a\n-# correct one, if not then print $2 in \"$todo\".badsha\n-# $1: the SHA-1 to test\n-# $2: the line number of the input\n-# $3: the input filename\n-check_commit_sha () {\n-\tbadsha=0\n-\tif test -z \"$1\"\n-\tthen\n-\t\tbadsha=1\n-\telse\n-\t\tsha1_verif=\"$(git rev-parse --verify --quiet $1^{commit})\"\n-\t\tif test -z \"$sha1_verif\"\n-\t\tthen\n-\t\t\tbadsha=1\n-\t\tfi\n-\tfi\n-\n-\tif test $badsha -ne 0\n-\tthen\n-\t\tline=\"$(sed -n -e \"${2}p\" \"$3\")\"\n-\t\twarn \"$(eval_gettext \"\\\n-Warning: the SHA-1 is missing or isn't a commit in the following line:\n- - \\$line\")\"\n-\t\twarn\n-\tfi\n-\n-\treturn $badsha\n-}\n-\n-# prints the bad commits and bad commands\n-# from the todolist in stdin\n-check_bad_cmd_and_sha () {\n-\tretval=0\n-\tlineno=0\n-\twhile read -r command rest\n-\tdo\n-\t\tlineno=$(( $lineno + 1 ))\n-\t\tcase $command in\n-\t\t\"$comment_char\"*|''|noop|x|exec)\n-\t\t\t# Doesn't expect a SHA-1\n-\t\t\t;;\n-\t\t\"$cr\")\n-\t\t\t# Work around CR left by \"read\" (e.g. with Git for\n-\t\t\t# Windows' Bash).\n-\t\t\t;;\n-\t\tpick|p|drop|d|reword|r|edit|e|squash|s|fixup|f)\n-\t\t\tif ! check_commit_sha \"${rest%%[ \t]*}\" \"$lineno\" \"$1\"\n-\t\t\tthen\n-\t\t\t\tretval=1\n-\t\t\tfi\n-\t\t\t;;\n-\t\t*)\n-\t\t\tline=\"$(sed -n -e \"${lineno}p\" \"$1\")\"\n-\t\t\twarn \"$(eval_gettext \"\\\n-Warning: the command isn't recognized in the following line:\n- - \\$line\")\"\n-\t\t\twarn\n-\t\t\tretval=1\n-\t\t\t;;\n-\t\tesac\n-\tdone <\"$1\"\n-\treturn $retval\n-}\n-\n-# Print the list of the SHA-1 of the commits\n-# from stdin to stdout\n-todo_list_to_sha_list () {\n-\tgit stripspace --strip-comments |\n-\twhile read -r command sha1 rest\n-\tdo\n-\t\tcase $command in\n-\t\t\"$comment_char\"*|''|noop|x|\"exec\")\n-\t\t\t;;\n-\t\t*)\n-\t\t\tlong_sha=$(git rev-list --no-walk \"$sha1\" 2>/dev/null)\n-\t\t\tprintf \"%s\\n\" \"$long_sha\"\n-\t\t\t;;\n-\t\tesac\n-\tdone\n-}\n-\n-# Use warn for each line in stdin\n-warn_lines () {\n-\twhile read -r line\n-\tdo\n-\t\twarn \" - $line\"\n-\tdone\n-}\n-\n # Switch to the branch in $into and notify it in the reflog\n checkout_onto () {\n \tGIT_REFLOG_ACTION=\"$GIT_REFLOG_ACTION: checkout $onto_name\"\n@@ -984,74 +894,6 @@ get_missing_commit_check_level () {\n \tprintf '%s' \"$check_level\" | tr 'A-Z' 'a-z'\n }\n \n-# Check if the user dropped some commits by mistake\n-# Behaviour determined by rebase.missingCommitsCheck.\n-# Check if there is an unrecognized command or a\n-# bad SHA-1 in a command.\n-check_todo_list () {\n-\traise_error=f\n-\n-\tcheck_level=$(get_missing_commit_check_level)\n-\n-\tcase \"$check_level\" in\n-\twarn|error)\n-\t\t# Get the SHA-1 of the commits\n-\t\ttodo_list_to_sha_list <\"$todo\".backup >\"$todo\".oldsha1\n-\t\ttodo_list_to_sha_list <\"$todo\" >\"$todo\".newsha1\n-\n-\t\t# Sort the SHA-1 and compare them\n-\t\tsort -u \"$todo\".oldsha1 >\"$todo\".oldsha1+\n-\t\tmv \"$todo\".oldsha1+ \"$todo\".oldsha1\n-\t\tsort -u \"$todo\".newsha1 >\"$todo\".newsha1+\n-\t\tmv \"$todo\".newsha1+ \"$todo\".newsha1\n-\t\tcomm -2 -3 \"$todo\".oldsha1 \"$todo\".newsha1 >\"$todo\".miss\n-\n-\t\t# Warn about missing commits\n-\t\tif test -s \"$todo\".miss\n-\t\tthen\n-\t\t\ttest \"$check_level\" = error && raise_error=t\n-\n-\t\t\twarn \"$(gettext \"\\\n-Warning: some commits may have been dropped accidentally.\n-Dropped commits (newer to older):\")\"\n-\n-\t\t\t# Make the list user-friendly and display\n-\t\t\topt=\"--no-walk=sorted --format=oneline --abbrev-commit --stdin\"\n-\t\t\tgit rev-list $opt <\"$todo\".miss | warn_lines\n-\n-\t\t\twarn \"$(gettext \"\\\n-To avoid this message, use \\\"drop\\\" to explicitly remove a commit.\n-\n-Use 'git config rebase.missingCommitsCheck' to change the level of warnings.\n-The possible behaviours are: ignore, warn, error.\")\"\n-\t\t\twarn\n-\t\tfi\n-\t\t;;\n-\tignore)\n-\t\t;;\n-\t*)\n-\t\twarn \"$(eval_gettext \"Unrecognized setting \\$check_level for option rebase.missingCommitsCheck. Ignoring.\")\"\n-\t\t;;\n-\tesac\n-\n-\tif ! check_bad_cmd_and_sha \"$todo\"\n-\tthen\n-\t\traise_error=t\n-\tfi\n-\n-\tif test $raise_error = t\n-\tthen\n-\t\t# Checkout before the first commit of the\n-\t\t# rebase: this way git rebase --continue\n-\t\t# will work correctly as it expects HEAD to be\n-\t\t# placed before the commit of the next action\n-\t\tcheckout_onto\n-\n-\t\twarn \"$(gettext \"You can fix this with 'git rebase --edit-todo'.\")\"\n-\t\tdie \"$(gettext \"Or you can abort the rebase with 'git rebase --abort'.\")\"\n-\tfi\n-}\n-\n # The whole contents of this file is run by dot-sourcing it from\n # inside a shell function. It used to be that \"return\"s we see\n # below were not inside any function, and expected to return\n@@ -1315,7 +1157,11 @@ git_sequence_editor \"$todo\" ||\n has_action \"$todo\" ||\n \treturn 2\n \n-check_todo_list\n+git rebase--helper --check-todo-list || {\n+\tret=$?\n+\tcheckout_onto\n+\texit $ret\n+}\n \n expand_todo_ids\n \ndiff --git a/sequencer.c b/sequencer.c\nindex ee4fdb0..0c82925 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2450,3 +2450,127 @@ int transform_todo_ids(int shorten_sha1s)\n \ttodo_list_release(&todo_list);\n \treturn 0;\n }\n+\n+enum check_level {\n+\tCHECK_IGNORE = 0, CHECK_WARN, CHECK_ERROR\n+};\n+\n+static enum check_level get_missing_commit_check_level(void)\n+{\n+\tconst char *value;\n+\n+\tif (git_config_get_value(\"rebase.missingcommitscheck\", &value) ||\n+\t\t\t!strcasecmp(\"ignore\", value))\n+\t\treturn CHECK_IGNORE;\n+\tif (!strcasecmp(\"warn\", value))\n+\t\treturn CHECK_WARN;\n+\tif (!strcasecmp(\"error\", value))\n+\t\treturn CHECK_ERROR;\n+\twarning(_(\"Unrecognized setting $check_level for option\"\n+\t\t\t\"rebase.missingCommitsCheck. Ignoring.\"));\n+\treturn CHECK_IGNORE;\n+}\n+\n+/*\n+ * Check if the user dropped some commits by mistake\n+ * Behaviour determined by rebase.missingCommitsCheck.\n+ * Check if there is an unrecognized command or a\n+ * bad SHA-1 in a command.\n+ */\n+int check_todo_list(void)\n+{\n+\tenum check_level check_level = get_missing_commit_check_level();\n+\tstruct strbuf todo_file = STRBUF_INIT;\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct commit_list *missing = NULL;\n+\tint raise_error = 0, res = 0, fd, i;\n+\n+\tstrbuf_addstr(&todo_file, rebase_path_todo());\n+\tfd = open(todo_file.buf, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tres = error_errno(_(\"Could not open %s\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\tres = error(_(\"Could not read %s.\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tclose(fd);\n+\traise_error = res =\n+\t\tparse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\n+\tif (check_level == CHECK_IGNORE)\n+\t\tgoto leave_check;\n+\n+\t/* Get the SHA-1 of the commits */\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct commit *commit = todo_list.items[i].commit;\n+\t\tif (commit)\n+\t\t\tcommit->util = todo_list.items + i;\n+\t}\n+\n+\ttodo_list_release(&todo_list);\n+\tstrbuf_addstr(&todo_file, \".backup\");\n+\tfd = open(todo_file.buf, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tres = error_errno(_(\"Could not open %s\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\tres = error(_(\"Could not read %s.\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tclose(fd);\n+\tstrbuf_release(&todo_file);\n+\tres = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\n+\t/* Find commits that are missing after editing */\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct commit *commit = todo_list.items[i].commit;\n+\t\tif (commit && !commit->util) {\n+\t\t\tcommit_list_insert(commit, &missing);\n+\t\t\tcommit->util = todo_list.items + i;\n+\t\t}\n+\t}\n+\n+\t/* Warn about missing commits */\n+\tif (!missing)\n+\t\tgoto leave_check;\n+\n+\tif (check_level == CHECK_ERROR)\n+\t\traise_error = res = 1;\n+\n+\tfprintf(stderr,\n+\t\t_(\"Warning: some commits may have been dropped accidentally.\\n\"\n+\t\t\"Dropped commits (newer to older):\\n\"));\n+\n+\t/* Make the list user-friendly and display */\n+\twhile (missing) {\n+\t\tstruct commit *commit = pop_commit(&missing);\n+\t\tstruct todo_item *item = commit->util;\n+\n+\t\tfprintf(stderr, \" - %s %.*s\\n\", short_commit_name(commit),\n+\t\t\titem->arg_len, item->arg);\n+\t}\n+\tfree_commit_list(missing);\n+\n+\tfprintf(stderr, _(\"To avoid this message, use \\\"drop\\\" to \"\n+\t\t\"explicitly remove a commit.\\n\\n\"\n+\t\t\"Use 'git config rebase.missingCommitsCheck' to change \"\n+\t\t\"the level of warnings.\\n\"\n+\t\t\"The possible behaviours are: ignore, warn, error.\\n\\n\"));\n+\n+leave_check:\n+\tstrbuf_release(&todo_file);\n+\ttodo_list_release(&todo_list);\n+\n+\tif (raise_error)\n+\t\tfprintf(stderr,\n+\t\t\t_(\"You can fix this with 'git rebase --edit-todo'.\\n\"\n+\t\t\t\"Or you can abort the rebase with 'git rebase\"\n+\t\t\t\" --abort'.\\n\"));\n+\n+\treturn res;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 5feb525..8e3daf9 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -62,6 +62,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv);\n \n int transform_todo_ids(int shorten_sha1s);\n+int check_todo_list(void);\n \n extern const char sign_off_header[];\n \n-- \n2.9.3.windows.3\n\n\n"},{"id":"300895","messageId":"9fc25ce0614ea1b38b0e56143510b0f7290ac3dd.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 8/9] t3415: test fixup with wrapped oneline","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:39Z","receivedAt":"2016-09-02T16:23:47Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The `git commit --fixup` command unwraps wrapped onelines when\nconstructing the commit message, without wrapping the result.\n\nWe need to make sure that `git rebase --autosquash` keeps handling such\ncases correctly, in particular since we are about to move the autosquash\nhandling into the rebase--helper.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3415-rebase-autosquash.sh | 14 ++++++++++++++\n 1 file changed, 14 insertions(+)\n\ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 48346f1..9fd629a 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -304,4 +304,18 @@ test_expect_success 'extra spaces after fixup!' '\n \ttest $base = $parent\n '\n \n+test_expect_success 'wrapped original subject' '\n+\tif test -d .git/rebase-merge; then git rebase --abort; fi &&\n+\tbase=$(git rev-parse HEAD) &&\n+\techo \"wrapped subject\" >wrapped &&\n+\tgit add wrapped &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m \"$(printf \"To\\nfixup\")\" &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m \"fixup! To fixup\" &&\n+\tgit rebase -i --autosquash --keep-empty HEAD~2 &&\n+\tparent=$(git rev-parse HEAD^) &&\n+\ttest $base = $parent\n+'\n+\n test_done\n-- \n2.9.3.windows.3\n\n\n"},{"id":"300896","messageId":"a9e6611f0137efac719766fc3295ead4d8567c20.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 7/9] rebase -i: skip unnecessary picks using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:35Z","receivedAt":"2016-09-02T16:23:52Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"In particular on Windows, where shell scripts are even more expensive\nthan on MacOSX or Linux, it makes sense to move a loop that forks\nGit at least once for every line in the todo list into a builtin.\n\nNote: The original code did not try to skip unnecessary picks of root\ncommits but punts instead (probably --root was not considered common\nenough of a use case to bother optimizing). We do the same, for now.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 6 +++-\n git-rebase--interactive.sh | 41 ++-------------------\n sequencer.c | 90 ++++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n 4 files changed, 99 insertions(+), 39 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex e706eac..de3ccd9 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tint keep_empty = 0;\n \tenum {\n \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n-\t\tCHECK_TODO_LIST\n+\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -31,6 +31,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n \t\tOPT_CMDMODE(0, \"check-todo-list\", &command,\n \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n+\t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n+\t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n \t\tOPT_END()\n \t};\n \n@@ -55,5 +57,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!transform_todo_ids(0);\n \tif (command == CHECK_TODO_LIST && argc == 1)\n \t\treturn !!check_todo_list();\n+\tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n+\t\treturn !!skip_unnecessary_picks();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 02a7698..a34ebdc 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -703,43 +703,6 @@ do_rest () {\n \tdone\n }\n \n-# skip picking commits whose parents are unchanged\n-skip_unnecessary_picks () {\n-\tfd=3\n-\twhile read -r command rest\n-\tdo\n-\t\t# fd=3 means we skip the command\n-\t\tcase \"$fd,$command\" in\n-\t\t3,pick|3,p)\n-\t\t\t# pick a commit whose parent is current $onto -> skip\n-\t\t\tsha1=${rest%% *}\n-\t\t\tcase \"$(git rev-parse --verify --quiet \"$sha1\"^)\" in\n-\t\t\t\"$onto\"*)\n-\t\t\t\tonto=$sha1\n-\t\t\t\t;;\n-\t\t\t*)\n-\t\t\t\tfd=1\n-\t\t\t\t;;\n-\t\t\tesac\n-\t\t\t;;\n-\t\t3,\"$comment_char\"*|3,)\n-\t\t\t# copy comments\n-\t\t\t;;\n-\t\t*)\n-\t\t\tfd=1\n-\t\t\t;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$command${rest:+ }$rest\" >&$fd\n-\tdone <\"$todo\" >\"$todo.new\" 3>>\"$done\" &&\n-\tmv -f \"$todo\".new \"$todo\" &&\n-\tcase \"$(peek_next_command)\" in\n-\tsquash|s|fixup|f)\n-\t\trecord_in_rewritten \"$onto\"\n-\t\t;;\n-\tesac ||\n-\t\tdie \"$(gettext \"Could not skip unnecessary pick commands\")\"\n-}\n-\n transform_todo_ids () {\n \twhile read -r command rest\n \tdo\n@@ -1165,7 +1128,9 @@ git rebase--helper --check-todo-list || {\n \n expand_todo_ids\n \n-test -d \"$rewritten\" || test -n \"$force_rebase\" || skip_unnecessary_picks\n+test -d \"$rewritten\" || test -n \"$force_rebase\" ||\n+onto=\"$(git rebase--helper --skip-unnecessary-picks)\" ||\n+die \"Could not skip unnecessary pick commands\"\n \n checkout_onto\n if test -z \"$rebase_root\" && test ! -d \"$rewritten\"\ndiff --git a/sequencer.c b/sequencer.c\nindex 0c82925..6cc94c9 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2574,3 +2574,93 @@ int check_todo_list(void)\n \n \treturn res;\n }\n+\n+/* skip picking commits whose parents are unchanged */\n+int skip_unnecessary_picks(void)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct object_id onto_oid, *oid = &onto_oid, *parent_oid;\n+\tint fd, i;\n+\n+\tif (!read_oneliner(&buf, rebase_path_onto(), 0))\n+\t\treturn error(\"Could not read 'onto'\");\n+\tif (get_sha1(buf.buf, onto_oid.hash)) {\n+\t\tstrbuf_release(&buf);\n+\t\treturn error(\"Need a HEAD to fixup\");\n+\t}\n+\tstrbuf_release(&buf);\n+\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0) {\n+\t\treturn error_errno(_(\"Could not open '%s'\"), todo_file);\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"Could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn -1;\n+\t}\n+\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\n+\t\tif (item->command >= TODO_NOOP)\n+\t\t\tcontinue;\n+\t\tif (item->command != TODO_PICK)\n+\t\t\tbreak;\n+\t\tif (parse_commit(item->commit)) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error(_(\"Could not parse commit '%s'\"),\n+\t\t\t\toid_to_hex(&item->commit->object.oid));\n+\t\t}\n+\t\tif (!item->commit->parents)\n+\t\t\tbreak; /* root commit */\n+\t\tif (item->commit->parents->next)\n+\t\t\tbreak; /* merge commit */\n+\t\tparent_oid = &item->commit->parents->item->object.oid;\n+\t\tif (hashcmp(parent_oid->hash, oid->hash))\n+\t\t\tbreak;\n+\t\toid = &item->commit->object.oid;\n+\t}\n+\tif (i > 0) {\n+\t\tint offset = i < todo_list.nr ?\n+\t\t\ttodo_list.items[i].offset_in_buf : todo_list.buf.len;\n+\t\tconst char *done_path = rebase_path_done();\n+\n+\t\tfd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n+\t\tif (write_in_full(fd, todo_list.buf.buf, offset) < 0) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error_errno(_(\"Could not write to '%s'\"),\n+\t\t\t\tdone_path);\n+\t\t}\n+\t\tclose(fd);\n+\n+\t\tfd = open(rebase_path_todo(), O_WRONLY, 0666);\n+\t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n+\t\t\t\ttodo_list.buf.len - offset) < 0) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error_errno(_(\"Could not write to '%s'\"),\n+\t\t\t\trebase_path_todo());\n+\t\t}\n+\t\tif (ftruncate(fd, todo_list.buf.len - offset) < 0) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error_errno(_(\"Could not truncate '%s'\"),\n+\t\t\t\trebase_path_todo());\n+\t\t}\n+\t\tclose(fd);\n+\n+\t\ttodo_list.current = i;\n+\t\tif (is_fixup(peek_command(&todo_list, 0)))\n+\t\t\trecord_in_rewritten(oid, peek_command(&todo_list, 0));\n+\t}\n+\n+\ttodo_list_release(&todo_list);\n+\tprintf(\"%s\\n\", oid_to_hex(oid));\n+\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 8e3daf9..4da215c 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -63,6 +63,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \n int transform_todo_ids(int shorten_sha1s);\n int check_todo_list(void);\n+int skip_unnecessary_picks(void);\n \n extern const char sign_off_header[];\n \n-- \n2.9.3.windows.3\n\n\n"},{"id":"300897","messageId":"3810bd4bddb3b850a78e5d960207157de64a2e56.1472833365.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH 9/9] rebase -i: rearrange fixup/squash lines using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-02T16:23:42Z","receivedAt":"2016-09-02T16:24:06Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This operation has quadratic complexity, which is especially painful\non Windows, where shell scripts are *already* slow (mainly due to the\noverhead of the POSIX emulation layer).\n\nLet's reimplement this with linear complexity (using a hash map to\nmatch the commits' subject lines) for the common case; Sadly, the\nfixup/squash feature's design neglected performance considerations,\nallowing arbitrary prefixes (read: `fixup! hell` will match the\ncommit subject `hello world`), which means that we are stuck with\nquadratic performance in the worst case.\n\nThe reimplemented logic also happens to fix a bug where commented-out\nlines (representing empty patches) were dropped by the previous code.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 6 +-\n git-rebase--interactive.sh | 90 +-------------------\n sequencer.c | 197 +++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n t/t3415-rebase-autosquash.sh | 2 +-\n 5 files changed, 205 insertions(+), 91 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex de3ccd9..e6591f0 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tint keep_empty = 0;\n \tenum {\n \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n-\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n+\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -33,6 +33,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n \t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n \t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n+\t\tOPT_CMDMODE(0, \"rearrange-squash\", &command,\n+\t\t\tN_(\"rearrange fixup/squash lines\"), REARRANGE_SQUASH),\n \t\tOPT_END()\n \t};\n \n@@ -59,5 +61,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!check_todo_list();\n \tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n \t\treturn !!skip_unnecessary_picks();\n+\tif (command == REARRANGE_SQUASH && argc == 1)\n+\t\treturn !!rearrange_squash();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex a34ebdc..68a6e6a 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -734,94 +734,6 @@ collapse_todo_ids() {\n \tgit rebase--helper --shorten-sha1s\n }\n \n-# Rearrange the todo list that has both \"pick sha1 msg\" and\n-# \"pick sha1 fixup!/squash! msg\" appears in it so that the latter\n-# comes immediately after the former, and change \"pick\" to\n-# \"fixup\"/\"squash\".\n-#\n-# Note that if the config has specified a custom instruction format\n-# each log message will be re-retrieved in order to normalize the\n-# autosquash arrangement\n-rearrange_squash () {\n-\tformat=$(git config --get rebase.instructionFormat)\n-\t# extract fixup!/squash! lines and resolve any referenced sha1's\n-\twhile read -r pick sha1 message\n-\tdo\n-\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n-\t\tcase \"$message\" in\n-\t\t\"squash! \"*|\"fixup! \"*)\n-\t\t\taction=\"${message%%!*}\"\n-\t\t\trest=$message\n-\t\t\tprefix=\n-\t\t\t# skip all squash! or fixup! (but save for later)\n-\t\t\twhile :\n-\t\t\tdo\n-\t\t\t\tcase \"$rest\" in\n-\t\t\t\t\"squash! \"*|\"fixup! \"*)\n-\t\t\t\t\tprefix=\"$prefix${rest%%!*},\"\n-\t\t\t\t\trest=\"${rest#*! }\"\n-\t\t\t\t\t;;\n-\t\t\t\t*)\n-\t\t\t\t\tbreak\n-\t\t\t\t\t;;\n-\t\t\t\tesac\n-\t\t\tdone\n-\t\t\tprintf '%s %s %s %s\\n' \"$sha1\" \"$action\" \"$prefix\" \"$rest\"\n-\t\t\t# if it's a single word, try to resolve to a full sha1 and\n-\t\t\t# emit a second copy. This allows us to match on both message\n-\t\t\t# and on sha1 prefix\n-\t\t\tif test \"${rest#* }\" = \"$rest\"; then\n-\t\t\t\tfullsha=\"$(git rev-parse -q --verify \"$rest\" 2>/dev/null)\"\n-\t\t\t\tif test -n \"$fullsha\"; then\n-\t\t\t\t\t# prefix the action to uniquely identify this line as\n-\t\t\t\t\t# intended for full sha1 match\n-\t\t\t\t\techo \"$sha1 +$action $prefix $fullsha\"\n-\t\t\t\tfi\n-\t\t\tfi\n-\t\tesac\n-\tdone >\"$1.sq\" <\"$1\"\n-\ttest -s \"$1.sq\" || return\n-\n-\tused=\n-\twhile read -r pick sha1 message\n-\tdo\n-\t\tcase \" $used\" in\n-\t\t*\" $sha1 \"*) continue ;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$pick $sha1 $message\"\n-\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n-\t\tused=\"$used$sha1 \"\n-\t\twhile read -r squash action msg_prefix msg_content\n-\t\tdo\n-\t\t\tcase \" $used\" in\n-\t\t\t*\" $squash \"*) continue ;;\n-\t\t\tesac\n-\t\t\temit=0\n-\t\t\tcase \"$action\" in\n-\t\t\t+*)\n-\t\t\t\taction=\"${action#+}\"\n-\t\t\t\t# full sha1 prefix test\n-\t\t\t\tcase \"$msg_content\" in \"$sha1\"*) emit=1;; esac ;;\n-\t\t\t*)\n-\t\t\t\t# message prefix test\n-\t\t\t\tcase \"$message\" in \"$msg_content\"*) emit=1;; esac ;;\n-\t\t\tesac\n-\t\t\tif test $emit = 1; then\n-\t\t\t\tif test -n \"${format}\"\n-\t\t\t\tthen\n-\t\t\t\t\tmsg_content=$(git log -n 1 --format=\"${format}\" ${squash})\n-\t\t\t\telse\n-\t\t\t\t\tmsg_content=\"$(echo \"$msg_prefix\" | sed \"s/,/! /g\")$msg_content\"\n-\t\t\t\tfi\n-\t\t\t\tprintf '%s\\n' \"$action $squash $msg_content\"\n-\t\t\t\tused=\"$used$squash \"\n-\t\t\tfi\n-\t\tdone <\"$1.sq\"\n-\tdone >\"$1.rearranged\" <\"$1\"\n-\tcat \"$1.rearranged\" >\"$1\"\n-\trm -f \"$1.sq\" \"$1.rearranged\"\n-}\n-\n # Add commands after a pick or after a squash/fixup serie\n # in the todo list.\n add_exec_commands () {\n@@ -1084,7 +996,7 @@ then\n fi\n \n test -s \"$todo\" || echo noop >> \"$todo\"\n-test -n \"$autosquash\" && rearrange_squash \"$todo\"\n+test -z \"$autosquash\" || git rebase--helper --rearrange-squash || exit\n test -n \"$cmd\" && add_exec_commands \"$todo\"\n \n todocount=$(git stripspace --strip-comments <\"$todo\" | wc -l)\ndiff --git a/sequencer.c b/sequencer.c\nindex 6cc94c9..4b50a4c 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -18,6 +18,7 @@\n #include \"quote.h\"\n #include \"log-tree.h\"\n #include \"wt-status.h\"\n+#include \"hashmap.h\"\n \n #define GIT_REFLOG_ACTION \"GIT_REFLOG_ACTION\"\n \n@@ -2664,3 +2665,199 @@ int skip_unnecessary_picks(void)\n \n \treturn 0;\n }\n+\n+struct subject2item_entry {\n+\tstruct hashmap_entry entry;\n+\tint i;\n+\tchar subject[FLEX_ARRAY];\n+};\n+\n+static int subject2item_cmp(const struct subject2item_entry *a,\n+\tconst struct subject2item_entry *b, const void *key)\n+{\n+\treturn key ? strcmp(a->subject, key) : strcmp(a->subject, b->subject);\n+}\n+\n+/*\n+ * Rearrange the todo list that has both \"pick sha1 msg\" and \"pick sha1\n+ * fixup!/squash! msg\" in it so that the latter is put immediately after the\n+ * former, and change \"pick\" to \"fixup\"/\"squash\".\n+ *\n+ * Note that if the config has specified a custom instruction format, each log\n+ * message will have to be retrieved from the commit (as the oneline in the\n+ * script cannot be trusted) in order to normalize the autosquash arrangement.\n+ */\n+int rearrange_squash(void)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct hashmap subject2item;\n+\tint res = 0, rearranged = 0, *next, *tail, fd, i;\n+\tchar **subjects;\n+\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0)\n+\t\treturn error_errno(_(\"Could not open %s\"), todo_file);\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"Could not read %s.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn -1;\n+\t}\n+\n+\t/*\n+\t * The hashmap maps onelines to the respective todo list index.\n+\t *\n+\t * If any items need to be rearranged, the next[i] value will indicate\n+\t * which item was moved directly after the i'th.\n+\t *\n+\t * In that case, last[i] will indicate the index of the latest item to\n+\t * be moved to appear after the i'th.\n+\t */\n+\thashmap_init(&subject2item, (hashmap_cmp_fn) subject2item_cmp,\n+\t\t todo_list.nr);\n+\tALLOC_ARRAY(next, todo_list.nr);\n+\tALLOC_ARRAY(tail, todo_list.nr);\n+\tALLOC_ARRAY(subjects, todo_list.nr);\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tconst char *commit_buffer, *subject, *p;\n+\t\tint i2 = -1;\n+\t\tstruct subject2item_entry *entry;\n+\n+\t\tnext[i] = tail[i] = -1;\n+\t\tif (item->command >= TODO_EXEC) {\n+\t\t\tsubjects[i] = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (is_fixup(item->command)) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error(_(\"The script was already rearranged.\"));\n+\t\t}\n+\n+\t\titem->commit->util = item;\n+\n+\t\tparse_commit(item->commit);\n+\t\tcommit_buffer = get_commit_buffer(item->commit, NULL);\n+\t\tfind_commit_subject(commit_buffer, &subject);\n+\t\tformat_subject(&buf, subject, \" \");\n+\t\tsubject = subjects[i] = buf.buf;\n+\t\tunuse_commit_buffer(item->commit, commit_buffer);\n+\t\tif ((skip_prefix(subject, \"fixup! \", &p) ||\n+\t\t skip_prefix(subject, \"squash! \", &p))) {\n+\t\t\tstruct commit *commit2;\n+\n+\t\t\tfor (;;) {\n+\t\t\t\twhile (isspace(*p))\n+\t\t\t\t\tp++;\n+\t\t\t\tif (!skip_prefix(p, \"fixup! \", &p) &&\n+\t\t\t\t !skip_prefix(p, \"squash! \", &p))\n+\t\t\t\t\tbreak;\n+\t\t\t}\n+\n+\t\t\tif ((entry = hashmap_get_from_hash(&subject2item,\n+\t\t\t\t\t\t\t strhash(p), p)))\n+\t\t\t\t/* found by title */\n+\t\t\t\ti2 = entry->i;\n+\t\t\telse if (!strchr(p, ' ') &&\n+\t\t\t\t (commit2 =\n+\t\t\t\t lookup_commit_reference_by_name(p)) &&\n+\t\t\t\t commit2->util)\n+\t\t\t\t/* found by commit name */\n+\t\t\t\ti2 = (struct todo_item *)commit2->util\n+\t\t\t\t\t- todo_list.items;\n+\t\t\telse {\n+\t\t\t\t/* copy can be a prefix of the commit subject */\n+\t\t\t\tfor (i2 = 0; i2 < i; i2++)\n+\t\t\t\t\tif (subjects[i2] &&\n+\t\t\t\t\t starts_with(subjects[i2], p))\n+\t\t\t\t\t\tbreak;\n+\t\t\t\tif (i2 == i)\n+\t\t\t\t\ti2 = -1;\n+\t\t\t}\n+\t\t}\n+\t\tif (i2 >= 0) {\n+\t\t\trearranged = 1;\n+\t\t\ttodo_list.items[i].command =\n+\t\t\t\tstarts_with(subject, \"fixup!\") ?\n+\t\t\t\tTODO_FIXUP : TODO_SQUASH;\n+\t\t\tif (next[i2] < 0)\n+\t\t\t\tnext[i2] = i;\n+\t\t\telse\n+\t\t\t\tnext[tail[i2]] = i;\n+\t\t\ttail[i2] = i;\n+\t\t}\n+\t\telse if (!hashmap_get_from_hash(&subject2item,\n+\t\t\t\t\t\tstrhash(subject), subject)) {\n+\t\t\tFLEX_ALLOC_MEM(entry, subject, buf.buf, buf.len);\n+\t\t\tentry->i = i;\n+\t\t\thashmap_entry_init(entry, strhash(entry->subject));\n+\t\t\thashmap_put(&subject2item, entry);\n+\t\t}\n+\t\tstrbuf_detach(&buf, NULL);\n+\t}\n+\n+\tif (rearranged) {\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\t\tchar *format = NULL;\n+\n+\t\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n+\t\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\t\tenum todo_command command = todo_list.items[i].command;\n+\t\t\tint cur = i;\n+\n+\t\t\t/*\n+\t\t\t * Initially, all commands are 'pick's. If it is a\n+\t\t\t * fixup or a squash now, we have rearranged it.\n+\t\t\t */\n+\t\t\tif (is_fixup(command))\n+\t\t\t\tcontinue;\n+\n+\t\t\twhile (cur >= 0) {\n+\t\t\t\tint offset = todo_list.items[cur].offset_in_buf;\n+\t\t\t\tint end_offset = cur + 1 < todo_list.nr ?\n+\t\t\t\t\ttodo_list.items[cur + 1].offset_in_buf :\n+\t\t\t\t\ttodo_list.buf.len;\n+\t\t\t\tchar *bol = todo_list.buf.buf + offset;\n+\t\t\t\tchar *eol = todo_list.buf.buf + end_offset;\n+\n+\t\t\t\t/* replace 'pick', by 'fixup' or 'squash' */\n+\t\t\t\tcommand = todo_list.items[cur].command;\n+\t\t\t\tif (is_fixup(command)) {\n+\t\t\t\t\tstrbuf_addstr(&buf,\n+\t\t\t\t\t\ttodo_command_info[command].str);\n+\t\t\t\t\tbol += strcspn(bol, \" \\t\");\n+\t\t\t\t}\n+\n+\t\t\t\tstrbuf_add(&buf, bol, eol - bol);\n+\n+\t\t\t\tcur = next[cur];\n+\t\t\t}\n+\t\t}\n+\n+\t\tfd = open(todo_file, O_WRONLY);\n+\t\tif (fd < 0)\n+\t\t\tres |= error_errno(_(\"Could not open %s\"), todo_file);\n+\t\telse if (write(fd, buf.buf, buf.len) < 0)\n+\t\t\tres |= error_errno(_(\"Could not read %s.\"), todo_file);\n+\t\telse if (ftruncate(fd, buf.len) < 0)\n+\t\t\tres |= error_errno(_(\"Could not finish %s\"), todo_file);\n+\t\tclose(fd);\n+\t\tstrbuf_release(&buf);\n+\t}\n+\n+\tfree(next);\n+\tfree(tail);\n+\tfor (i = 0; i < todo_list.nr; i++)\n+\t\tfree(subjects[i]);\n+\tfree(subjects);\n+\thashmap_free(&subject2item, 1);\n+\ttodo_list_release(&todo_list);\n+\n+\treturn res;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 4da215c..a3aa3cb 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -64,6 +64,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n int transform_todo_ids(int shorten_sha1s);\n int check_todo_list(void);\n int skip_unnecessary_picks(void);\n+int rearrange_squash(void);\n \n extern const char sign_off_header[];\n \ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 9fd629a..b9e2600 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -278,7 +278,7 @@ set_backup_editor () {\n \ttest_set_editor \"$PWD/backup-editor.sh\"\n }\n \n-test_expect_failure 'autosquash with multiple empty patches' '\n+test_expect_success 'autosquash with multiple empty patches' '\n \ttest_tick &&\n \tgit commit --allow-empty -m \"empty\" &&\n \ttest_tick &&\n-- \n2.9.3.windows.3\n"},{"id":"300913","messageId":"1472849798.4680.63.camel@kaarsemaker.net","threadId":"43983","inReplyTo":"fe53c109c757ede9cd1b0d2944d2b5e16575c49b.1472833365.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Dennis Kaarsemaker","fromEmail":"dennis@kaarsemaker.net","sentAt":"2016-09-02T20:56:38Z","receivedAt":"2016-09-02T20:56:46Z","isPatch":true,"sender":{"key":"dennis@kaarsemaker.net","avatar":"https://avatars.githubusercontent.com/u/200649?v=4"},"body":"On vr, 2016-09-02 at 18:23 +0200, Johannes Schindelin wrote:\n> This is crucial to improve performance on Windows, as the speed is now\n> mostly dominated by the SHA-1 transformation (because it spawns a new\n> rev-parse process for *every* line, and spawning processes is pretty\n> slow from Git for Windows' MSYS2 Bash).\n\nI see these functions only used as part of an shorten-edit-expand\nsequence. Why not do a git rebase-helper --edit-todo instead? Saves\nanother few process spawnings.\n\nSomething for yet another later followup patch?\n\nD.\n"},{"id":"300914","messageId":"1472849978.4680.66.camel@kaarsemaker.net","threadId":"43983","inReplyTo":"6d28acb59d7c217f1f8d6d364e5a52c8bde35085.1472833365.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH 6/9] rebase -i: check for missing commits in the rebase--helper","fromName":"Dennis Kaarsemaker","fromEmail":"dennis@kaarsemaker.net","sentAt":"2016-09-02T20:59:38Z","receivedAt":"2016-09-02T20:59:46Z","isPatch":true,"sender":{"key":"dennis@kaarsemaker.net","avatar":"https://avatars.githubusercontent.com/u/200649?v=4"},"body":"On vr, 2016-09-02 at 18:23 +0200, Johannes Schindelin wrote:\n> In particular on Windows, where shell scripts are even more expensive\n> than on MacOSX or Linux, it makes sense to move a loop that forks\n> Git at least once for every line in the todo list into a builtin.\n\nHeh, this was the one thing that made me hesitate sending the\nsuggestion about rebase-helper --edit-todo, but with this bit already\nmoved, I think rebase-helper --edit-todo makes even more sense to do.\n\nD.\n"},{"id":"300944","messageId":"alpine.DEB.2.20.1609030857480.129229@virtualbox","threadId":"43983","inReplyTo":"1472849798.4680.63.camel@kaarsemaker.net","subject":"Re: [PATCH 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-03T07:01:59Z","receivedAt":"2016-09-03T07:02:40Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Dennis,\n\nOn Fri, 2 Sep 2016, Dennis Kaarsemaker wrote:\n\n> On vr, 2016-09-02 at 18:23 +0200, Johannes Schindelin wrote:\n> > This is crucial to improve performance on Windows, as the speed is now\n> > mostly dominated by the SHA-1 transformation (because it spawns a new\n> > rev-parse process for *every* line, and spawning processes is pretty\n> > slow from Git for Windows' MSYS2 Bash).\n> \n> I see these functions only used as part of an shorten-edit-expand\n> sequence. Why not do a git rebase-helper --edit-todo instead? Saves\n> another few process spawnings.\n\nIt would make sense to consolidate the steps, yes. I just tried to be\ncareful in my incremental approach and am fairly certain about the current\nrevision faithfully replicating the previous behavior.\n\n> Something for yet another later followup patch?\n\nSure. Probably more than one patch, though: I could imagine that a minor\nrefactoring would allow us to read in the todo script once, then apply the\nindividual processing steps in-memory, and then write out the new todo\nscript.\n\nAnd then we can implement an --edit-todo with an optional --initial flag\nthat triggers the check for validity and the rearranging of the\nfixup/squash commands (when the user calls `git rebase --edit-todo`,\nneither of those steps should be run).\n\nMaybe you will want to have a look into that while I am mostly offline?\n\nCiao,\nDscho\n"},{"id":"300956","messageId":"20160903180344.truur5ey6j6ah2wh@x","threadId":"43983","inReplyTo":"3810bd4bddb3b850a78e5d960207157de64a2e56.1472833365.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH 9/9] rebase -i: rearrange fixup/squash lines using the rebase--helper","fromName":"Josh Triplett","fromEmail":"josh@joshtriplett.org","sentAt":"2016-09-03T18:03:45Z","receivedAt":"2016-09-03T18:03:59Z","isPatch":true,"sender":{"key":"josh@joshtriplett.org","avatar":"https://avatars.githubusercontent.com/u/162737?v=4"},"body":"On Fri, Sep 02, 2016 at 06:23:42PM +0200, Johannes Schindelin wrote:\n> Let's reimplement this with linear complexity (using a hash map to\n> match the commits' subject lines) for the common case; Sadly, the\n> fixup/squash feature's design neglected performance considerations,\n> allowing arbitrary prefixes (read: `fixup! hell` will match the\n> commit subject `hello world`), which means that we are stuck with\n> quadratic performance in the worst case.\n\nIf the performance of that case matters enough, we can do better than\nquadratic complexity: maintain a trie of the subjects, allowing prefix\nlookups. (Or hash all the prefixes, which you can do in linear time on\na string: hash next char, save hash, repeat.) However, that would\npessimize the normal case of either a complete subject or a sha1, due to\nthe extra time taken constructing the data structure. Probably not\nworth it, if you assume that most \"fixup!\" subjects come from `git\ncommit --fixup` or similar automated means.\n"},{"id":"300966","messageId":"alpine.DEB.2.20.1609031548540.129229@virtualbox","threadId":"43983","inReplyTo":"20160903180344.truur5ey6j6ah2wh@x","subject":"Re: [PATCH 9/9] rebase -i: rearrange fixup/squash lines using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-09-04T06:47:54Z","receivedAt":"2016-09-04T06:48:31Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Josh,\n\nOn Sat, 3 Sep 2016, Josh Triplett wrote:\n\n> On Fri, Sep 02, 2016 at 06:23:42PM +0200, Johannes Schindelin wrote:\n> > Let's reimplement this with linear complexity (using a hash map to\n> > match the commits' subject lines) for the common case; Sadly, the\n> > fixup/squash feature's design neglected performance considerations,\n> > allowing arbitrary prefixes (read: `fixup! hell` will match the\n> > commit subject `hello world`), which means that we are stuck with\n> > quadratic performance in the worst case.\n> \n> If the performance of that case matters enough, we can do better than\n> quadratic complexity: maintain a trie of the subjects, allowing prefix\n> lookups. (Or hash all the prefixes, which you can do in linear time on\n> a string: hash next char, save hash, repeat.) However, that would\n> pessimize the normal case of either a complete subject or a sha1, due to\n> the extra time taken constructing the data structure. Probably not\n> worth it, if you assume that most \"fixup!\" subjects come from `git\n> commit --fixup` or similar automated means.\n\nRight. My reaction to finding our that subject prefixes were allowed, too,\nwas \"WTF?\". And then: who uses that? And then: that's gonna hurt\nperformance! And then: but I can optimize for the common case!\n\nThe point is: only when people specify a strict prefix will the\nperformance be hurt. Meaning that the performance is linear in the most\ncommon cases.\n\nThat is good enough for me, and probably good enough for the vast majority\nof the users. If it ain't broke, don't fix it.\n\nIn the case that somebody needs strict prefixes to be handled more\nefficiently, which I do not expect, the \"hash all prefixes\" approach may\nwork well, but it would slow down the common case, so I'd suggest doing\nthat only as a fallback (i.e. if a fixup! could not be matched up, fall\nback to hashing the prefixes, re-hashing the commit subjects that were\nalready seen so far). If this needs to be implemented at all, I would also\nsuggest that the person in need of that improvement also needs to take\ncharge of this: I will not spend more time thinking about this.\n\nCiao,\nJohannes\n"},{"id":"317804","messageId":"cover.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1472833365.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 0/9] The final building block for a faster rebase -i","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:51:46Z","receivedAt":"2017-04-25T13:52:01Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This patch series reimplements the expensive pre- and post-processing of\nthe todo script in C.\n\nAnd it concludes the work I did to accelerate rebase -i.\n\nChanges since v1:\n\n- rebased to the current `master` (which advanced quite a bit since v1\n was sent to the list on September 2nd, 2016)\n\n- downcased error messages\n\n- marked error messages for translation\n\n- moved an \"else\" onto the same line as the preceding closing brace\n\n- described the priorities of `fixup! ` in the man page, as\n suggested by Philip Oakley\n\n\nJohannes Schindelin (9):\n rebase -i: generate the script via rebase--helper\n rebase -i: remove useless indentation\n rebase -i: do not invent onelines when expanding/collapsing SHA-1s\n rebase -i: also expand/collapse the SHA-1s via the rebase--helper\n t3404: relax rebase.missingCommitsCheck tests\n rebase -i: check for missing commits in the rebase--helper\n rebase -i: skip unnecessary picks using the rebase--helper\n t3415: test fixup with wrapped oneline\n rebase -i: rearrange fixup/squash lines using the rebase--helper\n\n Documentation/git-rebase.txt | 16 +-\n builtin/rebase--helper.c | 29 ++-\n git-rebase--interactive.sh | 362 ++++-------------------------\n sequencer.c | 515 ++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 8 +\n t/t3404-rebase-interactive.sh | 22 +-\n t/t3415-rebase-autosquash.sh | 16 +-\n 7 files changed, 625 insertions(+), 343 deletions(-)\n\n\nbase-commit: e2cb6ab84c94f147f1259260961513b40c36108a\nBased-On: rebase--helper at https://github.com/dscho/git\nFetch-Base-Via: git fetch https://github.com/dscho/git rebase--helper\nPublished-As: https://github.com/dscho/git/releases/tag/rebase-i-extra-v2\nFetch-It-Via: git fetch https://github.com/dscho/git rebase-i-extra-v2\n\nInterdiff vs v1:\n\n diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt\n index 67d48e68831..da79fbda5b3 100644\n --- a/Documentation/git-rebase.txt\n +++ b/Documentation/git-rebase.txt\n @@ -425,13 +425,15 @@ without an explicit `--interactive`.\n --autosquash::\n --no-autosquash::\n \tWhen the commit log message begins with \"squash! ...\" (or\n -\t\"fixup! ...\"), and there is a commit whose title begins with\n -\tthe same ..., automatically modify the todo list of rebase -i\n -\tso that the commit marked for squashing comes right after the\n -\tcommit to be modified, and change the action of the moved\n -\tcommit from `pick` to `squash` (or `fixup`). Ignores subsequent\n -\t\"fixup! \" or \"squash! \" after the first, in case you referred to an\n -\tearlier fixup/squash with `git commit --fixup/--squash`.\n +\t\"fixup! ...\"), and there is already a commit in the todo list that\n +\tmatches the same `...`, automatically modify the todo list of rebase\n +\t-i so that the commit marked for squashing comes right after the\n +\tcommit to be modified, and change the action of the moved commit\n +\tfrom `pick` to `squash` (or `fixup`). A commit matches the `...` if\n +\tthe commit subject matches, or if the `...` refers to the commit's\n +\thash. As a fall-back, partial matches of the commit subject work,\n +\ttoo. The recommended way to create fixup/squash commits is by using\n +\tthe `--fixup`/`--squash` options of linkgit:git-commit[1].\n +\n This option is only valid when the `--interactive` option is used.\n +\n diff --git a/sequencer.c b/sequencer.c\n index 5502b9006ed..2b07fb9e0ce 100644\n --- a/sequencer.c\n +++ b/sequencer.c\n @@ -2416,10 +2416,10 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \tpp.output_encoding = get_log_output_encoding();\n \n \tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n -\t\treturn error(\"make_script: unhandled options\");\n +\t\treturn error(_(\"make_script: unhandled options\"));\n \n \tif (prepare_revision_walk(&revs) < 0)\n -\t\treturn error(\"make_script: error preparing revisions\");\n +\t\treturn error(_(\"make_script: error preparing revisions\"));\n \n \twhile ((commit = get_revision(&revs))) {\n \t\tstrbuf_reset(&buf);\n @@ -2445,23 +2445,23 @@ int transform_todo_ids(int shorten_sha1s)\n \tstrbuf_reset(&todo_list.buf);\n \tfd = open(todo_file, O_RDONLY);\n \tif (fd < 0)\n -\t\treturn error_errno(_(\"Could not open '%s'\"), todo_file);\n +\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n \tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n \t\tclose(fd);\n -\t\treturn error(_(\"Could not read %s.\"), todo_file);\n +\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n \t}\n \tclose(fd);\n \n \tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n \tif (res) {\n \t\ttodo_list_release(&todo_list);\n -\t\treturn error(_(\"Unusable instruction sheet: %s\"), todo_file);\n +\t\treturn error(_(\"unusable instruction sheet: '%s'\"), todo_file);\n \t}\n \n \tout = fopen(todo_file, \"w\");\n \tif (!out) {\n \t\ttodo_list_release(&todo_list);\n -\t\treturn error(_(\"Unable to open '%s' for writing\"), todo_file);\n +\t\treturn error(_(\"unable to open '%s' for writing\"), todo_file);\n \t}\n \tfor (i = 0; i < todo_list.nr; i++) {\n \t\tstruct todo_item *item = todo_list.items + i;\n @@ -2508,8 +2508,8 @@ static enum check_level get_missing_commit_check_level(void)\n \t\treturn CHECK_WARN;\n \tif (!strcasecmp(\"error\", value))\n \t\treturn CHECK_ERROR;\n -\twarning(_(\"Unrecognized setting $check_level for option\"\n -\t\t\t\"rebase.missingCommitsCheck. Ignoring.\"));\n +\twarning(_(\"unrecognized setting %s for option\"\n +\t\t \"rebase.missingCommitsCheck. Ignoring.\"), value);\n \treturn CHECK_IGNORE;\n }\n \n @@ -2530,12 +2530,12 @@ int check_todo_list(void)\n \tstrbuf_addstr(&todo_file, rebase_path_todo());\n \tfd = open(todo_file.buf, O_RDONLY);\n \tif (fd < 0) {\n -\t\tres = error_errno(_(\"Could not open %s\"), todo_file.buf);\n +\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n \t\tgoto leave_check;\n \t}\n \tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n \t\tclose(fd);\n -\t\tres = error(_(\"Could not read %s.\"), todo_file.buf);\n +\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n \t\tgoto leave_check;\n \t}\n \tclose(fd);\n @@ -2556,12 +2556,12 @@ int check_todo_list(void)\n \tstrbuf_addstr(&todo_file, \".backup\");\n \tfd = open(todo_file.buf, O_RDONLY);\n \tif (fd < 0) {\n -\t\tres = error_errno(_(\"Could not open %s\"), todo_file.buf);\n +\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n \t\tgoto leave_check;\n \t}\n \tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n \t\tclose(fd);\n -\t\tres = error(_(\"Could not read %s.\"), todo_file.buf);\n +\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n \t\tgoto leave_check;\n \t}\n \tclose(fd);\n @@ -2610,9 +2610,10 @@ int check_todo_list(void)\n \n \tif (raise_error)\n \t\tfprintf(stderr,\n -\t\t\t_(\"You can fix this with 'git rebase --edit-todo'.\\n\"\n -\t\t\t\"Or you can abort the rebase with 'git rebase\"\n -\t\t\t\" --abort'.\\n\"));\n +\t\t\t_(\"You can fix this with 'git rebase --edit-todo' \"\n +\t\t\t \"and then run 'git rebase --continue'.\\n\"\n +\t\t\t \"Or you can abort the rebase with 'git rebase\"\n +\t\t\t \" --abort'.\\n\"));\n \n \treturn res;\n }\n @@ -2627,20 +2628,20 @@ int skip_unnecessary_picks(void)\n \tint fd, i;\n \n \tif (!read_oneliner(&buf, rebase_path_onto(), 0))\n -\t\treturn error(\"Could not read 'onto'\");\n +\t\treturn error(_(\"could not read 'onto'\"));\n \tif (get_sha1(buf.buf, onto_oid.hash)) {\n \t\tstrbuf_release(&buf);\n -\t\treturn error(\"Need a HEAD to fixup\");\n +\t\treturn error(_(\"need a HEAD to fixup\"));\n \t}\n \tstrbuf_release(&buf);\n \n \tfd = open(todo_file, O_RDONLY);\n \tif (fd < 0) {\n -\t\treturn error_errno(_(\"Could not open '%s'\"), todo_file);\n +\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n \t}\n \tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n \t\tclose(fd);\n -\t\treturn error(_(\"Could not read '%s'.\"), todo_file);\n +\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n \t}\n \tclose(fd);\n \tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n @@ -2657,7 +2658,7 @@ int skip_unnecessary_picks(void)\n \t\t\tbreak;\n \t\tif (parse_commit(item->commit)) {\n \t\t\ttodo_list_release(&todo_list);\n -\t\t\treturn error(_(\"Could not parse commit '%s'\"),\n +\t\t\treturn error(_(\"could not parse commit '%s'\"),\n \t\t\t\toid_to_hex(&item->commit->object.oid));\n \t\t}\n \t\tif (!item->commit->parents)\n @@ -2677,7 +2678,7 @@ int skip_unnecessary_picks(void)\n \t\tfd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n \t\tif (write_in_full(fd, todo_list.buf.buf, offset) < 0) {\n \t\t\ttodo_list_release(&todo_list);\n -\t\t\treturn error_errno(_(\"Could not write to '%s'\"),\n +\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n \t\t\t\tdone_path);\n \t\t}\n \t\tclose(fd);\n @@ -2686,12 +2687,12 @@ int skip_unnecessary_picks(void)\n \t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n \t\t\t\ttodo_list.buf.len - offset) < 0) {\n \t\t\ttodo_list_release(&todo_list);\n -\t\t\treturn error_errno(_(\"Could not write to '%s'\"),\n +\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n \t\t\t\trebase_path_todo());\n \t\t}\n \t\tif (ftruncate(fd, todo_list.buf.len - offset) < 0) {\n \t\t\ttodo_list_release(&todo_list);\n -\t\t\treturn error_errno(_(\"Could not truncate '%s'\"),\n +\t\t\treturn error_errno(_(\"could not truncate '%s'\"),\n \t\t\t\trebase_path_todo());\n \t\t}\n \t\tclose(fd);\n @@ -2738,10 +2739,10 @@ int rearrange_squash(void)\n \n \tfd = open(todo_file, O_RDONLY);\n \tif (fd < 0)\n -\t\treturn error_errno(_(\"Could not open %s\"), todo_file);\n +\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n \tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n \t\tclose(fd);\n -\t\treturn error(_(\"Could not read %s.\"), todo_file);\n +\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n \t}\n \tclose(fd);\n \tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n @@ -2778,7 +2779,7 @@ int rearrange_squash(void)\n \n \t\tif (is_fixup(item->command)) {\n \t\t\ttodo_list_release(&todo_list);\n -\t\t\treturn error(_(\"The script was already rearranged.\"));\n +\t\t\treturn error(_(\"the script was already rearranged.\"));\n \t\t}\n \n \t\titem->commit->util = item;\n @@ -2832,8 +2833,7 @@ int rearrange_squash(void)\n \t\t\telse\n \t\t\t\tnext[tail[i2]] = i;\n \t\t\ttail[i2] = i;\n -\t\t}\n -\t\telse if (!hashmap_get_from_hash(&subject2item,\n +\t\t} else if (!hashmap_get_from_hash(&subject2item,\n \t\t\t\t\t\tstrhash(subject), subject)) {\n \t\t\tFLEX_ALLOC_MEM(entry, subject, buf.buf, buf.len);\n \t\t\tentry->i = i;\n @@ -2883,11 +2883,12 @@ int rearrange_squash(void)\n \n \t\tfd = open(todo_file, O_WRONLY);\n \t\tif (fd < 0)\n -\t\t\tres |= error_errno(_(\"Could not open %s\"), todo_file);\n +\t\t\tres = error_errno(_(\"could not open '%s'\"), todo_file);\n \t\telse if (write(fd, buf.buf, buf.len) < 0)\n -\t\t\tres |= error_errno(_(\"Could not read %s.\"), todo_file);\n +\t\t\tres = error_errno(_(\"could not read '%s'.\"), todo_file);\n \t\telse if (ftruncate(fd, buf.len) < 0)\n -\t\t\tres |= error_errno(_(\"Could not finish %s\"), todo_file);\n +\t\t\tres = error_errno(_(\"could not finish '%s'\"),\n +\t\t\t\t\t todo_file);\n \t\tclose(fd);\n \t\tstrbuf_release(&buf);\n \t}\n\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n"},{"id":"317805","messageId":"8f08b5d385a2d35526ff9faf92e3c96ea89f134b.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 3/9] rebase -i: do not invent onelines when expanding/collapsing SHA-1s","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:51:56Z","receivedAt":"2017-04-25T13:52:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To avoid problems with short SHA-1s that become non-unique during the\nrebase, we rewrite the todo script with short/long SHA-1s before and\nafter letting the user edit the script. Since SHA-1s are not intuitive\nfor humans, rebase -i also provides the onelines (commit message\nsubjects) in the script, purely for the user's convenience.\n\nIt is very possible to generate a todo script via different means than\nrebase -i and then to let rebase -i run with it; In this case, these\nonelines are not required.\n\nAnd this is where the expand/collapse machinery has a bug: it *expects*\nthat oneline, and failing to find one reuses the previous SHA-1 as\n\"oneline\".\n\nIt was most likely an oversight, and made implementation in the (quite\nlimiting) shell script language less convoluted. However, we are about\nto reimplement performance-critical parts in C (and due to spawning a\ngit.exe process for every single line of the todo script, the\nexpansion/collapsing of the SHA-1s *is* performance-hampering on\nWindows), therefore let's fix this bug to make cross-validation with the\nC version of that functionality possible.\n\nSigned-off-by: Johannes Schindelin \n---\n git-rebase--interactive.sh | 7 ++++++-\n 1 file changed, 6 insertions(+), 1 deletion(-)\n\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex c40b1fd1d2e..214af0372ba 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -760,7 +760,12 @@ transform_todo_ids () {\n \t\t\t;;\n \t\t*)\n \t\t\tsha1=$(git rev-parse --verify --quiet \"$@\" ${rest%%[\t ]*}) &&\n-\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n+\t\t\tif test \"a$rest\" = \"a${rest#*[\t ]}\"\n+\t\t\tthen\n+\t\t\t\trest=$sha1\n+\t\t\telse\n+\t\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n+\t\t\tfi\n \t\t\t;;\n \t\tesac\n \t\tprintf '%s\\n' \"$command${rest:+ }$rest\"\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317806","messageId":"361682a84f421ff16a8c6b7120f132cd6b61f1fa.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:51:49Z","receivedAt":"2017-04-25T13:52:08Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The first step of an interactive rebase is to generate the so-called \"todo\nscript\", to be stored in the state directory as \"git-rebase-todo\" and to\nbe edited by the user.\n\nOriginally, we adjusted the output of `git log ` using a simple\nsed script. Over the course of the years, the code became more\ncomplicated. We now use shell scripting to edit the output of `git log`\nconditionally, depending whether to keep \"empty\" commits (i.e. commits\nthat do not change any files).\n\nOn platforms where shell scripting is not native, this can be a serious\ndrag. And it opens the door for incompatibilities between platforms when\nit comes to shell scripting or to Unix-y commands.\n\nLet's just re-implement the todo script generation in plain C, using the\nrevision machinery directly.\n\nThis is substantially faster, improving the speed relative to the\nshell script version of the interactive rebase from 2x to 3x on Windows.\n\nNote that the rearrange_squash() function in git-rebase--interactive\nrelied on the fact that we set the \"format\" variable to the config setting\nrebase.instructionFormat. Relying on a side effect like this is no good,\nhence we explicitly perform that assignment (possibly again) in\nrearrange_squash().\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 8 +++++++-\n git-rebase--interactive.sh | 44 +++++++++++++++++++++++---------------------\n sequencer.c | 44 ++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 3 +++\n 4 files changed, 77 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex ca1ebb2fa18..821058d452d 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -11,15 +11,19 @@ static const char * const builtin_rebase_helper_usage[] = {\n int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n {\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n+\tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n+\t\tOPT_BOOL(0, \"keep-empty\", &keep_empty, N_(\"keep empty commits\")),\n \t\tOPT_CMDMODE(0, \"continue\", &command, N_(\"continue rebase\"),\n \t\t\t\tCONTINUE),\n \t\tOPT_CMDMODE(0, \"abort\", &command, N_(\"abort rebase\"),\n \t\t\t\tABORT),\n+\t\tOPT_CMDMODE(0, \"make-script\", &command,\n+\t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n \t\tOPT_END()\n \t};\n \n@@ -36,5 +40,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!sequencer_continue(&opts);\n \tif (command == ABORT && argc == 1)\n \t\treturn !!sequencer_remove_state(&opts);\n+\tif (command == MAKE_SCRIPT && argc > 1)\n+\t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 2c9c0165b5a..609e150d38f 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -785,6 +785,7 @@ collapse_todo_ids() {\n # each log message will be re-retrieved in order to normalize the\n # autosquash arrangement\n rearrange_squash () {\n+\tformat=$(git config --get rebase.instructionFormat)\n \t# extract fixup!/squash! lines and resolve any referenced sha1's\n \twhile read -r pick sha1 message\n \tdo\n@@ -1210,26 +1211,27 @@ else\n \trevisions=$onto...$orig_head\n \tshortrevisions=$shorthead\n fi\n-format=$(git config --get rebase.instructionFormat)\n-# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n-git rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n-\t--reverse --left-right --topo-order \\\n-\t$revisions ${restrict_revision+^$restrict_revision} | \\\n-\tsed -n \"s/^>//p\" |\n-while read -r sha1 rest\n-do\n-\n-\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n-\tthen\n-\t\tcomment_out=\"$comment_char \"\n-\telse\n-\t\tcomment_out=\n-\tfi\n+if test t != \"$preserve_merges\"\n+then\n+\tgit rebase--helper --make-script ${keep_empty:+--keep-empty} \\\n+\t\t$revisions ${restrict_revision+^$restrict_revision} >\"$todo\"\n+else\n+\tformat=$(git config --get rebase.instructionFormat)\n+\t# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n+\tgit rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n+\t\t--reverse --left-right --topo-order \\\n+\t\t$revisions ${restrict_revision+^$restrict_revision} | \\\n+\t\tsed -n \"s/^>//p\" |\n+\twhile read -r sha1 rest\n+\tdo\n+\n+\t\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n+\t\tthen\n+\t\t\tcomment_out=\"$comment_char \"\n+\t\telse\n+\t\t\tcomment_out=\n+\t\tfi\n \n-\tif test t != \"$preserve_merges\"\n-\tthen\n-\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n-\telse\n \t\tif test -z \"$rebase_root\"\n \t\tthen\n \t\t\tpreserve=t\n@@ -1248,8 +1250,8 @@ do\n \t\t\ttouch \"$rewritten\"/$sha1\n \t\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n \t\tfi\n-\tfi\n-done\n+\tdone\n+fi\n \n # Watch for commits that been dropped by --cherry-pick\n if test t = \"$preserve_merges\"\ndiff --git a/sequencer.c b/sequencer.c\nindex 77afecaebf0..d2d5bcd9b43 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2388,3 +2388,47 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n \n \tstrbuf_release(&sob);\n }\n+\n+int sequencer_make_script(int keep_empty, FILE *out,\n+\t\tint argc, const char **argv)\n+{\n+\tchar *format = \"%s\";\n+\tstruct pretty_print_context pp = {0};\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct rev_info revs;\n+\tstruct commit *commit;\n+\n+\tinit_revisions(&revs, NULL);\n+\trevs.verbose_header = 1;\n+\trevs.max_parents = 1;\n+\trevs.cherry_pick = 1;\n+\trevs.limited = 1;\n+\trevs.reverse = 1;\n+\trevs.right_only = 1;\n+\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n+\trevs.topo_order = 1;\n+\n+\trevs.pretty_given = 1;\n+\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n+\tget_commit_format(format, &revs);\n+\tpp.fmt = revs.commit_format;\n+\tpp.output_encoding = get_log_output_encoding();\n+\n+\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n+\t\treturn error(_(\"make_script: unhandled options\"));\n+\n+\tif (prepare_revision_walk(&revs) < 0)\n+\t\treturn error(_(\"make_script: error preparing revisions\"));\n+\n+\twhile ((commit = get_revision(&revs))) {\n+\t\tstrbuf_reset(&buf);\n+\t\tif (!keep_empty && is_original_commit_empty(commit))\n+\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n+\t\tstrbuf_addf(&buf, \"pick %s \", oid_to_hex(&commit->object.oid));\n+\t\tpretty_print_commit(&pp, commit, &buf);\n+\t\tstrbuf_addch(&buf, '\\n');\n+\t\tfputs(buf.buf, out);\n+\t}\n+\tstrbuf_release(&buf);\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex f885b68395f..83f2943b7a9 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -45,6 +45,9 @@ int sequencer_continue(struct replay_opts *opts);\n int sequencer_rollback(struct replay_opts *opts);\n int sequencer_remove_state(struct replay_opts *opts);\n \n+int sequencer_make_script(int keep_empty, FILE *out,\n+\t\tint argc, const char **argv);\n+\n extern const char sign_off_header[];\n \n void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317807","messageId":"768857b7dfd19c3568f5e0c0ebed36157a0658e7.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 2/9] rebase -i: remove useless indentation","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:51:52Z","receivedAt":"2017-04-25T13:52:26Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The commands used to be indented, and it is nice to look at, but when we\ntransform the SHA-1s, the indentation is removed. So let's do away with it.\n\nFor the moment, at least: when we will use the upcoming rebase--helper\nto transform the SHA-1s, we *will* keep the indentation and can\nreintroduce it. Yet, to be able to validate the rebase--helper against\nthe output of the current shell script version, we need to remove the\nextra indentation.\n\nSigned-off-by: Johannes Schindelin \n---\n git-rebase--interactive.sh | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 609e150d38f..c40b1fd1d2e 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -155,13 +155,13 @@ reschedule_last_action () {\n append_todo_help () {\n \tgettext \"\n Commands:\n- p, pick = use commit\n- r, reword = use commit, but edit the commit message\n- e, edit = use commit, but stop for amending\n- s, squash = use commit, but meld into previous commit\n- f, fixup = like \\\"squash\\\", but discard this commit's log message\n- x, exec = run command (the rest of the line) using shell\n- d, drop = remove commit\n+p, pick = use commit\n+r, reword = use commit, but edit the commit message\n+e, edit = use commit, but stop for amending\n+s, squash = use commit, but meld into previous commit\n+f, fixup = like \\\"squash\\\", but discard this commit's log message\n+x, exec = run command (the rest of the line) using shell\n+d, drop = remove commit\n \n These lines can be re-ordered; they are executed from top to bottom.\n \" | git stripspace --comment-lines >>\"$todo\"\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317808","messageId":"b63c3c4bfa5d9686647dc107f143ce58fa5074ec.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 5/9] t3404: relax rebase.missingCommitsCheck tests","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:52:02Z","receivedAt":"2017-04-25T13:52:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"These tests were a bit anal about the *exact* warning/error message\nprinted by git rebase. But those messages are intended for the *end\nuser*, therefore it does not make sense to test so rigidly for the\n*exact* wording.\n\nIn the following, we will reimplement the missing commits check in\nthe sequencer, with slightly different words.\n\nSo let's just test for the parts in the warning/error message that\nwe *really* care about, nothing more, nothing less.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3404-rebase-interactive.sh | 22 ++++------------------\n 1 file changed, 4 insertions(+), 18 deletions(-)\n\ndiff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh\nindex 33d392ba112..61113be08a4 100755\n--- a/t/t3404-rebase-interactive.sh\n+++ b/t/t3404-rebase-interactive.sh\n@@ -1242,20 +1242,13 @@ test_expect_success 'rebase -i respects rebase.missingCommitsCheck = error' '\n \ttest B = $(git cat-file commit HEAD^ | sed -ne \\$p)\n '\n \n-cat >expect <actual &&\n-\ttest_i18ncmp expect actual &&\n+\ttest_i18ngrep \"badcmd $(git rev-list --oneline -1 master~1)\" actual &&\n+\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n \tFAKE_LINES=\"1 2 3 drop 4 5\" git rebase --edit-todo &&\n \tgit rebase --continue &&\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p) &&\n@@ -1277,20 +1270,13 @@ test_expect_success 'tabs and spaces are accepted in the todolist' '\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n '\n \n-cat >expect <actual &&\n-\ttest_i18ncmp expect actual &&\n+\ttest_i18ngrep \"edit XXXXXXX False commit\" actual &&\n+\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n \tFAKE_LINES=\"1 2 4 5 6\" git rebase --edit-todo &&\n \tgit rebase --continue &&\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317809","messageId":"449c8cf3819b92dfcc86a614815972c25f4974a2.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:51:59Z","receivedAt":"2017-04-25T13:52:33Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This is crucial to improve performance on Windows, as the speed is now\nmostly dominated by the SHA-1 transformation (because it spawns a new\nrev-parse process for *every* line, and spawning processes is pretty\nslow from Git for Windows' MSYS2 Bash).\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 10 +++++++-\n git-rebase--interactive.sh | 4 ++--\n sequencer.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 2 ++\n 4 files changed, 72 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex 821058d452d..9444c8d6c60 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n \tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -24,6 +24,10 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\t\tABORT),\n \t\tOPT_CMDMODE(0, \"make-script\", &command,\n \t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n+\t\tOPT_CMDMODE(0, \"shorten-sha1s\", &command,\n+\t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n+\t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n+\t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n \t\tOPT_END()\n \t};\n \n@@ -42,5 +46,9 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!sequencer_remove_state(&opts);\n \tif (command == MAKE_SCRIPT && argc > 1)\n \t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n+\tif (command == SHORTEN_SHA1S && argc == 1)\n+\t\treturn !!transform_todo_ids(1);\n+\tif (command == EXPAND_SHA1S && argc == 1)\n+\t\treturn !!transform_todo_ids(0);\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 214af0372ba..52a19e0bdb3 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -774,11 +774,11 @@ transform_todo_ids () {\n }\n \n expand_todo_ids() {\n-\ttransform_todo_ids\n+\tgit rebase--helper --expand-sha1s\n }\n \n collapse_todo_ids() {\n-\ttransform_todo_ids --short\n+\tgit rebase--helper --shorten-sha1s\n }\n \n # Rearrange the todo list that has both \"pick sha1 msg\" and\ndiff --git a/sequencer.c b/sequencer.c\nindex d2d5bcd9b43..d6ae6407546 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2432,3 +2432,62 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \tstrbuf_release(&buf);\n \treturn 0;\n }\n+\n+\n+int transform_todo_ids(int shorten_sha1s)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tint fd, res, i;\n+\tFILE *out;\n+\n+\tstrbuf_reset(&todo_list.buf);\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0)\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\n+\tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\tif (res) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn error(_(\"unusable instruction sheet: '%s'\"), todo_file);\n+\t}\n+\n+\tout = fopen(todo_file, \"w\");\n+\tif (!out) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn error(_(\"unable to open '%s' for writing\"), todo_file);\n+\t}\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tint bol = item->offset_in_buf;\n+\t\tconst char *p = todo_list.buf.buf + bol;\n+\t\tint eol = i + 1 < todo_list.nr ?\n+\t\t\ttodo_list.items[i + 1].offset_in_buf :\n+\t\t\ttodo_list.buf.len;\n+\n+\t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n+\t\t\tfwrite(p, eol - bol, 1, out);\n+\t\telse {\n+\t\t\tint eoc = strcspn(p, \" \\t\");\n+\t\t\tconst char *sha1 = shorten_sha1s ?\n+\t\t\t\tshort_commit_name(item->commit) :\n+\t\t\t\toid_to_hex(&item->commit->object.oid);\n+\n+\t\t\tif (!eoc) {\n+\t\t\t\tp += strspn(p, \" \\t\");\n+\t\t\t\teoc = strcspn(p, \" \\t\");\n+\t\t\t}\n+\n+\t\t\tfprintf(out, \"%.*s %s %.*s\\n\",\n+\t\t\t\teoc, p, sha1, item->arg_len, item->arg);\n+\t\t}\n+\t}\n+\tfclose(out);\n+\ttodo_list_release(&todo_list);\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 83f2943b7a9..47a81034e76 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -48,6 +48,8 @@ int sequencer_remove_state(struct replay_opts *opts);\n int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv);\n \n+int transform_todo_ids(int shorten_sha1s);\n+\n extern const char sign_off_header[];\n \n void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317810","messageId":"669be37bd417294913596a1d47e046b03eb23f6d.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 8/9] t3415: test fixup with wrapped oneline","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:52:14Z","receivedAt":"2017-04-25T13:52:34Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The `git commit --fixup` command unwraps wrapped onelines when\nconstructing the commit message, without wrapping the result.\n\nWe need to make sure that `git rebase --autosquash` keeps handling such\ncases correctly, in particular since we are about to move the autosquash\nhandling into the rebase--helper.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3415-rebase-autosquash.sh | 14 ++++++++++++++\n 1 file changed, 14 insertions(+)\n\ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 48346f1cc0c..9fd629a6e21 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -304,4 +304,18 @@ test_expect_success 'extra spaces after fixup!' '\n \ttest $base = $parent\n '\n \n+test_expect_success 'wrapped original subject' '\n+\tif test -d .git/rebase-merge; then git rebase --abort; fi &&\n+\tbase=$(git rev-parse HEAD) &&\n+\techo \"wrapped subject\" >wrapped &&\n+\tgit add wrapped &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m \"$(printf \"To\\nfixup\")\" &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m \"fixup! To fixup\" &&\n+\tgit rebase -i --autosquash --keep-empty HEAD~2 &&\n+\tparent=$(git rev-parse HEAD^) &&\n+\ttest $base = $parent\n+'\n+\n test_done\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317811","messageId":"01e494b3c7a713ba42c6a197e91e0d62ec19709f.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 6/9] rebase -i: check for missing commits in the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:52:06Z","receivedAt":"2017-04-25T13:52:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"In particular on Windows, where shell scripts are even more expensive\nthan on MacOSX or Linux, it makes sense to move a loop that forks\nGit at least once for every line in the todo list into a builtin.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 7 +-\n git-rebase--interactive.sh | 164 ++-------------------------------------------\n sequencer.c | 125 ++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n 4 files changed, 137 insertions(+), 160 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex 9444c8d6c60..e706eac710d 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -13,7 +13,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n \tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n+\t\tCHECK_TODO_LIST\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -28,6 +29,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n \t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n \t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n+\t\tOPT_CMDMODE(0, \"check-todo-list\", &command,\n+\t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n \t\tOPT_END()\n \t};\n \n@@ -50,5 +53,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!transform_todo_ids(1);\n \tif (command == EXPAND_SHA1S && argc == 1)\n \t\treturn !!transform_todo_ids(0);\n+\tif (command == CHECK_TODO_LIST && argc == 1)\n+\t\treturn !!check_todo_list();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 52a19e0bdb3..1649506e1e4 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -890,96 +890,6 @@ add_exec_commands () {\n \tmv \"$1.new\" \"$1\"\n }\n \n-# Check if the SHA-1 passed as an argument is a\n-# correct one, if not then print $2 in \"$todo\".badsha\n-# $1: the SHA-1 to test\n-# $2: the line number of the input\n-# $3: the input filename\n-check_commit_sha () {\n-\tbadsha=0\n-\tif test -z \"$1\"\n-\tthen\n-\t\tbadsha=1\n-\telse\n-\t\tsha1_verif=\"$(git rev-parse --verify --quiet $1^{commit})\"\n-\t\tif test -z \"$sha1_verif\"\n-\t\tthen\n-\t\t\tbadsha=1\n-\t\tfi\n-\tfi\n-\n-\tif test $badsha -ne 0\n-\tthen\n-\t\tline=\"$(sed -n -e \"${2}p\" \"$3\")\"\n-\t\twarn \"$(eval_gettext \"\\\n-Warning: the SHA-1 is missing or isn't a commit in the following line:\n- - \\$line\")\"\n-\t\twarn\n-\tfi\n-\n-\treturn $badsha\n-}\n-\n-# prints the bad commits and bad commands\n-# from the todolist in stdin\n-check_bad_cmd_and_sha () {\n-\tretval=0\n-\tlineno=0\n-\twhile read -r command rest\n-\tdo\n-\t\tlineno=$(( $lineno + 1 ))\n-\t\tcase $command in\n-\t\t\"$comment_char\"*|''|noop|x|exec)\n-\t\t\t# Doesn't expect a SHA-1\n-\t\t\t;;\n-\t\t\"$cr\")\n-\t\t\t# Work around CR left by \"read\" (e.g. with Git for\n-\t\t\t# Windows' Bash).\n-\t\t\t;;\n-\t\tpick|p|drop|d|reword|r|edit|e|squash|s|fixup|f)\n-\t\t\tif ! check_commit_sha \"${rest%%[ \t]*}\" \"$lineno\" \"$1\"\n-\t\t\tthen\n-\t\t\t\tretval=1\n-\t\t\tfi\n-\t\t\t;;\n-\t\t*)\n-\t\t\tline=\"$(sed -n -e \"${lineno}p\" \"$1\")\"\n-\t\t\twarn \"$(eval_gettext \"\\\n-Warning: the command isn't recognized in the following line:\n- - \\$line\")\"\n-\t\t\twarn\n-\t\t\tretval=1\n-\t\t\t;;\n-\t\tesac\n-\tdone <\"$1\"\n-\treturn $retval\n-}\n-\n-# Print the list of the SHA-1 of the commits\n-# from stdin to stdout\n-todo_list_to_sha_list () {\n-\tgit stripspace --strip-comments |\n-\twhile read -r command sha1 rest\n-\tdo\n-\t\tcase $command in\n-\t\t\"$comment_char\"*|''|noop|x|\"exec\")\n-\t\t\t;;\n-\t\t*)\n-\t\t\tlong_sha=$(git rev-list --no-walk \"$sha1\" 2>/dev/null)\n-\t\t\tprintf \"%s\\n\" \"$long_sha\"\n-\t\t\t;;\n-\t\tesac\n-\tdone\n-}\n-\n-# Use warn for each line in stdin\n-warn_lines () {\n-\twhile read -r line\n-\tdo\n-\t\twarn \" - $line\"\n-\tdone\n-}\n-\n # Switch to the branch in $into and notify it in the reflog\n checkout_onto () {\n \tGIT_REFLOG_ACTION=\"$GIT_REFLOG_ACTION: checkout $onto_name\"\n@@ -994,74 +904,6 @@ get_missing_commit_check_level () {\n \tprintf '%s' \"$check_level\" | tr 'A-Z' 'a-z'\n }\n \n-# Check if the user dropped some commits by mistake\n-# Behaviour determined by rebase.missingCommitsCheck.\n-# Check if there is an unrecognized command or a\n-# bad SHA-1 in a command.\n-check_todo_list () {\n-\traise_error=f\n-\n-\tcheck_level=$(get_missing_commit_check_level)\n-\n-\tcase \"$check_level\" in\n-\twarn|error)\n-\t\t# Get the SHA-1 of the commits\n-\t\ttodo_list_to_sha_list <\"$todo\".backup >\"$todo\".oldsha1\n-\t\ttodo_list_to_sha_list <\"$todo\" >\"$todo\".newsha1\n-\n-\t\t# Sort the SHA-1 and compare them\n-\t\tsort -u \"$todo\".oldsha1 >\"$todo\".oldsha1+\n-\t\tmv \"$todo\".oldsha1+ \"$todo\".oldsha1\n-\t\tsort -u \"$todo\".newsha1 >\"$todo\".newsha1+\n-\t\tmv \"$todo\".newsha1+ \"$todo\".newsha1\n-\t\tcomm -2 -3 \"$todo\".oldsha1 \"$todo\".newsha1 >\"$todo\".miss\n-\n-\t\t# Warn about missing commits\n-\t\tif test -s \"$todo\".miss\n-\t\tthen\n-\t\t\ttest \"$check_level\" = error && raise_error=t\n-\n-\t\t\twarn \"$(gettext \"\\\n-Warning: some commits may have been dropped accidentally.\n-Dropped commits (newer to older):\")\"\n-\n-\t\t\t# Make the list user-friendly and display\n-\t\t\topt=\"--no-walk=sorted --format=oneline --abbrev-commit --stdin\"\n-\t\t\tgit rev-list $opt <\"$todo\".miss | warn_lines\n-\n-\t\t\twarn \"$(gettext \"\\\n-To avoid this message, use \\\"drop\\\" to explicitly remove a commit.\n-\n-Use 'git config rebase.missingCommitsCheck' to change the level of warnings.\n-The possible behaviours are: ignore, warn, error.\")\"\n-\t\t\twarn\n-\t\tfi\n-\t\t;;\n-\tignore)\n-\t\t;;\n-\t*)\n-\t\twarn \"$(eval_gettext \"Unrecognized setting \\$check_level for option rebase.missingCommitsCheck. Ignoring.\")\"\n-\t\t;;\n-\tesac\n-\n-\tif ! check_bad_cmd_and_sha \"$todo\"\n-\tthen\n-\t\traise_error=t\n-\tfi\n-\n-\tif test $raise_error = t\n-\tthen\n-\t\t# Checkout before the first commit of the\n-\t\t# rebase: this way git rebase --continue\n-\t\t# will work correctly as it expects HEAD to be\n-\t\t# placed before the commit of the next action\n-\t\tcheckout_onto\n-\n-\t\twarn \"$(gettext \"You can fix this with 'git rebase --edit-todo' and then run 'git rebase --continue'.\")\"\n-\t\tdie \"$(gettext \"Or you can abort the rebase with 'git rebase --abort'.\")\"\n-\tfi\n-}\n-\n # The whole contents of this file is run by dot-sourcing it from\n # inside a shell function. It used to be that \"return\"s we see\n # below were not inside any function, and expected to return\n@@ -1322,7 +1164,11 @@ git_sequence_editor \"$todo\" ||\n has_action \"$todo\" ||\n \treturn 2\n \n-check_todo_list\n+git rebase--helper --check-todo-list || {\n+\tret=$?\n+\tcheckout_onto\n+\texit $ret\n+}\n \n expand_todo_ids\n \ndiff --git a/sequencer.c b/sequencer.c\nindex d6ae6407546..3a935fa4cbc 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2491,3 +2491,128 @@ int transform_todo_ids(int shorten_sha1s)\n \ttodo_list_release(&todo_list);\n \treturn 0;\n }\n+\n+enum check_level {\n+\tCHECK_IGNORE = 0, CHECK_WARN, CHECK_ERROR\n+};\n+\n+static enum check_level get_missing_commit_check_level(void)\n+{\n+\tconst char *value;\n+\n+\tif (git_config_get_value(\"rebase.missingcommitscheck\", &value) ||\n+\t\t\t!strcasecmp(\"ignore\", value))\n+\t\treturn CHECK_IGNORE;\n+\tif (!strcasecmp(\"warn\", value))\n+\t\treturn CHECK_WARN;\n+\tif (!strcasecmp(\"error\", value))\n+\t\treturn CHECK_ERROR;\n+\twarning(_(\"unrecognized setting %s for option\"\n+\t\t \"rebase.missingCommitsCheck. Ignoring.\"), value);\n+\treturn CHECK_IGNORE;\n+}\n+\n+/*\n+ * Check if the user dropped some commits by mistake\n+ * Behaviour determined by rebase.missingCommitsCheck.\n+ * Check if there is an unrecognized command or a\n+ * bad SHA-1 in a command.\n+ */\n+int check_todo_list(void)\n+{\n+\tenum check_level check_level = get_missing_commit_check_level();\n+\tstruct strbuf todo_file = STRBUF_INIT;\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct commit_list *missing = NULL;\n+\tint raise_error = 0, res = 0, fd, i;\n+\n+\tstrbuf_addstr(&todo_file, rebase_path_todo());\n+\tfd = open(todo_file.buf, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tclose(fd);\n+\traise_error = res =\n+\t\tparse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\n+\tif (check_level == CHECK_IGNORE)\n+\t\tgoto leave_check;\n+\n+\t/* Get the SHA-1 of the commits */\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct commit *commit = todo_list.items[i].commit;\n+\t\tif (commit)\n+\t\t\tcommit->util = todo_list.items + i;\n+\t}\n+\n+\ttodo_list_release(&todo_list);\n+\tstrbuf_addstr(&todo_file, \".backup\");\n+\tfd = open(todo_file.buf, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tclose(fd);\n+\tstrbuf_release(&todo_file);\n+\tres = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\n+\t/* Find commits that are missing after editing */\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct commit *commit = todo_list.items[i].commit;\n+\t\tif (commit && !commit->util) {\n+\t\t\tcommit_list_insert(commit, &missing);\n+\t\t\tcommit->util = todo_list.items + i;\n+\t\t}\n+\t}\n+\n+\t/* Warn about missing commits */\n+\tif (!missing)\n+\t\tgoto leave_check;\n+\n+\tif (check_level == CHECK_ERROR)\n+\t\traise_error = res = 1;\n+\n+\tfprintf(stderr,\n+\t\t_(\"Warning: some commits may have been dropped accidentally.\\n\"\n+\t\t\"Dropped commits (newer to older):\\n\"));\n+\n+\t/* Make the list user-friendly and display */\n+\twhile (missing) {\n+\t\tstruct commit *commit = pop_commit(&missing);\n+\t\tstruct todo_item *item = commit->util;\n+\n+\t\tfprintf(stderr, \" - %s %.*s\\n\", short_commit_name(commit),\n+\t\t\titem->arg_len, item->arg);\n+\t}\n+\tfree_commit_list(missing);\n+\n+\tfprintf(stderr, _(\"To avoid this message, use \\\"drop\\\" to \"\n+\t\t\"explicitly remove a commit.\\n\\n\"\n+\t\t\"Use 'git config rebase.missingCommitsCheck' to change \"\n+\t\t\"the level of warnings.\\n\"\n+\t\t\"The possible behaviours are: ignore, warn, error.\\n\\n\"));\n+\n+leave_check:\n+\tstrbuf_release(&todo_file);\n+\ttodo_list_release(&todo_list);\n+\n+\tif (raise_error)\n+\t\tfprintf(stderr,\n+\t\t\t_(\"You can fix this with 'git rebase --edit-todo' \"\n+\t\t\t \"and then run 'git rebase --continue'.\\n\"\n+\t\t\t \"Or you can abort the rebase with 'git rebase\"\n+\t\t\t \" --abort'.\\n\"));\n+\n+\treturn res;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 47a81034e76..4978a61b83b 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -49,6 +49,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv);\n \n int transform_todo_ids(int shorten_sha1s);\n+int check_todo_list(void);\n \n extern const char sign_off_header[];\n \n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317812","messageId":"0bc80dfece5c0685473e4d3f75afc13c4762f72a.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 9/9] rebase -i: rearrange fixup/squash lines using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:52:19Z","receivedAt":"2017-04-25T13:52:46Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This operation has quadratic complexity, which is especially painful\non Windows, where shell scripts are *already* slow (mainly due to the\noverhead of the POSIX emulation layer).\n\nLet's reimplement this with linear complexity (using a hash map to\nmatch the commits' subject lines) for the common case; Sadly, the\nfixup/squash feature's design neglected performance considerations,\nallowing arbitrary prefixes (read: `fixup! hell` will match the\ncommit subject `hello world`), which means that we are stuck with\nquadratic performance in the worst case.\n\nThe reimplemented logic also happens to fix a bug where commented-out\nlines (representing empty patches) were dropped by the previous code.\n\nWhile at it, clarify how the fixup/squash feature works in `git rebase\n-i`'s man page.\n\nSigned-off-by: Johannes Schindelin \n---\n Documentation/git-rebase.txt | 16 ++--\n builtin/rebase--helper.c | 6 +-\n git-rebase--interactive.sh | 90 +-------------------\n sequencer.c | 197 +++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n t/t3415-rebase-autosquash.sh | 2 +-\n 6 files changed, 214 insertions(+), 98 deletions(-)\n\ndiff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt\nindex 67d48e68831..da79fbda5b3 100644\n--- a/Documentation/git-rebase.txt\n+++ b/Documentation/git-rebase.txt\n@@ -425,13 +425,15 @@ without an explicit `--interactive`.\n --autosquash::\n --no-autosquash::\n \tWhen the commit log message begins with \"squash! ...\" (or\n-\t\"fixup! ...\"), and there is a commit whose title begins with\n-\tthe same ..., automatically modify the todo list of rebase -i\n-\tso that the commit marked for squashing comes right after the\n-\tcommit to be modified, and change the action of the moved\n-\tcommit from `pick` to `squash` (or `fixup`). Ignores subsequent\n-\t\"fixup! \" or \"squash! \" after the first, in case you referred to an\n-\tearlier fixup/squash with `git commit --fixup/--squash`.\n+\t\"fixup! ...\"), and there is already a commit in the todo list that\n+\tmatches the same `...`, automatically modify the todo list of rebase\n+\t-i so that the commit marked for squashing comes right after the\n+\tcommit to be modified, and change the action of the moved commit\n+\tfrom `pick` to `squash` (or `fixup`). A commit matches the `...` if\n+\tthe commit subject matches, or if the `...` refers to the commit's\n+\thash. As a fall-back, partial matches of the commit subject work,\n+\ttoo. The recommended way to create fixup/squash commits is by using\n+\tthe `--fixup`/`--squash` options of linkgit:git-commit[1].\n +\n This option is only valid when the `--interactive` option is used.\n +\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex de3ccd9bfbc..e6591f01112 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tint keep_empty = 0;\n \tenum {\n \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n-\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n+\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -33,6 +33,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n \t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n \t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n+\t\tOPT_CMDMODE(0, \"rearrange-squash\", &command,\n+\t\t\tN_(\"rearrange fixup/squash lines\"), REARRANGE_SQUASH),\n \t\tOPT_END()\n \t};\n \n@@ -59,5 +61,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!check_todo_list();\n \tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n \t\treturn !!skip_unnecessary_picks();\n+\tif (command == REARRANGE_SQUASH && argc == 1)\n+\t\treturn !!rearrange_squash();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 931bc09e0cf..d39fe4f5fb7 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -744,94 +744,6 @@ collapse_todo_ids() {\n \tgit rebase--helper --shorten-sha1s\n }\n \n-# Rearrange the todo list that has both \"pick sha1 msg\" and\n-# \"pick sha1 fixup!/squash! msg\" appears in it so that the latter\n-# comes immediately after the former, and change \"pick\" to\n-# \"fixup\"/\"squash\".\n-#\n-# Note that if the config has specified a custom instruction format\n-# each log message will be re-retrieved in order to normalize the\n-# autosquash arrangement\n-rearrange_squash () {\n-\tformat=$(git config --get rebase.instructionFormat)\n-\t# extract fixup!/squash! lines and resolve any referenced sha1's\n-\twhile read -r pick sha1 message\n-\tdo\n-\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n-\t\tcase \"$message\" in\n-\t\t\"squash! \"*|\"fixup! \"*)\n-\t\t\taction=\"${message%%!*}\"\n-\t\t\trest=$message\n-\t\t\tprefix=\n-\t\t\t# skip all squash! or fixup! (but save for later)\n-\t\t\twhile :\n-\t\t\tdo\n-\t\t\t\tcase \"$rest\" in\n-\t\t\t\t\"squash! \"*|\"fixup! \"*)\n-\t\t\t\t\tprefix=\"$prefix${rest%%!*},\"\n-\t\t\t\t\trest=\"${rest#*! }\"\n-\t\t\t\t\t;;\n-\t\t\t\t*)\n-\t\t\t\t\tbreak\n-\t\t\t\t\t;;\n-\t\t\t\tesac\n-\t\t\tdone\n-\t\t\tprintf '%s %s %s %s\\n' \"$sha1\" \"$action\" \"$prefix\" \"$rest\"\n-\t\t\t# if it's a single word, try to resolve to a full sha1 and\n-\t\t\t# emit a second copy. This allows us to match on both message\n-\t\t\t# and on sha1 prefix\n-\t\t\tif test \"${rest#* }\" = \"$rest\"; then\n-\t\t\t\tfullsha=\"$(git rev-parse -q --verify \"$rest\" 2>/dev/null)\"\n-\t\t\t\tif test -n \"$fullsha\"; then\n-\t\t\t\t\t# prefix the action to uniquely identify this line as\n-\t\t\t\t\t# intended for full sha1 match\n-\t\t\t\t\techo \"$sha1 +$action $prefix $fullsha\"\n-\t\t\t\tfi\n-\t\t\tfi\n-\t\tesac\n-\tdone >\"$1.sq\" <\"$1\"\n-\ttest -s \"$1.sq\" || return\n-\n-\tused=\n-\twhile read -r pick sha1 message\n-\tdo\n-\t\tcase \" $used\" in\n-\t\t*\" $sha1 \"*) continue ;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$pick $sha1 $message\"\n-\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n-\t\tused=\"$used$sha1 \"\n-\t\twhile read -r squash action msg_prefix msg_content\n-\t\tdo\n-\t\t\tcase \" $used\" in\n-\t\t\t*\" $squash \"*) continue ;;\n-\t\t\tesac\n-\t\t\temit=0\n-\t\t\tcase \"$action\" in\n-\t\t\t+*)\n-\t\t\t\taction=\"${action#+}\"\n-\t\t\t\t# full sha1 prefix test\n-\t\t\t\tcase \"$msg_content\" in \"$sha1\"*) emit=1;; esac ;;\n-\t\t\t*)\n-\t\t\t\t# message prefix test\n-\t\t\t\tcase \"$message\" in \"$msg_content\"*) emit=1;; esac ;;\n-\t\t\tesac\n-\t\t\tif test $emit = 1; then\n-\t\t\t\tif test -n \"${format}\"\n-\t\t\t\tthen\n-\t\t\t\t\tmsg_content=$(git log -n 1 --format=\"${format}\" ${squash})\n-\t\t\t\telse\n-\t\t\t\t\tmsg_content=\"$(echo \"$msg_prefix\" | sed \"s/,/! /g\")$msg_content\"\n-\t\t\t\tfi\n-\t\t\t\tprintf '%s\\n' \"$action $squash $msg_content\"\n-\t\t\t\tused=\"$used$squash \"\n-\t\t\tfi\n-\t\tdone <\"$1.sq\"\n-\tdone >\"$1.rearranged\" <\"$1\"\n-\tcat \"$1.rearranged\" >\"$1\"\n-\trm -f \"$1.sq\" \"$1.rearranged\"\n-}\n-\n # Add commands after a pick or after a squash/fixup serie\n # in the todo list.\n add_exec_commands () {\n@@ -1091,7 +1003,7 @@ then\n fi\n \n test -s \"$todo\" || echo noop >> \"$todo\"\n-test -n \"$autosquash\" && rearrange_squash \"$todo\"\n+test -z \"$autosquash\" || git rebase--helper --rearrange-squash || exit\n test -n \"$cmd\" && add_exec_commands \"$todo\"\n \n todocount=$(git stripspace --strip-comments <\"$todo\" | wc -l)\ndiff --git a/sequencer.c b/sequencer.c\nindex bbbc98c9116..2b07fb9e0ce 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -19,6 +19,7 @@\n #include \"trailer.h\"\n #include \"log-tree.h\"\n #include \"wt-status.h\"\n+#include \"hashmap.h\"\n \n #define GIT_REFLOG_ACTION \"GIT_REFLOG_ACTION\"\n \n@@ -2706,3 +2707,199 @@ int skip_unnecessary_picks(void)\n \n \treturn 0;\n }\n+\n+struct subject2item_entry {\n+\tstruct hashmap_entry entry;\n+\tint i;\n+\tchar subject[FLEX_ARRAY];\n+};\n+\n+static int subject2item_cmp(const struct subject2item_entry *a,\n+\tconst struct subject2item_entry *b, const void *key)\n+{\n+\treturn key ? strcmp(a->subject, key) : strcmp(a->subject, b->subject);\n+}\n+\n+/*\n+ * Rearrange the todo list that has both \"pick sha1 msg\" and \"pick sha1\n+ * fixup!/squash! msg\" in it so that the latter is put immediately after the\n+ * former, and change \"pick\" to \"fixup\"/\"squash\".\n+ *\n+ * Note that if the config has specified a custom instruction format, each log\n+ * message will have to be retrieved from the commit (as the oneline in the\n+ * script cannot be trusted) in order to normalize the autosquash arrangement.\n+ */\n+int rearrange_squash(void)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct hashmap subject2item;\n+\tint res = 0, rearranged = 0, *next, *tail, fd, i;\n+\tchar **subjects;\n+\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0)\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn -1;\n+\t}\n+\n+\t/*\n+\t * The hashmap maps onelines to the respective todo list index.\n+\t *\n+\t * If any items need to be rearranged, the next[i] value will indicate\n+\t * which item was moved directly after the i'th.\n+\t *\n+\t * In that case, last[i] will indicate the index of the latest item to\n+\t * be moved to appear after the i'th.\n+\t */\n+\thashmap_init(&subject2item, (hashmap_cmp_fn) subject2item_cmp,\n+\t\t todo_list.nr);\n+\tALLOC_ARRAY(next, todo_list.nr);\n+\tALLOC_ARRAY(tail, todo_list.nr);\n+\tALLOC_ARRAY(subjects, todo_list.nr);\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tconst char *commit_buffer, *subject, *p;\n+\t\tint i2 = -1;\n+\t\tstruct subject2item_entry *entry;\n+\n+\t\tnext[i] = tail[i] = -1;\n+\t\tif (item->command >= TODO_EXEC) {\n+\t\t\tsubjects[i] = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (is_fixup(item->command)) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error(_(\"the script was already rearranged.\"));\n+\t\t}\n+\n+\t\titem->commit->util = item;\n+\n+\t\tparse_commit(item->commit);\n+\t\tcommit_buffer = get_commit_buffer(item->commit, NULL);\n+\t\tfind_commit_subject(commit_buffer, &subject);\n+\t\tformat_subject(&buf, subject, \" \");\n+\t\tsubject = subjects[i] = buf.buf;\n+\t\tunuse_commit_buffer(item->commit, commit_buffer);\n+\t\tif ((skip_prefix(subject, \"fixup! \", &p) ||\n+\t\t skip_prefix(subject, \"squash! \", &p))) {\n+\t\t\tstruct commit *commit2;\n+\n+\t\t\tfor (;;) {\n+\t\t\t\twhile (isspace(*p))\n+\t\t\t\t\tp++;\n+\t\t\t\tif (!skip_prefix(p, \"fixup! \", &p) &&\n+\t\t\t\t !skip_prefix(p, \"squash! \", &p))\n+\t\t\t\t\tbreak;\n+\t\t\t}\n+\n+\t\t\tif ((entry = hashmap_get_from_hash(&subject2item,\n+\t\t\t\t\t\t\t strhash(p), p)))\n+\t\t\t\t/* found by title */\n+\t\t\t\ti2 = entry->i;\n+\t\t\telse if (!strchr(p, ' ') &&\n+\t\t\t\t (commit2 =\n+\t\t\t\t lookup_commit_reference_by_name(p)) &&\n+\t\t\t\t commit2->util)\n+\t\t\t\t/* found by commit name */\n+\t\t\t\ti2 = (struct todo_item *)commit2->util\n+\t\t\t\t\t- todo_list.items;\n+\t\t\telse {\n+\t\t\t\t/* copy can be a prefix of the commit subject */\n+\t\t\t\tfor (i2 = 0; i2 < i; i2++)\n+\t\t\t\t\tif (subjects[i2] &&\n+\t\t\t\t\t starts_with(subjects[i2], p))\n+\t\t\t\t\t\tbreak;\n+\t\t\t\tif (i2 == i)\n+\t\t\t\t\ti2 = -1;\n+\t\t\t}\n+\t\t}\n+\t\tif (i2 >= 0) {\n+\t\t\trearranged = 1;\n+\t\t\ttodo_list.items[i].command =\n+\t\t\t\tstarts_with(subject, \"fixup!\") ?\n+\t\t\t\tTODO_FIXUP : TODO_SQUASH;\n+\t\t\tif (next[i2] < 0)\n+\t\t\t\tnext[i2] = i;\n+\t\t\telse\n+\t\t\t\tnext[tail[i2]] = i;\n+\t\t\ttail[i2] = i;\n+\t\t} else if (!hashmap_get_from_hash(&subject2item,\n+\t\t\t\t\t\tstrhash(subject), subject)) {\n+\t\t\tFLEX_ALLOC_MEM(entry, subject, buf.buf, buf.len);\n+\t\t\tentry->i = i;\n+\t\t\thashmap_entry_init(entry, strhash(entry->subject));\n+\t\t\thashmap_put(&subject2item, entry);\n+\t\t}\n+\t\tstrbuf_detach(&buf, NULL);\n+\t}\n+\n+\tif (rearranged) {\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\t\tchar *format = NULL;\n+\n+\t\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n+\t\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\t\tenum todo_command command = todo_list.items[i].command;\n+\t\t\tint cur = i;\n+\n+\t\t\t/*\n+\t\t\t * Initially, all commands are 'pick's. If it is a\n+\t\t\t * fixup or a squash now, we have rearranged it.\n+\t\t\t */\n+\t\t\tif (is_fixup(command))\n+\t\t\t\tcontinue;\n+\n+\t\t\twhile (cur >= 0) {\n+\t\t\t\tint offset = todo_list.items[cur].offset_in_buf;\n+\t\t\t\tint end_offset = cur + 1 < todo_list.nr ?\n+\t\t\t\t\ttodo_list.items[cur + 1].offset_in_buf :\n+\t\t\t\t\ttodo_list.buf.len;\n+\t\t\t\tchar *bol = todo_list.buf.buf + offset;\n+\t\t\t\tchar *eol = todo_list.buf.buf + end_offset;\n+\n+\t\t\t\t/* replace 'pick', by 'fixup' or 'squash' */\n+\t\t\t\tcommand = todo_list.items[cur].command;\n+\t\t\t\tif (is_fixup(command)) {\n+\t\t\t\t\tstrbuf_addstr(&buf,\n+\t\t\t\t\t\ttodo_command_info[command].str);\n+\t\t\t\t\tbol += strcspn(bol, \" \\t\");\n+\t\t\t\t}\n+\n+\t\t\t\tstrbuf_add(&buf, bol, eol - bol);\n+\n+\t\t\t\tcur = next[cur];\n+\t\t\t}\n+\t\t}\n+\n+\t\tfd = open(todo_file, O_WRONLY);\n+\t\tif (fd < 0)\n+\t\t\tres = error_errno(_(\"could not open '%s'\"), todo_file);\n+\t\telse if (write(fd, buf.buf, buf.len) < 0)\n+\t\t\tres = error_errno(_(\"could not read '%s'.\"), todo_file);\n+\t\telse if (ftruncate(fd, buf.len) < 0)\n+\t\t\tres = error_errno(_(\"could not finish '%s'\"),\n+\t\t\t\t\t todo_file);\n+\t\tclose(fd);\n+\t\tstrbuf_release(&buf);\n+\t}\n+\n+\tfree(next);\n+\tfree(tail);\n+\tfor (i = 0; i < todo_list.nr; i++)\n+\t\tfree(subjects[i]);\n+\tfree(subjects);\n+\thashmap_free(&subject2item, 1);\n+\ttodo_list_release(&todo_list);\n+\n+\treturn res;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 28e1fc1e9bb..1c94bec7622 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -51,6 +51,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n int transform_todo_ids(int shorten_sha1s);\n int check_todo_list(void);\n int skip_unnecessary_picks(void);\n+int rearrange_squash(void);\n \n extern const char sign_off_header[];\n \ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 9fd629a6e21..b9e26008a79 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -278,7 +278,7 @@ set_backup_editor () {\n \ttest_set_editor \"$PWD/backup-editor.sh\"\n }\n \n-test_expect_failure 'autosquash with multiple empty patches' '\n+test_expect_success 'autosquash with multiple empty patches' '\n \ttest_tick &&\n \tgit commit --allow-empty -m \"empty\" &&\n \ttest_tick &&\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n"},{"id":"317813","messageId":"ecab086a62e96388f4dfc7e36a679821fdc7c8c3.1493128210.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v2 7/9] rebase -i: skip unnecessary picks using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-25T13:52:10Z","receivedAt":"2017-04-25T13:52:56Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"In particular on Windows, where shell scripts are even more expensive\nthan on MacOSX or Linux, it makes sense to move a loop that forks\nGit at least once for every line in the todo list into a builtin.\n\nNote: The original code did not try to skip unnecessary picks of root\ncommits but punts instead (probably --root was not considered common\nenough of a use case to bother optimizing). We do the same, for now.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 6 +++-\n git-rebase--interactive.sh | 41 ++-------------------\n sequencer.c | 90 ++++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n 4 files changed, 99 insertions(+), 39 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex e706eac710d..de3ccd9bfbc 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tint keep_empty = 0;\n \tenum {\n \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n-\t\tCHECK_TODO_LIST\n+\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -31,6 +31,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n \t\tOPT_CMDMODE(0, \"check-todo-list\", &command,\n \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n+\t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n+\t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n \t\tOPT_END()\n \t};\n \n@@ -55,5 +57,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!transform_todo_ids(0);\n \tif (command == CHECK_TODO_LIST && argc == 1)\n \t\treturn !!check_todo_list();\n+\tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n+\t\treturn !!skip_unnecessary_picks();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 1649506e1e4..931bc09e0cf 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -713,43 +713,6 @@ do_rest () {\n \tdone\n }\n \n-# skip picking commits whose parents are unchanged\n-skip_unnecessary_picks () {\n-\tfd=3\n-\twhile read -r command rest\n-\tdo\n-\t\t# fd=3 means we skip the command\n-\t\tcase \"$fd,$command\" in\n-\t\t3,pick|3,p)\n-\t\t\t# pick a commit whose parent is current $onto -> skip\n-\t\t\tsha1=${rest%% *}\n-\t\t\tcase \"$(git rev-parse --verify --quiet \"$sha1\"^)\" in\n-\t\t\t\"$onto\"*)\n-\t\t\t\tonto=$sha1\n-\t\t\t\t;;\n-\t\t\t*)\n-\t\t\t\tfd=1\n-\t\t\t\t;;\n-\t\t\tesac\n-\t\t\t;;\n-\t\t3,\"$comment_char\"*|3,)\n-\t\t\t# copy comments\n-\t\t\t;;\n-\t\t*)\n-\t\t\tfd=1\n-\t\t\t;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$command${rest:+ }$rest\" >&$fd\n-\tdone <\"$todo\" >\"$todo.new\" 3>>\"$done\" &&\n-\tmv -f \"$todo\".new \"$todo\" &&\n-\tcase \"$(peek_next_command)\" in\n-\tsquash|s|fixup|f)\n-\t\trecord_in_rewritten \"$onto\"\n-\t\t;;\n-\tesac ||\n-\t\tdie \"$(gettext \"Could not skip unnecessary pick commands\")\"\n-}\n-\n transform_todo_ids () {\n \twhile read -r command rest\n \tdo\n@@ -1172,7 +1135,9 @@ git rebase--helper --check-todo-list || {\n \n expand_todo_ids\n \n-test -d \"$rewritten\" || test -n \"$force_rebase\" || skip_unnecessary_picks\n+test -d \"$rewritten\" || test -n \"$force_rebase\" ||\n+onto=\"$(git rebase--helper --skip-unnecessary-picks)\" ||\n+die \"Could not skip unnecessary pick commands\"\n \n checkout_onto\n if test -z \"$rebase_root\" && test ! -d \"$rewritten\"\ndiff --git a/sequencer.c b/sequencer.c\nindex 3a935fa4cbc..bbbc98c9116 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2616,3 +2616,93 @@ int check_todo_list(void)\n \n \treturn res;\n }\n+\n+/* skip picking commits whose parents are unchanged */\n+int skip_unnecessary_picks(void)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct object_id onto_oid, *oid = &onto_oid, *parent_oid;\n+\tint fd, i;\n+\n+\tif (!read_oneliner(&buf, rebase_path_onto(), 0))\n+\t\treturn error(_(\"could not read 'onto'\"));\n+\tif (get_sha1(buf.buf, onto_oid.hash)) {\n+\t\tstrbuf_release(&buf);\n+\t\treturn error(_(\"need a HEAD to fixup\"));\n+\t}\n+\tstrbuf_release(&buf);\n+\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0) {\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn -1;\n+\t}\n+\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\n+\t\tif (item->command >= TODO_NOOP)\n+\t\t\tcontinue;\n+\t\tif (item->command != TODO_PICK)\n+\t\t\tbreak;\n+\t\tif (parse_commit(item->commit)) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error(_(\"could not parse commit '%s'\"),\n+\t\t\t\toid_to_hex(&item->commit->object.oid));\n+\t\t}\n+\t\tif (!item->commit->parents)\n+\t\t\tbreak; /* root commit */\n+\t\tif (item->commit->parents->next)\n+\t\t\tbreak; /* merge commit */\n+\t\tparent_oid = &item->commit->parents->item->object.oid;\n+\t\tif (hashcmp(parent_oid->hash, oid->hash))\n+\t\t\tbreak;\n+\t\toid = &item->commit->object.oid;\n+\t}\n+\tif (i > 0) {\n+\t\tint offset = i < todo_list.nr ?\n+\t\t\ttodo_list.items[i].offset_in_buf : todo_list.buf.len;\n+\t\tconst char *done_path = rebase_path_done();\n+\n+\t\tfd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n+\t\tif (write_in_full(fd, todo_list.buf.buf, offset) < 0) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n+\t\t\t\tdone_path);\n+\t\t}\n+\t\tclose(fd);\n+\n+\t\tfd = open(rebase_path_todo(), O_WRONLY, 0666);\n+\t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n+\t\t\t\ttodo_list.buf.len - offset) < 0) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n+\t\t\t\trebase_path_todo());\n+\t\t}\n+\t\tif (ftruncate(fd, todo_list.buf.len - offset) < 0) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error_errno(_(\"could not truncate '%s'\"),\n+\t\t\t\trebase_path_todo());\n+\t\t}\n+\t\tclose(fd);\n+\n+\t\ttodo_list.current = i;\n+\t\tif (is_fixup(peek_command(&todo_list, 0)))\n+\t\t\trecord_in_rewritten(oid, peek_command(&todo_list, 0));\n+\t}\n+\n+\ttodo_list_release(&todo_list);\n+\tprintf(\"%s\\n\", oid_to_hex(oid));\n+\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 4978a61b83b..28e1fc1e9bb 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -50,6 +50,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \n int transform_todo_ids(int shorten_sha1s);\n int check_todo_list(void);\n+int skip_unnecessary_picks(void);\n \n extern const char sign_off_header[];\n \n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317895","messageId":"xmqq1ssfu9mo.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v2 0/9] The final building block for a faster rebase -i","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-26T03:32:15Z","receivedAt":"2017-04-26T03:32:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> This patch series reimplements the expensive pre- and post-processing of\n> the todo script in C.\n>\n> And it concludes the work I did to accelerate rebase -i.\n\nI noticed (this is merely \"I felt\"; I didn't measure) that recent\n\"rebase -i\" sessions I do on a Linux box is already plenty faster\nthan before, and it is good to finally see the end of the long road.\n\nThanks.\n"},{"id":"317915","messageId":"20170426104511.nuk3fcxofhrtxsma@sigill.intra.peff.net","threadId":"43983","inReplyTo":"361682a84f421ff16a8c6b7120f132cd6b61f1fa.1493128210.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v2 1/9] rebase -i: generate the script via rebase--helper","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-04-26T10:45:11Z","receivedAt":"2017-04-26T10:45:38Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Apr 25, 2017 at 03:51:49PM +0200, Johannes Schindelin wrote:\n\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> [...]\n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv)\n> +{\n> +\tchar *format = \"%s\";\n\nI'm surprised the compiler doesn't complain about assigning a string\nliteral to a non-const pointer. It makes me worried that we would call\nfree() on it later. We don't, but that means...\n\n> +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n\n...that this assignment to \"format\" leaks.\n\nSo perhaps you'd want to xstrdup the literal, and then make sure the\nresult is freed? Or alternatively use an extra level of indirection (a\nto_free pointer to store the config value, and then a const pointer for\n\"format\").\n\n-Peff\n"},{"id":"317916","messageId":"20170426105503.wewlbefrsdmjqnob@sigill.intra.peff.net","threadId":"43983","inReplyTo":"ecab086a62e96388f4dfc7e36a679821fdc7c8c3.1493128210.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v2 7/9] rebase -i: skip unnecessary picks using the rebase--helper","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-04-26T10:55:03Z","receivedAt":"2017-04-26T10:55:12Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Apr 25, 2017 at 03:52:10PM +0200, Johannes Schindelin wrote:\n\n> diff --git a/sequencer.c b/sequencer.c\n> index 3a935fa4cbc..bbbc98c9116 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -2616,3 +2616,93 @@ int check_todo_list(void)\n> \n> \treturn res;\n> }\n> +\n> +/* skip picking commits whose parents are unchanged */\n> +int skip_unnecessary_picks(void)\n\nCoverity warns of some descriptor leaks in this function (and in\nrearrange_squash). I think you get those emails, so I won't repeat the\ndetails here. But I while looking at them I did notice something it\ndidn't mention:\n\n> +\tif (i > 0) {\n> +\t\tint offset = i < todo_list.nr ?\n> +\t\t\ttodo_list.items[i].offset_in_buf : todo_list.buf.len;\n> +\t\tconst char *done_path = rebase_path_done();\n> +\n> +\t\tfd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n> +\t\tif (write_in_full(fd, todo_list.buf.buf, offset) < 0) {\n> +\t\t\ttodo_list_release(&todo_list);\n> +\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n> +\t\t\t\tdone_path);\n> +\t\t}\n> +\t\tclose(fd);\n\nThis should probably check the result of open(). I know write_in_full()\nwill fail if fd is -1, but we'd rather show the user the errno from\nopen(), not EBADF.\n\nTechnically the free() calls from todo_list_release() can also munge\nerrno before you print it. You might want to just call error_errno()\nfirst, then do the cleanup (including the missing close()).\n\n> +\n> +\t\tfd = open(rebase_path_todo(), O_WRONLY, 0666);\n> +\t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n> +\t\t\t\ttodo_list.buf.len - offset) < 0) {\n> +\t\t\ttodo_list_release(&todo_list);\n> +\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n> +\t\t\t\trebase_path_todo());\n> +\t\t}\n\nDitto here.\n\n-Peff\n"},{"id":"317917","messageId":"alpine.DEB.2.20.1704261321470.3480@virtualbox","threadId":"43983","inReplyTo":"20170426105503.wewlbefrsdmjqnob@sigill.intra.peff.net","subject":"Re: [PATCH v2 7/9] rebase -i: skip unnecessary picks using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:31:20Z","receivedAt":"2017-04-26T11:31:40Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Wed, 26 Apr 2017, Jeff King wrote:\n\n> On Tue, Apr 25, 2017 at 03:52:10PM +0200, Johannes Schindelin wrote:\n> \n> > diff --git a/sequencer.c b/sequencer.c\n> > index 3a935fa4cbc..bbbc98c9116 100644\n> > --- a/sequencer.c\n> > +++ b/sequencer.c\n> > @@ -2616,3 +2616,93 @@ int check_todo_list(void)\n> > \n> > \treturn res;\n> > }\n> > +\n> > +/* skip picking commits whose parents are unchanged */\n> > +int skip_unnecessary_picks(void)\n> \n> Coverity warns of some descriptor leaks in this function (and in\n> rearrange_squash). I think you get those emails, so I won't repeat the\n> details here.\n\nI do. The leaks in rearrange_squash() seem to be false positives (I will\nhave to have another look later, I spent way too many hours pouring over\nthose Coverity reports this week).\n\nThe leaks in skip_unnecessary_picks() are real, though, and I have a patch\nto fix them this way:\n\n-- snip --\nSubject: [PATCH] sequencer: plug resource leak when skipping unnecessary picks\n\nThe resource leak only happens in case of an error writing or truncating\nthe file, therefore it seems less critical, but we should still fix it\nnonetheless.\n\nDiscovered by Coverity.\n\nSigned-off-by: Johannes Schindelin \n---\n sequencer.c | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/sequencer.c b/sequencer.c\nindex e25a4e1180c..9c765e8850a 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2678,11 +2678,13 @@ int skip_unnecessary_picks(void)\n \t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n \t\t\t\ttodo_list.buf.len - offset) < 0) {\n \t\t\ttodo_list_release(&todo_list);\n+\t\t\tclose(fd);\n \t\t\treturn error_errno(_(\"could not write to '%s'\"),\n \t\t\t\trebase_path_todo());\n \t\t}\n \t\tif (ftruncate(fd, todo_list.buf.len - offset) < 0) {\n \t\t\ttodo_list_release(&todo_list);\n+\t\t\tclose(fd);\n \t\t\treturn error_errno(_(\"could not truncate '%s'\"),\n \t\t\t\trebase_path_todo());\n \t\t}\n-- snap --\n\n> But I while looking at them I did notice something it didn't mention:\n> \n> > +\tif (i > 0) {\n> > +\t\tint offset = i < todo_list.nr ?\n> > +\t\t\ttodo_list.items[i].offset_in_buf : todo_list.buf.len;\n> > +\t\tconst char *done_path = rebase_path_done();\n> > +\n> > +\t\tfd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n> > +\t\tif (write_in_full(fd, todo_list.buf.buf, offset) < 0) {\n> > +\t\t\ttodo_list_release(&todo_list);\n> > +\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n> > +\t\t\t\tdone_path);\n> > +\t\t}\n> > +\t\tclose(fd);\n> \n> This should probably check the result of open().\n\nIndeed.\n\n> I know write_in_full() will fail if fd is -1, but we'd rather show the\n> user the errno from open(), not EBADF.\n\nI guess Coverity follows that code path and determines that it is handled.\n\nIf only it would also follow the code paths of the FLEX_ARRAYs and figure\nout that we play games with struct definitions whose last entries are\ntechnically incorrect.\n\n> Technically the free() calls from todo_list_release() can also munge\n> errno before you print it. You might want to just call error_errno()\n> first, then do the cleanup (including the missing close()).\n\nAh, you're right. I'll have to rework that patch I mentioned above.\n\n> > +\t\tfd = open(rebase_path_todo(), O_WRONLY, 0666);\n> > +\t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n> > +\t\t\t\ttodo_list.buf.len - offset) < 0) {\n> > +\t\t\ttodo_list_release(&todo_list);\n> > +\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n> > +\t\t\t\trebase_path_todo());\n> > +\t\t}\n> \n> Ditto here.\n\nRight.\n\nCiao,\nDscho\n"},{"id":"317918","messageId":"alpine.DEB.2.20.1704261333480.3480@virtualbox","threadId":"43983","inReplyTo":"20170426104511.nuk3fcxofhrtxsma@sigill.intra.peff.net","subject":"Re: [PATCH v2 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:34:16Z","receivedAt":"2017-04-26T11:34:50Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Wed, 26 Apr 2017, Jeff King wrote:\n\n> On Tue, Apr 25, 2017 at 03:51:49PM +0200, Johannes Schindelin wrote:\n> \n> > --- a/sequencer.c\n> > +++ b/sequencer.c\n> > [...]\n> > +int sequencer_make_script(int keep_empty, FILE *out,\n> > +\t\tint argc, const char **argv)\n> > +{\n> > +\tchar *format = \"%s\";\n> \n> I'm surprised the compiler doesn't complain about assigning a string\n> literal to a non-const pointer. It makes me worried that we would call\n> free() on it later. We don't, but that means...\n> \n> > +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n> \n> ...that this assignment to \"format\" leaks.\n> \n> So perhaps you'd want to xstrdup the literal, and then make sure the\n> result is freed? Or alternatively use an extra level of indirection (a\n> to_free pointer to store the config value, and then a const pointer for\n> \"format\").\n\nGood suggestion. Will be part of v3,\nDscho\n"},{"id":"317919","messageId":"c44a15ed1f1015d7e9377e18610a0c428786995b.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:59:13Z","receivedAt":"2017-04-26T11:59:38Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The first step of an interactive rebase is to generate the so-called \"todo\nscript\", to be stored in the state directory as \"git-rebase-todo\" and to\nbe edited by the user.\n\nOriginally, we adjusted the output of `git log ` using a simple\nsed script. Over the course of the years, the code became more\ncomplicated. We now use shell scripting to edit the output of `git log`\nconditionally, depending whether to keep \"empty\" commits (i.e. commits\nthat do not change any files).\n\nOn platforms where shell scripting is not native, this can be a serious\ndrag. And it opens the door for incompatibilities between platforms when\nit comes to shell scripting or to Unix-y commands.\n\nLet's just re-implement the todo script generation in plain C, using the\nrevision machinery directly.\n\nThis is substantially faster, improving the speed relative to the\nshell script version of the interactive rebase from 2x to 3x on Windows.\n\nNote that the rearrange_squash() function in git-rebase--interactive\nrelied on the fact that we set the \"format\" variable to the config setting\nrebase.instructionFormat. Relying on a side effect like this is no good,\nhence we explicitly perform that assignment (possibly again) in\nrearrange_squash().\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 8 +++++++-\n git-rebase--interactive.sh | 44 +++++++++++++++++++++++---------------------\n sequencer.c | 45 +++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 3 +++\n 4 files changed, 78 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex ca1ebb2fa18..821058d452d 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -11,15 +11,19 @@ static const char * const builtin_rebase_helper_usage[] = {\n int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n {\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n+\tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n+\t\tOPT_BOOL(0, \"keep-empty\", &keep_empty, N_(\"keep empty commits\")),\n \t\tOPT_CMDMODE(0, \"continue\", &command, N_(\"continue rebase\"),\n \t\t\t\tCONTINUE),\n \t\tOPT_CMDMODE(0, \"abort\", &command, N_(\"abort rebase\"),\n \t\t\t\tABORT),\n+\t\tOPT_CMDMODE(0, \"make-script\", &command,\n+\t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n \t\tOPT_END()\n \t};\n \n@@ -36,5 +40,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!sequencer_continue(&opts);\n \tif (command == ABORT && argc == 1)\n \t\treturn !!sequencer_remove_state(&opts);\n+\tif (command == MAKE_SCRIPT && argc > 1)\n+\t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 2c9c0165b5a..609e150d38f 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -785,6 +785,7 @@ collapse_todo_ids() {\n # each log message will be re-retrieved in order to normalize the\n # autosquash arrangement\n rearrange_squash () {\n+\tformat=$(git config --get rebase.instructionFormat)\n \t# extract fixup!/squash! lines and resolve any referenced sha1's\n \twhile read -r pick sha1 message\n \tdo\n@@ -1210,26 +1211,27 @@ else\n \trevisions=$onto...$orig_head\n \tshortrevisions=$shorthead\n fi\n-format=$(git config --get rebase.instructionFormat)\n-# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n-git rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n-\t--reverse --left-right --topo-order \\\n-\t$revisions ${restrict_revision+^$restrict_revision} | \\\n-\tsed -n \"s/^>//p\" |\n-while read -r sha1 rest\n-do\n-\n-\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n-\tthen\n-\t\tcomment_out=\"$comment_char \"\n-\telse\n-\t\tcomment_out=\n-\tfi\n+if test t != \"$preserve_merges\"\n+then\n+\tgit rebase--helper --make-script ${keep_empty:+--keep-empty} \\\n+\t\t$revisions ${restrict_revision+^$restrict_revision} >\"$todo\"\n+else\n+\tformat=$(git config --get rebase.instructionFormat)\n+\t# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n+\tgit rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n+\t\t--reverse --left-right --topo-order \\\n+\t\t$revisions ${restrict_revision+^$restrict_revision} | \\\n+\t\tsed -n \"s/^>//p\" |\n+\twhile read -r sha1 rest\n+\tdo\n+\n+\t\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n+\t\tthen\n+\t\t\tcomment_out=\"$comment_char \"\n+\t\telse\n+\t\t\tcomment_out=\n+\t\tfi\n \n-\tif test t != \"$preserve_merges\"\n-\tthen\n-\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n-\telse\n \t\tif test -z \"$rebase_root\"\n \t\tthen\n \t\t\tpreserve=t\n@@ -1248,8 +1250,8 @@ do\n \t\t\ttouch \"$rewritten\"/$sha1\n \t\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n \t\tfi\n-\tfi\n-done\n+\tdone\n+fi\n \n # Watch for commits that been dropped by --cherry-pick\n if test t = \"$preserve_merges\"\ndiff --git a/sequencer.c b/sequencer.c\nindex 77afecaebf0..e858a976279 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2388,3 +2388,48 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n \n \tstrbuf_release(&sob);\n }\n+\n+int sequencer_make_script(int keep_empty, FILE *out,\n+\t\tint argc, const char **argv)\n+{\n+\tchar *format = xstrdup(\"%s\");\n+\tstruct pretty_print_context pp = {0};\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct rev_info revs;\n+\tstruct commit *commit;\n+\n+\tinit_revisions(&revs, NULL);\n+\trevs.verbose_header = 1;\n+\trevs.max_parents = 1;\n+\trevs.cherry_pick = 1;\n+\trevs.limited = 1;\n+\trevs.reverse = 1;\n+\trevs.right_only = 1;\n+\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n+\trevs.topo_order = 1;\n+\n+\trevs.pretty_given = 1;\n+\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n+\tget_commit_format(format, &revs);\n+\tfree(format);\n+\tpp.fmt = revs.commit_format;\n+\tpp.output_encoding = get_log_output_encoding();\n+\n+\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n+\t\treturn error(_(\"make_script: unhandled options\"));\n+\n+\tif (prepare_revision_walk(&revs) < 0)\n+\t\treturn error(_(\"make_script: error preparing revisions\"));\n+\n+\twhile ((commit = get_revision(&revs))) {\n+\t\tstrbuf_reset(&buf);\n+\t\tif (!keep_empty && is_original_commit_empty(commit))\n+\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n+\t\tstrbuf_addf(&buf, \"pick %s \", oid_to_hex(&commit->object.oid));\n+\t\tpretty_print_commit(&pp, commit, &buf);\n+\t\tstrbuf_addch(&buf, '\\n');\n+\t\tfputs(buf.buf, out);\n+\t}\n+\tstrbuf_release(&buf);\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex f885b68395f..83f2943b7a9 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -45,6 +45,9 @@ int sequencer_continue(struct replay_opts *opts);\n int sequencer_rollback(struct replay_opts *opts);\n int sequencer_remove_state(struct replay_opts *opts);\n \n+int sequencer_make_script(int keep_empty, FILE *out,\n+\t\tint argc, const char **argv);\n+\n extern const char sign_off_header[];\n \n void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317920","messageId":"12ea7b6754b3378be2b8e69d0fc5bb146bd2df63.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 2/9] rebase -i: remove useless indentation","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:59:17Z","receivedAt":"2017-04-26T11:59:40Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The commands used to be indented, and it is nice to look at, but when we\ntransform the SHA-1s, the indentation is removed. So let's do away with it.\n\nFor the moment, at least: when we will use the upcoming rebase--helper\nto transform the SHA-1s, we *will* keep the indentation and can\nreintroduce it. Yet, to be able to validate the rebase--helper against\nthe output of the current shell script version, we need to remove the\nextra indentation.\n\nSigned-off-by: Johannes Schindelin \n---\n git-rebase--interactive.sh | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 609e150d38f..c40b1fd1d2e 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -155,13 +155,13 @@ reschedule_last_action () {\n append_todo_help () {\n \tgettext \"\n Commands:\n- p, pick = use commit\n- r, reword = use commit, but edit the commit message\n- e, edit = use commit, but stop for amending\n- s, squash = use commit, but meld into previous commit\n- f, fixup = like \\\"squash\\\", but discard this commit's log message\n- x, exec = run command (the rest of the line) using shell\n- d, drop = remove commit\n+p, pick = use commit\n+r, reword = use commit, but edit the commit message\n+e, edit = use commit, but stop for amending\n+s, squash = use commit, but meld into previous commit\n+f, fixup = like \\\"squash\\\", but discard this commit's log message\n+x, exec = run command (the rest of the line) using shell\n+d, drop = remove commit\n \n These lines can be re-ordered; they are executed from top to bottom.\n \" | git stripspace --comment-lines >>\"$todo\"\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317921","messageId":"d82ccad6fa09da821f19ca57cc6e418996e88e8c.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 3/9] rebase -i: do not invent onelines when expanding/collapsing SHA-1s","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:59:20Z","receivedAt":"2017-04-26T11:59:41Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To avoid problems with short SHA-1s that become non-unique during the\nrebase, we rewrite the todo script with short/long SHA-1s before and\nafter letting the user edit the script. Since SHA-1s are not intuitive\nfor humans, rebase -i also provides the onelines (commit message\nsubjects) in the script, purely for the user's convenience.\n\nIt is very possible to generate a todo script via different means than\nrebase -i and then to let rebase -i run with it; In this case, these\nonelines are not required.\n\nAnd this is where the expand/collapse machinery has a bug: it *expects*\nthat oneline, and failing to find one reuses the previous SHA-1 as\n\"oneline\".\n\nIt was most likely an oversight, and made implementation in the (quite\nlimiting) shell script language less convoluted. However, we are about\nto reimplement performance-critical parts in C (and due to spawning a\ngit.exe process for every single line of the todo script, the\nexpansion/collapsing of the SHA-1s *is* performance-hampering on\nWindows), therefore let's fix this bug to make cross-validation with the\nC version of that functionality possible.\n\nSigned-off-by: Johannes Schindelin \n---\n git-rebase--interactive.sh | 7 ++++++-\n 1 file changed, 6 insertions(+), 1 deletion(-)\n\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex c40b1fd1d2e..214af0372ba 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -760,7 +760,12 @@ transform_todo_ids () {\n \t\t\t;;\n \t\t*)\n \t\t\tsha1=$(git rev-parse --verify --quiet \"$@\" ${rest%%[\t ]*}) &&\n-\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n+\t\t\tif test \"a$rest\" = \"a${rest#*[\t ]}\"\n+\t\t\tthen\n+\t\t\t\trest=$sha1\n+\t\t\telse\n+\t\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n+\t\t\tfi\n \t\t\t;;\n \t\tesac\n \t\tprintf '%s\\n' \"$command${rest:+ }$rest\"\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317922","messageId":"975e7749c2fba9dc263cca1d04879c703c0b3c66.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 5/9] t3404: relax rebase.missingCommitsCheck tests","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:59:28Z","receivedAt":"2017-04-26T11:59:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"These tests were a bit anal about the *exact* warning/error message\nprinted by git rebase. But those messages are intended for the *end\nuser*, therefore it does not make sense to test so rigidly for the\n*exact* wording.\n\nIn the following, we will reimplement the missing commits check in\nthe sequencer, with slightly different words.\n\nSo let's just test for the parts in the warning/error message that\nwe *really* care about, nothing more, nothing less.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3404-rebase-interactive.sh | 22 ++++------------------\n 1 file changed, 4 insertions(+), 18 deletions(-)\n\ndiff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh\nindex 33d392ba112..61113be08a4 100755\n--- a/t/t3404-rebase-interactive.sh\n+++ b/t/t3404-rebase-interactive.sh\n@@ -1242,20 +1242,13 @@ test_expect_success 'rebase -i respects rebase.missingCommitsCheck = error' '\n \ttest B = $(git cat-file commit HEAD^ | sed -ne \\$p)\n '\n \n-cat >expect <actual &&\n-\ttest_i18ncmp expect actual &&\n+\ttest_i18ngrep \"badcmd $(git rev-list --oneline -1 master~1)\" actual &&\n+\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n \tFAKE_LINES=\"1 2 3 drop 4 5\" git rebase --edit-todo &&\n \tgit rebase --continue &&\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p) &&\n@@ -1277,20 +1270,13 @@ test_expect_success 'tabs and spaces are accepted in the todolist' '\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n '\n \n-cat >expect <actual &&\n-\ttest_i18ncmp expect actual &&\n+\ttest_i18ngrep \"edit XXXXXXX False commit\" actual &&\n+\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n \tFAKE_LINES=\"1 2 4 5 6\" git rebase --edit-todo &&\n \tgit rebase --continue &&\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317923","messageId":"df5263cc0cad5cffbd1932d4b9b8e5c0507536e6.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:59:23Z","receivedAt":"2017-04-26T11:59:48Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This is crucial to improve performance on Windows, as the speed is now\nmostly dominated by the SHA-1 transformation (because it spawns a new\nrev-parse process for *every* line, and spawning processes is pretty\nslow from Git for Windows' MSYS2 Bash).\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 10 +++++++-\n git-rebase--interactive.sh | 4 ++--\n sequencer.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 2 ++\n 4 files changed, 72 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex 821058d452d..9444c8d6c60 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n \tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -24,6 +24,10 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\t\tABORT),\n \t\tOPT_CMDMODE(0, \"make-script\", &command,\n \t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n+\t\tOPT_CMDMODE(0, \"shorten-sha1s\", &command,\n+\t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n+\t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n+\t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n \t\tOPT_END()\n \t};\n \n@@ -42,5 +46,9 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!sequencer_remove_state(&opts);\n \tif (command == MAKE_SCRIPT && argc > 1)\n \t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n+\tif (command == SHORTEN_SHA1S && argc == 1)\n+\t\treturn !!transform_todo_ids(1);\n+\tif (command == EXPAND_SHA1S && argc == 1)\n+\t\treturn !!transform_todo_ids(0);\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 214af0372ba..52a19e0bdb3 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -774,11 +774,11 @@ transform_todo_ids () {\n }\n \n expand_todo_ids() {\n-\ttransform_todo_ids\n+\tgit rebase--helper --expand-sha1s\n }\n \n collapse_todo_ids() {\n-\ttransform_todo_ids --short\n+\tgit rebase--helper --shorten-sha1s\n }\n \n # Rearrange the todo list that has both \"pick sha1 msg\" and\ndiff --git a/sequencer.c b/sequencer.c\nindex e858a976279..4b7f88b338f 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2433,3 +2433,62 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \tstrbuf_release(&buf);\n \treturn 0;\n }\n+\n+\n+int transform_todo_ids(int shorten_sha1s)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tint fd, res, i;\n+\tFILE *out;\n+\n+\tstrbuf_reset(&todo_list.buf);\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0)\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\n+\tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\tif (res) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn error(_(\"unusable instruction sheet: '%s'\"), todo_file);\n+\t}\n+\n+\tout = fopen(todo_file, \"w\");\n+\tif (!out) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn error(_(\"unable to open '%s' for writing\"), todo_file);\n+\t}\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tint bol = item->offset_in_buf;\n+\t\tconst char *p = todo_list.buf.buf + bol;\n+\t\tint eol = i + 1 < todo_list.nr ?\n+\t\t\ttodo_list.items[i + 1].offset_in_buf :\n+\t\t\ttodo_list.buf.len;\n+\n+\t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n+\t\t\tfwrite(p, eol - bol, 1, out);\n+\t\telse {\n+\t\t\tint eoc = strcspn(p, \" \\t\");\n+\t\t\tconst char *sha1 = shorten_sha1s ?\n+\t\t\t\tshort_commit_name(item->commit) :\n+\t\t\t\toid_to_hex(&item->commit->object.oid);\n+\n+\t\t\tif (!eoc) {\n+\t\t\t\tp += strspn(p, \" \\t\");\n+\t\t\t\teoc = strcspn(p, \" \\t\");\n+\t\t\t}\n+\n+\t\t\tfprintf(out, \"%.*s %s %.*s\\n\",\n+\t\t\t\teoc, p, sha1, item->arg_len, item->arg);\n+\t\t}\n+\t}\n+\tfclose(out);\n+\ttodo_list_release(&todo_list);\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 83f2943b7a9..47a81034e76 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -48,6 +48,8 @@ int sequencer_remove_state(struct replay_opts *opts);\n int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv);\n \n+int transform_todo_ids(int shorten_sha1s);\n+\n extern const char sign_off_header[];\n \n void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317924","messageId":"cover.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493128210.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 0/9] The final building block for a faster rebase -i","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:59:07Z","receivedAt":"2017-04-26T11:59:50Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This patch series reimplements the expensive pre- and post-processing of\nthe todo script in C.\n\nAnd it concludes the work I did to accelerate rebase -i.\n\nChanges since v2:\n\n- rearranged error_errno() calls to come before any subsequent free()\n and close()\n\n- now call close(fd) in case of error to avoid resource leaks\n\n- removed unused `format` variable holding the value of\n rebase.instructionFormat from rearrange_squash()\n\n- modified rearrange_squash() to make it easier to reason about\n subjects[i] taking custody of a strbuf's buffer (this should enable\n Coverity to see that there is no resource leak here)\n\n\nJohannes Schindelin (9):\n rebase -i: generate the script via rebase--helper\n rebase -i: remove useless indentation\n rebase -i: do not invent onelines when expanding/collapsing SHA-1s\n rebase -i: also expand/collapse the SHA-1s via the rebase--helper\n t3404: relax rebase.missingCommitsCheck tests\n rebase -i: check for missing commits in the rebase--helper\n rebase -i: skip unnecessary picks using the rebase--helper\n t3415: test fixup with wrapped oneline\n rebase -i: rearrange fixup/squash lines using the rebase--helper\n\n Documentation/git-rebase.txt | 16 +-\n builtin/rebase--helper.c | 29 ++-\n git-rebase--interactive.sh | 362 ++++------------------------\n sequencer.c | 531 ++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 8 +\n t/t3404-rebase-interactive.sh | 22 +-\n t/t3415-rebase-autosquash.sh | 16 +-\n 7 files changed, 641 insertions(+), 343 deletions(-)\n\n\nbase-commit: e2cb6ab84c94f147f1259260961513b40c36108a\nBased-On: rebase--helper at https://github.com/dscho/git\nFetch-Base-Via: git fetch https://github.com/dscho/git rebase--helper\nPublished-As: https://github.com/dscho/git/releases/tag/rebase-i-extra-v3\nFetch-It-Via: git fetch https://github.com/dscho/git rebase-i-extra-v3\n\nInterdiff vs v2:\n\n diff --git a/sequencer.c b/sequencer.c\n index 2b07fb9e0ce..7ac1792311e 100644\n --- a/sequencer.c\n +++ b/sequencer.c\n @@ -2393,7 +2393,7 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv)\n {\n -\tchar *format = \"%s\";\n +\tchar *format = xstrdup(\"%s\");\n \tstruct pretty_print_context pp = {0};\n \tstruct strbuf buf = STRBUF_INIT;\n \tstruct rev_info revs;\n @@ -2412,6 +2412,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \trevs.pretty_given = 1;\n \tgit_config_get_string(\"rebase.instructionFormat\", &format);\n \tget_commit_format(format, &revs);\n +\tfree(format);\n \tpp.fmt = revs.commit_format;\n \tpp.output_encoding = get_log_output_encoding();\n \n @@ -2676,24 +2677,41 @@ int skip_unnecessary_picks(void)\n \t\tconst char *done_path = rebase_path_done();\n \n \t\tfd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n +\t\tif (fd < 0) {\n +\t\t\terror_errno(_(\"could not open '%s' for writing\"),\n +\t\t\t\t done_path);\n +\t\t\ttodo_list_release(&todo_list);\n +\t\t\treturn -1;\n +\t\t}\n \t\tif (write_in_full(fd, todo_list.buf.buf, offset) < 0) {\n +\t\t\terror_errno(_(\"could not write to '%s'\"), done_path);\n \t\t\ttodo_list_release(&todo_list);\n -\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n -\t\t\t\tdone_path);\n +\t\t\tclose(fd);\n +\t\t\treturn -1;\n \t\t}\n \t\tclose(fd);\n \n \t\tfd = open(rebase_path_todo(), O_WRONLY, 0666);\n +\t\tif (fd < 0) {\n +\t\t\terror_errno(_(\"could not open '%s' for writing\"),\n +\t\t\t\t rebase_path_todo());\n +\t\t\ttodo_list_release(&todo_list);\n +\t\t\treturn -1;\n +\t\t}\n \t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n \t\t\t\ttodo_list.buf.len - offset) < 0) {\n +\t\t\terror_errno(_(\"could not write to '%s'\"),\n +\t\t\t\t rebase_path_todo());\n +\t\t\tclose(fd);\n \t\t\ttodo_list_release(&todo_list);\n -\t\t\treturn error_errno(_(\"could not write to '%s'\"),\n -\t\t\t\trebase_path_todo());\n +\t\t\treturn -1;\n \t\t}\n \t\tif (ftruncate(fd, todo_list.buf.len - offset) < 0) {\n +\t\t\terror_errno(_(\"could not truncate '%s'\"),\n +\t\t\t\t rebase_path_todo());\n \t\t\ttodo_list_release(&todo_list);\n -\t\t\treturn error_errno(_(\"could not truncate '%s'\"),\n -\t\t\t\trebase_path_todo());\n +\t\t\tclose(fd);\n +\t\t\treturn -1;\n \t\t}\n \t\tclose(fd);\n \n @@ -2768,6 +2786,7 @@ int rearrange_squash(void)\n \t\tstruct strbuf buf = STRBUF_INIT;\n \t\tstruct todo_item *item = todo_list.items + i;\n \t\tconst char *commit_buffer, *subject, *p;\n +\t\tsize_t subject_len;\n \t\tint i2 = -1;\n \t\tstruct subject2item_entry *entry;\n \n @@ -2788,7 +2807,7 @@ int rearrange_squash(void)\n \t\tcommit_buffer = get_commit_buffer(item->commit, NULL);\n \t\tfind_commit_subject(commit_buffer, &subject);\n \t\tformat_subject(&buf, subject, \" \");\n -\t\tsubject = subjects[i] = buf.buf;\n +\t\tsubject = subjects[i] = strbuf_detach(&buf, &subject_len);\n \t\tunuse_commit_buffer(item->commit, commit_buffer);\n \t\tif ((skip_prefix(subject, \"fixup! \", &p) ||\n \t\t skip_prefix(subject, \"squash! \", &p))) {\n @@ -2835,19 +2854,16 @@ int rearrange_squash(void)\n \t\t\ttail[i2] = i;\n \t\t} else if (!hashmap_get_from_hash(&subject2item,\n \t\t\t\t\t\tstrhash(subject), subject)) {\n -\t\t\tFLEX_ALLOC_MEM(entry, subject, buf.buf, buf.len);\n +\t\t\tFLEX_ALLOC_MEM(entry, subject, subject, subject_len);\n \t\t\tentry->i = i;\n \t\t\thashmap_entry_init(entry, strhash(entry->subject));\n \t\t\thashmap_put(&subject2item, entry);\n \t\t}\n -\t\tstrbuf_detach(&buf, NULL);\n \t}\n \n \tif (rearranged) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n -\t\tchar *format = NULL;\n \n -\t\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n \t\tfor (i = 0; i < todo_list.nr; i++) {\n \t\t\tenum todo_command command = todo_list.items[i].command;\n \t\t\tint cur = i;\n\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n"},{"id":"317925","messageId":"38bf320ec802386975843de6b5da4bbf823fb184.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 6/9] rebase -i: check for missing commits in the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T11:59:31Z","receivedAt":"2017-04-26T12:00:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"In particular on Windows, where shell scripts are even more expensive\nthan on MacOSX or Linux, it makes sense to move a loop that forks\nGit at least once for every line in the todo list into a builtin.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 7 +-\n git-rebase--interactive.sh | 164 ++-------------------------------------------\n sequencer.c | 125 ++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n 4 files changed, 137 insertions(+), 160 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex 9444c8d6c60..e706eac710d 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -13,7 +13,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n \tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n+\t\tCHECK_TODO_LIST\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -28,6 +29,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n \t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n \t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n+\t\tOPT_CMDMODE(0, \"check-todo-list\", &command,\n+\t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n \t\tOPT_END()\n \t};\n \n@@ -50,5 +53,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!transform_todo_ids(1);\n \tif (command == EXPAND_SHA1S && argc == 1)\n \t\treturn !!transform_todo_ids(0);\n+\tif (command == CHECK_TODO_LIST && argc == 1)\n+\t\treturn !!check_todo_list();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 52a19e0bdb3..1649506e1e4 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -890,96 +890,6 @@ add_exec_commands () {\n \tmv \"$1.new\" \"$1\"\n }\n \n-# Check if the SHA-1 passed as an argument is a\n-# correct one, if not then print $2 in \"$todo\".badsha\n-# $1: the SHA-1 to test\n-# $2: the line number of the input\n-# $3: the input filename\n-check_commit_sha () {\n-\tbadsha=0\n-\tif test -z \"$1\"\n-\tthen\n-\t\tbadsha=1\n-\telse\n-\t\tsha1_verif=\"$(git rev-parse --verify --quiet $1^{commit})\"\n-\t\tif test -z \"$sha1_verif\"\n-\t\tthen\n-\t\t\tbadsha=1\n-\t\tfi\n-\tfi\n-\n-\tif test $badsha -ne 0\n-\tthen\n-\t\tline=\"$(sed -n -e \"${2}p\" \"$3\")\"\n-\t\twarn \"$(eval_gettext \"\\\n-Warning: the SHA-1 is missing or isn't a commit in the following line:\n- - \\$line\")\"\n-\t\twarn\n-\tfi\n-\n-\treturn $badsha\n-}\n-\n-# prints the bad commits and bad commands\n-# from the todolist in stdin\n-check_bad_cmd_and_sha () {\n-\tretval=0\n-\tlineno=0\n-\twhile read -r command rest\n-\tdo\n-\t\tlineno=$(( $lineno + 1 ))\n-\t\tcase $command in\n-\t\t\"$comment_char\"*|''|noop|x|exec)\n-\t\t\t# Doesn't expect a SHA-1\n-\t\t\t;;\n-\t\t\"$cr\")\n-\t\t\t# Work around CR left by \"read\" (e.g. with Git for\n-\t\t\t# Windows' Bash).\n-\t\t\t;;\n-\t\tpick|p|drop|d|reword|r|edit|e|squash|s|fixup|f)\n-\t\t\tif ! check_commit_sha \"${rest%%[ \t]*}\" \"$lineno\" \"$1\"\n-\t\t\tthen\n-\t\t\t\tretval=1\n-\t\t\tfi\n-\t\t\t;;\n-\t\t*)\n-\t\t\tline=\"$(sed -n -e \"${lineno}p\" \"$1\")\"\n-\t\t\twarn \"$(eval_gettext \"\\\n-Warning: the command isn't recognized in the following line:\n- - \\$line\")\"\n-\t\t\twarn\n-\t\t\tretval=1\n-\t\t\t;;\n-\t\tesac\n-\tdone <\"$1\"\n-\treturn $retval\n-}\n-\n-# Print the list of the SHA-1 of the commits\n-# from stdin to stdout\n-todo_list_to_sha_list () {\n-\tgit stripspace --strip-comments |\n-\twhile read -r command sha1 rest\n-\tdo\n-\t\tcase $command in\n-\t\t\"$comment_char\"*|''|noop|x|\"exec\")\n-\t\t\t;;\n-\t\t*)\n-\t\t\tlong_sha=$(git rev-list --no-walk \"$sha1\" 2>/dev/null)\n-\t\t\tprintf \"%s\\n\" \"$long_sha\"\n-\t\t\t;;\n-\t\tesac\n-\tdone\n-}\n-\n-# Use warn for each line in stdin\n-warn_lines () {\n-\twhile read -r line\n-\tdo\n-\t\twarn \" - $line\"\n-\tdone\n-}\n-\n # Switch to the branch in $into and notify it in the reflog\n checkout_onto () {\n \tGIT_REFLOG_ACTION=\"$GIT_REFLOG_ACTION: checkout $onto_name\"\n@@ -994,74 +904,6 @@ get_missing_commit_check_level () {\n \tprintf '%s' \"$check_level\" | tr 'A-Z' 'a-z'\n }\n \n-# Check if the user dropped some commits by mistake\n-# Behaviour determined by rebase.missingCommitsCheck.\n-# Check if there is an unrecognized command or a\n-# bad SHA-1 in a command.\n-check_todo_list () {\n-\traise_error=f\n-\n-\tcheck_level=$(get_missing_commit_check_level)\n-\n-\tcase \"$check_level\" in\n-\twarn|error)\n-\t\t# Get the SHA-1 of the commits\n-\t\ttodo_list_to_sha_list <\"$todo\".backup >\"$todo\".oldsha1\n-\t\ttodo_list_to_sha_list <\"$todo\" >\"$todo\".newsha1\n-\n-\t\t# Sort the SHA-1 and compare them\n-\t\tsort -u \"$todo\".oldsha1 >\"$todo\".oldsha1+\n-\t\tmv \"$todo\".oldsha1+ \"$todo\".oldsha1\n-\t\tsort -u \"$todo\".newsha1 >\"$todo\".newsha1+\n-\t\tmv \"$todo\".newsha1+ \"$todo\".newsha1\n-\t\tcomm -2 -3 \"$todo\".oldsha1 \"$todo\".newsha1 >\"$todo\".miss\n-\n-\t\t# Warn about missing commits\n-\t\tif test -s \"$todo\".miss\n-\t\tthen\n-\t\t\ttest \"$check_level\" = error && raise_error=t\n-\n-\t\t\twarn \"$(gettext \"\\\n-Warning: some commits may have been dropped accidentally.\n-Dropped commits (newer to older):\")\"\n-\n-\t\t\t# Make the list user-friendly and display\n-\t\t\topt=\"--no-walk=sorted --format=oneline --abbrev-commit --stdin\"\n-\t\t\tgit rev-list $opt <\"$todo\".miss | warn_lines\n-\n-\t\t\twarn \"$(gettext \"\\\n-To avoid this message, use \\\"drop\\\" to explicitly remove a commit.\n-\n-Use 'git config rebase.missingCommitsCheck' to change the level of warnings.\n-The possible behaviours are: ignore, warn, error.\")\"\n-\t\t\twarn\n-\t\tfi\n-\t\t;;\n-\tignore)\n-\t\t;;\n-\t*)\n-\t\twarn \"$(eval_gettext \"Unrecognized setting \\$check_level for option rebase.missingCommitsCheck. Ignoring.\")\"\n-\t\t;;\n-\tesac\n-\n-\tif ! check_bad_cmd_and_sha \"$todo\"\n-\tthen\n-\t\traise_error=t\n-\tfi\n-\n-\tif test $raise_error = t\n-\tthen\n-\t\t# Checkout before the first commit of the\n-\t\t# rebase: this way git rebase --continue\n-\t\t# will work correctly as it expects HEAD to be\n-\t\t# placed before the commit of the next action\n-\t\tcheckout_onto\n-\n-\t\twarn \"$(gettext \"You can fix this with 'git rebase --edit-todo' and then run 'git rebase --continue'.\")\"\n-\t\tdie \"$(gettext \"Or you can abort the rebase with 'git rebase --abort'.\")\"\n-\tfi\n-}\n-\n # The whole contents of this file is run by dot-sourcing it from\n # inside a shell function. It used to be that \"return\"s we see\n # below were not inside any function, and expected to return\n@@ -1322,7 +1164,11 @@ git_sequence_editor \"$todo\" ||\n has_action \"$todo\" ||\n \treturn 2\n \n-check_todo_list\n+git rebase--helper --check-todo-list || {\n+\tret=$?\n+\tcheckout_onto\n+\texit $ret\n+}\n \n expand_todo_ids\n \ndiff --git a/sequencer.c b/sequencer.c\nindex 4b7f88b338f..fb3915ee39e 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2492,3 +2492,128 @@ int transform_todo_ids(int shorten_sha1s)\n \ttodo_list_release(&todo_list);\n \treturn 0;\n }\n+\n+enum check_level {\n+\tCHECK_IGNORE = 0, CHECK_WARN, CHECK_ERROR\n+};\n+\n+static enum check_level get_missing_commit_check_level(void)\n+{\n+\tconst char *value;\n+\n+\tif (git_config_get_value(\"rebase.missingcommitscheck\", &value) ||\n+\t\t\t!strcasecmp(\"ignore\", value))\n+\t\treturn CHECK_IGNORE;\n+\tif (!strcasecmp(\"warn\", value))\n+\t\treturn CHECK_WARN;\n+\tif (!strcasecmp(\"error\", value))\n+\t\treturn CHECK_ERROR;\n+\twarning(_(\"unrecognized setting %s for option\"\n+\t\t \"rebase.missingCommitsCheck. Ignoring.\"), value);\n+\treturn CHECK_IGNORE;\n+}\n+\n+/*\n+ * Check if the user dropped some commits by mistake\n+ * Behaviour determined by rebase.missingCommitsCheck.\n+ * Check if there is an unrecognized command or a\n+ * bad SHA-1 in a command.\n+ */\n+int check_todo_list(void)\n+{\n+\tenum check_level check_level = get_missing_commit_check_level();\n+\tstruct strbuf todo_file = STRBUF_INIT;\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct commit_list *missing = NULL;\n+\tint raise_error = 0, res = 0, fd, i;\n+\n+\tstrbuf_addstr(&todo_file, rebase_path_todo());\n+\tfd = open(todo_file.buf, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tclose(fd);\n+\traise_error = res =\n+\t\tparse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\n+\tif (check_level == CHECK_IGNORE)\n+\t\tgoto leave_check;\n+\n+\t/* Get the SHA-1 of the commits */\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct commit *commit = todo_list.items[i].commit;\n+\t\tif (commit)\n+\t\t\tcommit->util = todo_list.items + i;\n+\t}\n+\n+\ttodo_list_release(&todo_list);\n+\tstrbuf_addstr(&todo_file, \".backup\");\n+\tfd = open(todo_file.buf, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tclose(fd);\n+\tstrbuf_release(&todo_file);\n+\tres = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\n+\t/* Find commits that are missing after editing */\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct commit *commit = todo_list.items[i].commit;\n+\t\tif (commit && !commit->util) {\n+\t\t\tcommit_list_insert(commit, &missing);\n+\t\t\tcommit->util = todo_list.items + i;\n+\t\t}\n+\t}\n+\n+\t/* Warn about missing commits */\n+\tif (!missing)\n+\t\tgoto leave_check;\n+\n+\tif (check_level == CHECK_ERROR)\n+\t\traise_error = res = 1;\n+\n+\tfprintf(stderr,\n+\t\t_(\"Warning: some commits may have been dropped accidentally.\\n\"\n+\t\t\"Dropped commits (newer to older):\\n\"));\n+\n+\t/* Make the list user-friendly and display */\n+\twhile (missing) {\n+\t\tstruct commit *commit = pop_commit(&missing);\n+\t\tstruct todo_item *item = commit->util;\n+\n+\t\tfprintf(stderr, \" - %s %.*s\\n\", short_commit_name(commit),\n+\t\t\titem->arg_len, item->arg);\n+\t}\n+\tfree_commit_list(missing);\n+\n+\tfprintf(stderr, _(\"To avoid this message, use \\\"drop\\\" to \"\n+\t\t\"explicitly remove a commit.\\n\\n\"\n+\t\t\"Use 'git config rebase.missingCommitsCheck' to change \"\n+\t\t\"the level of warnings.\\n\"\n+\t\t\"The possible behaviours are: ignore, warn, error.\\n\\n\"));\n+\n+leave_check:\n+\tstrbuf_release(&todo_file);\n+\ttodo_list_release(&todo_list);\n+\n+\tif (raise_error)\n+\t\tfprintf(stderr,\n+\t\t\t_(\"You can fix this with 'git rebase --edit-todo' \"\n+\t\t\t \"and then run 'git rebase --continue'.\\n\"\n+\t\t\t \"Or you can abort the rebase with 'git rebase\"\n+\t\t\t \" --abort'.\\n\"));\n+\n+\treturn res;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 47a81034e76..4978a61b83b 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -49,6 +49,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv);\n \n int transform_todo_ids(int shorten_sha1s);\n+int check_todo_list(void);\n \n extern const char sign_off_header[];\n \n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317926","messageId":"1a5e609efb3b464e6350927df90c8d7d30d6b97b.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 7/9] rebase -i: skip unnecessary picks using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T12:00:21Z","receivedAt":"2017-04-26T12:00:43Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"In particular on Windows, where shell scripts are even more expensive\nthan on MacOSX or Linux, it makes sense to move a loop that forks\nGit at least once for every line in the todo list into a builtin.\n\nNote: The original code did not try to skip unnecessary picks of root\ncommits but punts instead (probably --root was not considered common\nenough of a use case to bother optimizing). We do the same, for now.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 6 ++-\n git-rebase--interactive.sh | 41 ++---------------\n sequencer.c | 107 +++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n 4 files changed, 116 insertions(+), 39 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex e706eac710d..de3ccd9bfbc 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tint keep_empty = 0;\n \tenum {\n \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n-\t\tCHECK_TODO_LIST\n+\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -31,6 +31,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n \t\tOPT_CMDMODE(0, \"check-todo-list\", &command,\n \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n+\t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n+\t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n \t\tOPT_END()\n \t};\n \n@@ -55,5 +57,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!transform_todo_ids(0);\n \tif (command == CHECK_TODO_LIST && argc == 1)\n \t\treturn !!check_todo_list();\n+\tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n+\t\treturn !!skip_unnecessary_picks();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 1649506e1e4..931bc09e0cf 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -713,43 +713,6 @@ do_rest () {\n \tdone\n }\n \n-# skip picking commits whose parents are unchanged\n-skip_unnecessary_picks () {\n-\tfd=3\n-\twhile read -r command rest\n-\tdo\n-\t\t# fd=3 means we skip the command\n-\t\tcase \"$fd,$command\" in\n-\t\t3,pick|3,p)\n-\t\t\t# pick a commit whose parent is current $onto -> skip\n-\t\t\tsha1=${rest%% *}\n-\t\t\tcase \"$(git rev-parse --verify --quiet \"$sha1\"^)\" in\n-\t\t\t\"$onto\"*)\n-\t\t\t\tonto=$sha1\n-\t\t\t\t;;\n-\t\t\t*)\n-\t\t\t\tfd=1\n-\t\t\t\t;;\n-\t\t\tesac\n-\t\t\t;;\n-\t\t3,\"$comment_char\"*|3,)\n-\t\t\t# copy comments\n-\t\t\t;;\n-\t\t*)\n-\t\t\tfd=1\n-\t\t\t;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$command${rest:+ }$rest\" >&$fd\n-\tdone <\"$todo\" >\"$todo.new\" 3>>\"$done\" &&\n-\tmv -f \"$todo\".new \"$todo\" &&\n-\tcase \"$(peek_next_command)\" in\n-\tsquash|s|fixup|f)\n-\t\trecord_in_rewritten \"$onto\"\n-\t\t;;\n-\tesac ||\n-\t\tdie \"$(gettext \"Could not skip unnecessary pick commands\")\"\n-}\n-\n transform_todo_ids () {\n \twhile read -r command rest\n \tdo\n@@ -1172,7 +1135,9 @@ git rebase--helper --check-todo-list || {\n \n expand_todo_ids\n \n-test -d \"$rewritten\" || test -n \"$force_rebase\" || skip_unnecessary_picks\n+test -d \"$rewritten\" || test -n \"$force_rebase\" ||\n+onto=\"$(git rebase--helper --skip-unnecessary-picks)\" ||\n+die \"Could not skip unnecessary pick commands\"\n \n checkout_onto\n if test -z \"$rebase_root\" && test ! -d \"$rewritten\"\ndiff --git a/sequencer.c b/sequencer.c\nindex fb3915ee39e..b51faa0120f 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2617,3 +2617,110 @@ int check_todo_list(void)\n \n \treturn res;\n }\n+\n+/* skip picking commits whose parents are unchanged */\n+int skip_unnecessary_picks(void)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct object_id onto_oid, *oid = &onto_oid, *parent_oid;\n+\tint fd, i;\n+\n+\tif (!read_oneliner(&buf, rebase_path_onto(), 0))\n+\t\treturn error(_(\"could not read 'onto'\"));\n+\tif (get_sha1(buf.buf, onto_oid.hash)) {\n+\t\tstrbuf_release(&buf);\n+\t\treturn error(_(\"need a HEAD to fixup\"));\n+\t}\n+\tstrbuf_release(&buf);\n+\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0) {\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn -1;\n+\t}\n+\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\n+\t\tif (item->command >= TODO_NOOP)\n+\t\t\tcontinue;\n+\t\tif (item->command != TODO_PICK)\n+\t\t\tbreak;\n+\t\tif (parse_commit(item->commit)) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error(_(\"could not parse commit '%s'\"),\n+\t\t\t\toid_to_hex(&item->commit->object.oid));\n+\t\t}\n+\t\tif (!item->commit->parents)\n+\t\t\tbreak; /* root commit */\n+\t\tif (item->commit->parents->next)\n+\t\t\tbreak; /* merge commit */\n+\t\tparent_oid = &item->commit->parents->item->object.oid;\n+\t\tif (hashcmp(parent_oid->hash, oid->hash))\n+\t\t\tbreak;\n+\t\toid = &item->commit->object.oid;\n+\t}\n+\tif (i > 0) {\n+\t\tint offset = i < todo_list.nr ?\n+\t\t\ttodo_list.items[i].offset_in_buf : todo_list.buf.len;\n+\t\tconst char *done_path = rebase_path_done();\n+\n+\t\tfd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n+\t\tif (fd < 0) {\n+\t\t\terror_errno(_(\"could not open '%s' for writing\"),\n+\t\t\t\t done_path);\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tif (write_in_full(fd, todo_list.buf.buf, offset) < 0) {\n+\t\t\terror_errno(_(\"could not write to '%s'\"), done_path);\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\tclose(fd);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tclose(fd);\n+\n+\t\tfd = open(rebase_path_todo(), O_WRONLY, 0666);\n+\t\tif (fd < 0) {\n+\t\t\terror_errno(_(\"could not open '%s' for writing\"),\n+\t\t\t\t rebase_path_todo());\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n+\t\t\t\ttodo_list.buf.len - offset) < 0) {\n+\t\t\terror_errno(_(\"could not write to '%s'\"),\n+\t\t\t\t rebase_path_todo());\n+\t\t\tclose(fd);\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tif (ftruncate(fd, todo_list.buf.len - offset) < 0) {\n+\t\t\terror_errno(_(\"could not truncate '%s'\"),\n+\t\t\t\t rebase_path_todo());\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\tclose(fd);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tclose(fd);\n+\n+\t\ttodo_list.current = i;\n+\t\tif (is_fixup(peek_command(&todo_list, 0)))\n+\t\t\trecord_in_rewritten(oid, peek_command(&todo_list, 0));\n+\t}\n+\n+\ttodo_list_release(&todo_list);\n+\tprintf(\"%s\\n\", oid_to_hex(oid));\n+\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 4978a61b83b..28e1fc1e9bb 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -50,6 +50,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \n int transform_todo_ids(int shorten_sha1s);\n int check_todo_list(void);\n+int skip_unnecessary_picks(void);\n \n extern const char sign_off_header[];\n \n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317927","messageId":"6e4c116952e0e86484a5a0264071e64f0e5b4d94.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 8/9] t3415: test fixup with wrapped oneline","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T12:00:24Z","receivedAt":"2017-04-26T12:00:45Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The `git commit --fixup` command unwraps wrapped onelines when\nconstructing the commit message, without wrapping the result.\n\nWe need to make sure that `git rebase --autosquash` keeps handling such\ncases correctly, in particular since we are about to move the autosquash\nhandling into the rebase--helper.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3415-rebase-autosquash.sh | 14 ++++++++++++++\n 1 file changed, 14 insertions(+)\n\ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 48346f1cc0c..9fd629a6e21 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -304,4 +304,18 @@ test_expect_success 'extra spaces after fixup!' '\n \ttest $base = $parent\n '\n \n+test_expect_success 'wrapped original subject' '\n+\tif test -d .git/rebase-merge; then git rebase --abort; fi &&\n+\tbase=$(git rev-parse HEAD) &&\n+\techo \"wrapped subject\" >wrapped &&\n+\tgit add wrapped &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m \"$(printf \"To\\nfixup\")\" &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m \"fixup! To fixup\" &&\n+\tgit rebase -i --autosquash --keep-empty HEAD~2 &&\n+\tparent=$(git rev-parse HEAD^) &&\n+\ttest $base = $parent\n+'\n+\n test_done\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n\n\n"},{"id":"317928","messageId":"1d3d10b9e150dad86bbb9eec6b7baf2a03256d5e.1493207864.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v3 9/9] rebase -i: rearrange fixup/squash lines using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-26T12:00:29Z","receivedAt":"2017-04-26T12:01:09Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This operation has quadratic complexity, which is especially painful\non Windows, where shell scripts are *already* slow (mainly due to the\noverhead of the POSIX emulation layer).\n\nLet's reimplement this with linear complexity (using a hash map to\nmatch the commits' subject lines) for the common case; Sadly, the\nfixup/squash feature's design neglected performance considerations,\nallowing arbitrary prefixes (read: `fixup! hell` will match the\ncommit subject `hello world`), which means that we are stuck with\nquadratic performance in the worst case.\n\nThe reimplemented logic also happens to fix a bug where commented-out\nlines (representing empty patches) were dropped by the previous code.\n\nWhile at it, clarify how the fixup/squash feature works in `git rebase\n-i`'s man page.\n\nSigned-off-by: Johannes Schindelin \n---\n Documentation/git-rebase.txt | 16 ++--\n builtin/rebase--helper.c | 6 +-\n git-rebase--interactive.sh | 90 +-------------------\n sequencer.c | 195 +++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n t/t3415-rebase-autosquash.sh | 2 +-\n 6 files changed, 212 insertions(+), 98 deletions(-)\n\ndiff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt\nindex 67d48e68831..da79fbda5b3 100644\n--- a/Documentation/git-rebase.txt\n+++ b/Documentation/git-rebase.txt\n@@ -425,13 +425,15 @@ without an explicit `--interactive`.\n --autosquash::\n --no-autosquash::\n \tWhen the commit log message begins with \"squash! ...\" (or\n-\t\"fixup! ...\"), and there is a commit whose title begins with\n-\tthe same ..., automatically modify the todo list of rebase -i\n-\tso that the commit marked for squashing comes right after the\n-\tcommit to be modified, and change the action of the moved\n-\tcommit from `pick` to `squash` (or `fixup`). Ignores subsequent\n-\t\"fixup! \" or \"squash! \" after the first, in case you referred to an\n-\tearlier fixup/squash with `git commit --fixup/--squash`.\n+\t\"fixup! ...\"), and there is already a commit in the todo list that\n+\tmatches the same `...`, automatically modify the todo list of rebase\n+\t-i so that the commit marked for squashing comes right after the\n+\tcommit to be modified, and change the action of the moved commit\n+\tfrom `pick` to `squash` (or `fixup`). A commit matches the `...` if\n+\tthe commit subject matches, or if the `...` refers to the commit's\n+\thash. As a fall-back, partial matches of the commit subject work,\n+\ttoo. The recommended way to create fixup/squash commits is by using\n+\tthe `--fixup`/`--squash` options of linkgit:git-commit[1].\n +\n This option is only valid when the `--interactive` option is used.\n +\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex de3ccd9bfbc..e6591f01112 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tint keep_empty = 0;\n \tenum {\n \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n-\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n+\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -33,6 +33,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n \t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n \t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n+\t\tOPT_CMDMODE(0, \"rearrange-squash\", &command,\n+\t\t\tN_(\"rearrange fixup/squash lines\"), REARRANGE_SQUASH),\n \t\tOPT_END()\n \t};\n \n@@ -59,5 +61,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!check_todo_list();\n \tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n \t\treturn !!skip_unnecessary_picks();\n+\tif (command == REARRANGE_SQUASH && argc == 1)\n+\t\treturn !!rearrange_squash();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 931bc09e0cf..d39fe4f5fb7 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -744,94 +744,6 @@ collapse_todo_ids() {\n \tgit rebase--helper --shorten-sha1s\n }\n \n-# Rearrange the todo list that has both \"pick sha1 msg\" and\n-# \"pick sha1 fixup!/squash! msg\" appears in it so that the latter\n-# comes immediately after the former, and change \"pick\" to\n-# \"fixup\"/\"squash\".\n-#\n-# Note that if the config has specified a custom instruction format\n-# each log message will be re-retrieved in order to normalize the\n-# autosquash arrangement\n-rearrange_squash () {\n-\tformat=$(git config --get rebase.instructionFormat)\n-\t# extract fixup!/squash! lines and resolve any referenced sha1's\n-\twhile read -r pick sha1 message\n-\tdo\n-\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n-\t\tcase \"$message\" in\n-\t\t\"squash! \"*|\"fixup! \"*)\n-\t\t\taction=\"${message%%!*}\"\n-\t\t\trest=$message\n-\t\t\tprefix=\n-\t\t\t# skip all squash! or fixup! (but save for later)\n-\t\t\twhile :\n-\t\t\tdo\n-\t\t\t\tcase \"$rest\" in\n-\t\t\t\t\"squash! \"*|\"fixup! \"*)\n-\t\t\t\t\tprefix=\"$prefix${rest%%!*},\"\n-\t\t\t\t\trest=\"${rest#*! }\"\n-\t\t\t\t\t;;\n-\t\t\t\t*)\n-\t\t\t\t\tbreak\n-\t\t\t\t\t;;\n-\t\t\t\tesac\n-\t\t\tdone\n-\t\t\tprintf '%s %s %s %s\\n' \"$sha1\" \"$action\" \"$prefix\" \"$rest\"\n-\t\t\t# if it's a single word, try to resolve to a full sha1 and\n-\t\t\t# emit a second copy. This allows us to match on both message\n-\t\t\t# and on sha1 prefix\n-\t\t\tif test \"${rest#* }\" = \"$rest\"; then\n-\t\t\t\tfullsha=\"$(git rev-parse -q --verify \"$rest\" 2>/dev/null)\"\n-\t\t\t\tif test -n \"$fullsha\"; then\n-\t\t\t\t\t# prefix the action to uniquely identify this line as\n-\t\t\t\t\t# intended for full sha1 match\n-\t\t\t\t\techo \"$sha1 +$action $prefix $fullsha\"\n-\t\t\t\tfi\n-\t\t\tfi\n-\t\tesac\n-\tdone >\"$1.sq\" <\"$1\"\n-\ttest -s \"$1.sq\" || return\n-\n-\tused=\n-\twhile read -r pick sha1 message\n-\tdo\n-\t\tcase \" $used\" in\n-\t\t*\" $sha1 \"*) continue ;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$pick $sha1 $message\"\n-\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n-\t\tused=\"$used$sha1 \"\n-\t\twhile read -r squash action msg_prefix msg_content\n-\t\tdo\n-\t\t\tcase \" $used\" in\n-\t\t\t*\" $squash \"*) continue ;;\n-\t\t\tesac\n-\t\t\temit=0\n-\t\t\tcase \"$action\" in\n-\t\t\t+*)\n-\t\t\t\taction=\"${action#+}\"\n-\t\t\t\t# full sha1 prefix test\n-\t\t\t\tcase \"$msg_content\" in \"$sha1\"*) emit=1;; esac ;;\n-\t\t\t*)\n-\t\t\t\t# message prefix test\n-\t\t\t\tcase \"$message\" in \"$msg_content\"*) emit=1;; esac ;;\n-\t\t\tesac\n-\t\t\tif test $emit = 1; then\n-\t\t\t\tif test -n \"${format}\"\n-\t\t\t\tthen\n-\t\t\t\t\tmsg_content=$(git log -n 1 --format=\"${format}\" ${squash})\n-\t\t\t\telse\n-\t\t\t\t\tmsg_content=\"$(echo \"$msg_prefix\" | sed \"s/,/! /g\")$msg_content\"\n-\t\t\t\tfi\n-\t\t\t\tprintf '%s\\n' \"$action $squash $msg_content\"\n-\t\t\t\tused=\"$used$squash \"\n-\t\t\tfi\n-\t\tdone <\"$1.sq\"\n-\tdone >\"$1.rearranged\" <\"$1\"\n-\tcat \"$1.rearranged\" >\"$1\"\n-\trm -f \"$1.sq\" \"$1.rearranged\"\n-}\n-\n # Add commands after a pick or after a squash/fixup serie\n # in the todo list.\n add_exec_commands () {\n@@ -1091,7 +1003,7 @@ then\n fi\n \n test -s \"$todo\" || echo noop >> \"$todo\"\n-test -n \"$autosquash\" && rearrange_squash \"$todo\"\n+test -z \"$autosquash\" || git rebase--helper --rearrange-squash || exit\n test -n \"$cmd\" && add_exec_commands \"$todo\"\n \n todocount=$(git stripspace --strip-comments <\"$todo\" | wc -l)\ndiff --git a/sequencer.c b/sequencer.c\nindex b51faa0120f..7ac1792311e 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -19,6 +19,7 @@\n #include \"trailer.h\"\n #include \"log-tree.h\"\n #include \"wt-status.h\"\n+#include \"hashmap.h\"\n \n #define GIT_REFLOG_ACTION \"GIT_REFLOG_ACTION\"\n \n@@ -2724,3 +2725,197 @@ int skip_unnecessary_picks(void)\n \n \treturn 0;\n }\n+\n+struct subject2item_entry {\n+\tstruct hashmap_entry entry;\n+\tint i;\n+\tchar subject[FLEX_ARRAY];\n+};\n+\n+static int subject2item_cmp(const struct subject2item_entry *a,\n+\tconst struct subject2item_entry *b, const void *key)\n+{\n+\treturn key ? strcmp(a->subject, key) : strcmp(a->subject, b->subject);\n+}\n+\n+/*\n+ * Rearrange the todo list that has both \"pick sha1 msg\" and \"pick sha1\n+ * fixup!/squash! msg\" in it so that the latter is put immediately after the\n+ * former, and change \"pick\" to \"fixup\"/\"squash\".\n+ *\n+ * Note that if the config has specified a custom instruction format, each log\n+ * message will have to be retrieved from the commit (as the oneline in the\n+ * script cannot be trusted) in order to normalize the autosquash arrangement.\n+ */\n+int rearrange_squash(void)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct hashmap subject2item;\n+\tint res = 0, rearranged = 0, *next, *tail, fd, i;\n+\tchar **subjects;\n+\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0)\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn -1;\n+\t}\n+\n+\t/*\n+\t * The hashmap maps onelines to the respective todo list index.\n+\t *\n+\t * If any items need to be rearranged, the next[i] value will indicate\n+\t * which item was moved directly after the i'th.\n+\t *\n+\t * In that case, last[i] will indicate the index of the latest item to\n+\t * be moved to appear after the i'th.\n+\t */\n+\thashmap_init(&subject2item, (hashmap_cmp_fn) subject2item_cmp,\n+\t\t todo_list.nr);\n+\tALLOC_ARRAY(next, todo_list.nr);\n+\tALLOC_ARRAY(tail, todo_list.nr);\n+\tALLOC_ARRAY(subjects, todo_list.nr);\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tconst char *commit_buffer, *subject, *p;\n+\t\tsize_t subject_len;\n+\t\tint i2 = -1;\n+\t\tstruct subject2item_entry *entry;\n+\n+\t\tnext[i] = tail[i] = -1;\n+\t\tif (item->command >= TODO_EXEC) {\n+\t\t\tsubjects[i] = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (is_fixup(item->command)) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error(_(\"the script was already rearranged.\"));\n+\t\t}\n+\n+\t\titem->commit->util = item;\n+\n+\t\tparse_commit(item->commit);\n+\t\tcommit_buffer = get_commit_buffer(item->commit, NULL);\n+\t\tfind_commit_subject(commit_buffer, &subject);\n+\t\tformat_subject(&buf, subject, \" \");\n+\t\tsubject = subjects[i] = strbuf_detach(&buf, &subject_len);\n+\t\tunuse_commit_buffer(item->commit, commit_buffer);\n+\t\tif ((skip_prefix(subject, \"fixup! \", &p) ||\n+\t\t skip_prefix(subject, \"squash! \", &p))) {\n+\t\t\tstruct commit *commit2;\n+\n+\t\t\tfor (;;) {\n+\t\t\t\twhile (isspace(*p))\n+\t\t\t\t\tp++;\n+\t\t\t\tif (!skip_prefix(p, \"fixup! \", &p) &&\n+\t\t\t\t !skip_prefix(p, \"squash! \", &p))\n+\t\t\t\t\tbreak;\n+\t\t\t}\n+\n+\t\t\tif ((entry = hashmap_get_from_hash(&subject2item,\n+\t\t\t\t\t\t\t strhash(p), p)))\n+\t\t\t\t/* found by title */\n+\t\t\t\ti2 = entry->i;\n+\t\t\telse if (!strchr(p, ' ') &&\n+\t\t\t\t (commit2 =\n+\t\t\t\t lookup_commit_reference_by_name(p)) &&\n+\t\t\t\t commit2->util)\n+\t\t\t\t/* found by commit name */\n+\t\t\t\ti2 = (struct todo_item *)commit2->util\n+\t\t\t\t\t- todo_list.items;\n+\t\t\telse {\n+\t\t\t\t/* copy can be a prefix of the commit subject */\n+\t\t\t\tfor (i2 = 0; i2 < i; i2++)\n+\t\t\t\t\tif (subjects[i2] &&\n+\t\t\t\t\t starts_with(subjects[i2], p))\n+\t\t\t\t\t\tbreak;\n+\t\t\t\tif (i2 == i)\n+\t\t\t\t\ti2 = -1;\n+\t\t\t}\n+\t\t}\n+\t\tif (i2 >= 0) {\n+\t\t\trearranged = 1;\n+\t\t\ttodo_list.items[i].command =\n+\t\t\t\tstarts_with(subject, \"fixup!\") ?\n+\t\t\t\tTODO_FIXUP : TODO_SQUASH;\n+\t\t\tif (next[i2] < 0)\n+\t\t\t\tnext[i2] = i;\n+\t\t\telse\n+\t\t\t\tnext[tail[i2]] = i;\n+\t\t\ttail[i2] = i;\n+\t\t} else if (!hashmap_get_from_hash(&subject2item,\n+\t\t\t\t\t\tstrhash(subject), subject)) {\n+\t\t\tFLEX_ALLOC_MEM(entry, subject, subject, subject_len);\n+\t\t\tentry->i = i;\n+\t\t\thashmap_entry_init(entry, strhash(entry->subject));\n+\t\t\thashmap_put(&subject2item, entry);\n+\t\t}\n+\t}\n+\n+\tif (rearranged) {\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\n+\t\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\t\tenum todo_command command = todo_list.items[i].command;\n+\t\t\tint cur = i;\n+\n+\t\t\t/*\n+\t\t\t * Initially, all commands are 'pick's. If it is a\n+\t\t\t * fixup or a squash now, we have rearranged it.\n+\t\t\t */\n+\t\t\tif (is_fixup(command))\n+\t\t\t\tcontinue;\n+\n+\t\t\twhile (cur >= 0) {\n+\t\t\t\tint offset = todo_list.items[cur].offset_in_buf;\n+\t\t\t\tint end_offset = cur + 1 < todo_list.nr ?\n+\t\t\t\t\ttodo_list.items[cur + 1].offset_in_buf :\n+\t\t\t\t\ttodo_list.buf.len;\n+\t\t\t\tchar *bol = todo_list.buf.buf + offset;\n+\t\t\t\tchar *eol = todo_list.buf.buf + end_offset;\n+\n+\t\t\t\t/* replace 'pick', by 'fixup' or 'squash' */\n+\t\t\t\tcommand = todo_list.items[cur].command;\n+\t\t\t\tif (is_fixup(command)) {\n+\t\t\t\t\tstrbuf_addstr(&buf,\n+\t\t\t\t\t\ttodo_command_info[command].str);\n+\t\t\t\t\tbol += strcspn(bol, \" \\t\");\n+\t\t\t\t}\n+\n+\t\t\t\tstrbuf_add(&buf, bol, eol - bol);\n+\n+\t\t\t\tcur = next[cur];\n+\t\t\t}\n+\t\t}\n+\n+\t\tfd = open(todo_file, O_WRONLY);\n+\t\tif (fd < 0)\n+\t\t\tres = error_errno(_(\"could not open '%s'\"), todo_file);\n+\t\telse if (write(fd, buf.buf, buf.len) < 0)\n+\t\t\tres = error_errno(_(\"could not read '%s'.\"), todo_file);\n+\t\telse if (ftruncate(fd, buf.len) < 0)\n+\t\t\tres = error_errno(_(\"could not finish '%s'\"),\n+\t\t\t\t\t todo_file);\n+\t\tclose(fd);\n+\t\tstrbuf_release(&buf);\n+\t}\n+\n+\tfree(next);\n+\tfree(tail);\n+\tfor (i = 0; i < todo_list.nr; i++)\n+\t\tfree(subjects[i]);\n+\tfree(subjects);\n+\thashmap_free(&subject2item, 1);\n+\ttodo_list_release(&todo_list);\n+\n+\treturn res;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 28e1fc1e9bb..1c94bec7622 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -51,6 +51,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n int transform_todo_ids(int shorten_sha1s);\n int check_todo_list(void);\n int skip_unnecessary_picks(void);\n+int rearrange_squash(void);\n \n extern const char sign_off_header[];\n \ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 9fd629a6e21..b9e26008a79 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -278,7 +278,7 @@ set_backup_editor () {\n \ttest_set_editor \"$PWD/backup-editor.sh\"\n }\n \n-test_expect_failure 'autosquash with multiple empty patches' '\n+test_expect_success 'autosquash with multiple empty patches' '\n \ttest_tick &&\n \tgit commit --allow-empty -m \"empty\" &&\n \ttest_tick &&\n-- \n2.12.2.windows.2.406.gd14a8f8640f\n"},{"id":"318036","messageId":"xmqqvapqo4i8.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"c44a15ed1f1015d7e9377e18610a0c428786995b.1493207864.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-27T04:31:43Z","receivedAt":"2017-04-27T04:31:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> diff --git a/sequencer.c b/sequencer.c\n> index 77afecaebf0..e858a976279 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -2388,3 +2388,48 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> \n> \tstrbuf_release(&sob);\n> }\n> +\n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv)\n> +{\n> +\tchar *format = xstrdup(\"%s\");\n> +\tstruct pretty_print_context pp = {0};\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tstruct rev_info revs;\n> +\tstruct commit *commit;\n> +\n> +\tinit_revisions(&revs, NULL);\n> +\trevs.verbose_header = 1;\n> +\trevs.max_parents = 1;\n> +\trevs.cherry_pick = 1;\n> +\trevs.limited = 1;\n> +\trevs.reverse = 1;\n> +\trevs.right_only = 1;\n> +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> +\trevs.topo_order = 1;\n> +\n> +\trevs.pretty_given = 1;\n> +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n> +\tget_commit_format(format, &revs);\n> +\tfree(format);\n> +\tpp.fmt = revs.commit_format;\n> +\tpp.output_encoding = get_log_output_encoding();\n\nAll of the above feels like inviting unnecessary future breakages by\nknowing too much about the implementation the current version of\nrevision.c happens to use. A more careful implementation would be\nto allocate our own av[] and prepare \"--reverse\", \"--left-right\",\n\"--cherry-pick\", etc. to be parsed by setup_revisions() call we see\nbelow. The parsing is not an expensive part of the operation\nanyway, and that way we do not have to worry about one less thing.\n\n> +\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n> +\t\treturn error(_(\"make_script: unhandled options\"));\n> +\n> +\tif (prepare_revision_walk(&revs) < 0)\n> +\t\treturn error(_(\"make_script: error preparing revisions\"));\n> +\n> +\twhile ((commit = get_revision(&revs))) {\n> +\t\tstrbuf_reset(&buf);\n> +\t\tif (!keep_empty && is_original_commit_empty(commit))\n> +\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n\nPresumably callers of this function (which does not exist yet at\nthis step) are expected to have done the configuration dance to\nprepare comment_line_char to whatever the end-user specified?\n\n> +\t\tstrbuf_addf(&buf, \"pick %s \", oid_to_hex(&commit->object.oid));\n> +\t\tpretty_print_commit(&pp, commit, &buf);\n> +\t\tstrbuf_addch(&buf, '\\n');\n> +\t\tfputs(buf.buf, out);\n> +\t}\n> +\tstrbuf_release(&buf);\n> +\treturn 0;\n> +}\n\nOther than that, this looks reasonable.\n\n> diff --git a/sequencer.h b/sequencer.h\n> index f885b68395f..83f2943b7a9 100644\n> --- a/sequencer.h\n> +++ b/sequencer.h\n> @@ -45,6 +45,9 @@ int sequencer_continue(struct replay_opts *opts);\n> int sequencer_rollback(struct replay_opts *opts);\n> int sequencer_remove_state(struct replay_opts *opts);\n> \n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv);\n> +\n> extern const char sign_off_header[];\n> \n> void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n"},{"id":"318039","messageId":"xmqqr30eo36n.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"df5263cc0cad5cffbd1932d4b9b8e5c0507536e6.1493207864.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v3 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-27T05:00:16Z","receivedAt":"2017-04-27T05:00:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n> index 214af0372ba..52a19e0bdb3 100644\n> --- a/git-rebase--interactive.sh\n> +++ b/git-rebase--interactive.sh\n> @@ -774,11 +774,11 @@ transform_todo_ids () {\n> }\n> \n> expand_todo_ids() {\n> -\ttransform_todo_ids\n> +\tgit rebase--helper --expand-sha1s\n> }\n> \n> collapse_todo_ids() {\n> -\ttransform_todo_ids --short\n> +\tgit rebase--helper --shorten-sha1s\n> }\n\nObviously correct ;-) But doesn't this make transform_todo_ids ()\nhelper unused and removable?\n\n> +int transform_todo_ids(int shorten_sha1s)\n> +{\n> +\tconst char *todo_file = rebase_path_todo();\n> +\tstruct todo_list todo_list = TODO_LIST_INIT;\n> +\tint fd, res, i;\n> +\tFILE *out;\n> +\n> +\tstrbuf_reset(&todo_list.buf);\n> +\tfd = open(todo_file, O_RDONLY);\n> +\tif (fd < 0)\n> +\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n> +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> +\t\tclose(fd);\n> +\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n> +\t}\n> +\tclose(fd);\n> +\n> +\tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n> +\tif (res) {\n> +\t\ttodo_list_release(&todo_list);\n> +\t\treturn error(_(\"unusable instruction sheet: '%s'\"), todo_file);\n> +\t}\n> +\n> +\tout = fopen(todo_file, \"w\");\n\nThe usual \"open lockfile, write to it and then rename\" dance is not\nnecessary for the purpose of preventing other people from reading\nthis file while we are writing to it. But if we fail inside this\nfunction before we fclose(3) \"out\", the user will lose the todo\nlist. It probably is not a big deal, though.\n\n> +\tif (!out) {\n> +\t\ttodo_list_release(&todo_list);\n> +\t\treturn error(_(\"unable to open '%s' for writing\"), todo_file);\n> +\t}\n> +\tfor (i = 0; i < todo_list.nr; i++) {\n> +\t\tstruct todo_item *item = todo_list.items + i;\n> +\t\tint bol = item->offset_in_buf;\n> +\t\tconst char *p = todo_list.buf.buf + bol;\n> +\t\tint eol = i + 1 < todo_list.nr ?\n> +\t\t\ttodo_list.items[i + 1].offset_in_buf :\n> +\t\t\ttodo_list.buf.len;\n> +\n> +\t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n> +\t\t\tfwrite(p, eol - bol, 1, out);\n> +\t\telse {\n> +\t\t\tint eoc = strcspn(p, \" \\t\");\n> +\t\t\tconst char *sha1 = shorten_sha1s ?\n> +\t\t\t\tshort_commit_name(item->commit) :\n> +\t\t\t\toid_to_hex(&item->commit->object.oid);\n> +\n> +\t\t\tif (!eoc) {\n> +\t\t\t\tp += strspn(p, \" \\t\");\n> +\t\t\t\teoc = strcspn(p, \" \\t\");\n> +\t\t\t}\n\nIt would be much easier to follow the logic if \"int eoc\" above were\na mere declaration without initialization and \"skip to the\nwhitespaces\" is done immediately before this if() statement. It's\nnot like the initialized value of eoc is needed there because it\nparticipates in the computation of sha1, and also having the\nassignment followed by \"oops, the line begins with a whitespace\"\nrecovery that is done here.\n\nWouldn't it be simpler to do:\n\n\telse {\n\t\tint eoc;\n\t\tconst char *sha1 = ...\n\t\tp += strspn(p, \" \\t\"); /* skip optional indent */\n\t\teoc = strcspn(p, \" \\t\"); /* grab the command word */\n\nwithout conditional?\n\n> +\t\t\tfprintf(out, \"%.*s %s %.*s\\n\",\n> +\t\t\t\teoc, p, sha1, item->arg_len, item->arg);\n> +\t\t}\n> +\t}\n> +\tfclose(out);\n> +\ttodo_list_release(&todo_list);\n> +\treturn 0;\n> +}\n> diff --git a/sequencer.h b/sequencer.h\n> index 83f2943b7a9..47a81034e76 100644\n> --- a/sequencer.h\n> +++ b/sequencer.h\n> @@ -48,6 +48,8 @@ int sequencer_remove_state(struct replay_opts *opts);\n> int sequencer_make_script(int keep_empty, FILE *out,\n> \t\tint argc, const char **argv);\n> \n> +int transform_todo_ids(int shorten_sha1s);\n> +\n> extern const char sign_off_header[];\n> \n> void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n"},{"id":"318041","messageId":"xmqqmvb2o2x7.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"975e7749c2fba9dc263cca1d04879c703c0b3c66.1493207864.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v3 5/9] t3404: relax rebase.missingCommitsCheck tests","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-27T05:05:56Z","receivedAt":"2017-04-27T05:06:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> These tests were a bit anal about the *exact* warning/error message\n> printed by git rebase. But those messages are intended for the *end\n> user*, therefore it does not make sense to test so rigidly for the\n> *exact* wording.\n>\n> In the following, we will reimplement the missing commits check in\n> the sequencer, with slightly different words.\n\nUp to this point I thought your strategy was to mimic the original\nas closely as possible, and changes to update (and/or improve) the\nend user experience (like rewording messages) are left as a separate\nstep. Changes to the test can be used to demonstrate the improved\nend-user experience that way, and I found it a good way to structure\nyour series. Is there a technical reason why you deviate from that\npattern here? \n\nJust being curious, as I do not particularly mind seeing things done\ndifferently (especially if there is a good reason).\n\n> -cat >expect < -Warning: the command isn't recognized in the following line:\n> - - badcmd $(git rev-list --oneline -1 master~1)\n> -\n> -You can fix this with 'git rebase --edit-todo' and then run 'git rebase --continue'.\n> -Or you can abort the rebase with 'git rebase --abort'.\n> -EOF\n> -\n> test_expect_success 'static check of bad command' '\n> \trebase_setup_and_clean bad-cmd &&\n> \tset_fake_editor &&\n> \ttest_must_fail env FAKE_LINES=\"1 2 3 bad 4 5\" \\\n> \t\tgit rebase -i --root 2>actual &&\n> -\ttest_i18ncmp expect actual &&\n> +\ttest_i18ngrep \"badcmd $(git rev-list --oneline -1 master~1)\" actual &&\n> +\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n> \tFAKE_LINES=\"1 2 3 drop 4 5\" git rebase --edit-todo &&\n> \tgit rebase --continue &&\n> \ttest E = $(git cat-file commit HEAD | sed -ne \\$p) &&\n> @@ -1277,20 +1270,13 @@ test_expect_success 'tabs and spaces are accepted in the todolist' '\n> \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n> '\n> \n> -cat >expect < -Warning: the SHA-1 is missing or isn't a commit in the following line:\n> - - edit XXXXXXX False commit\n> -\n> -You can fix this with 'git rebase --edit-todo' and then run 'git rebase --continue'.\n> -Or you can abort the rebase with 'git rebase --abort'.\n> -EOF\n> -\n> test_expect_success 'static check of bad SHA-1' '\n> \trebase_setup_and_clean bad-sha &&\n> \tset_fake_editor &&\n> \ttest_must_fail env FAKE_LINES=\"1 2 edit fakesha 3 4 5 #\" \\\n> \t\tgit rebase -i --root 2>actual &&\n> -\ttest_i18ncmp expect actual &&\n> +\ttest_i18ngrep \"edit XXXXXXX False commit\" actual &&\n> +\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n> \tFAKE_LINES=\"1 2 4 5 6\" git rebase --edit-todo &&\n> \tgit rebase --continue &&\n> \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n"},{"id":"318044","messageId":"xmqqh91ao1of.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"38bf320ec802386975843de6b5da4bbf823fb184.1493207864.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v3 6/9] rebase -i: check for missing commits in the rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-27T05:32:48Z","receivedAt":"2017-04-27T05:33:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> -check_todo_list\n> +git rebase--helper --check-todo-list || {\n> +\tret=$?\n> +\tcheckout_onto\n> +\texit $ret\n> +}\n\nI find this a better division of labor between \"check_todo_list\" and\nits caller. Compared to the original that did the \"recover and exit\nwith failure\" inside the helper, this is much easier to see what is\ngoing on.\n\n> +/*\n> + * Check if the user dropped some commits by mistake\n> + * Behaviour determined by rebase.missingCommitsCheck.\n> + * Check if there is an unrecognized command or a\n> + * bad SHA-1 in a command.\n> + */\n> +int check_todo_list(void)\n> +{\n> +\tenum check_level check_level = get_missing_commit_check_level();\n> +\tstruct strbuf todo_file = STRBUF_INIT;\n> +\tstruct todo_list todo_list = TODO_LIST_INIT;\n> +\tstruct commit_list *missing = NULL;\n> +\tint raise_error = 0, res = 0, fd, i;\n> +\n> +\tstrbuf_addstr(&todo_file, rebase_path_todo());\n> +\tfd = open(todo_file.buf, O_RDONLY);\n> +\tif (fd < 0) {\n> +\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n> +\t\tgoto leave_check;\n> +\t}\n> +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> +\t\tclose(fd);\n> +\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n> +\t\tgoto leave_check;\n> +\t}\n> +\tclose(fd);\n> +\traise_error = res =\n> +\t\tparse_insn_buffer(todo_list.buf.buf, &todo_list);\n> +\n> +\tif (check_level == CHECK_IGNORE)\n> +\t\tgoto leave_check;\n\nOK, so even it is set to ignore, unreadable todo list will be shown\nwith a loud error message that tells the user to use --edit-todo.\n\nWhat should happen when it is not set to ignore and we found the\ntodo list unacceptable, I wonder? Let's read on.\n\n> +\t/* Get the SHA-1 of the commits */\n> +\tfor (i = 0; i < todo_list.nr; i++) {\n> +\t\tstruct commit *commit = todo_list.items[i].commit;\n> +\t\tif (commit)\n> +\t\t\tcommit->util = todo_list.items + i;\n> +\t}\n\nIt does not look like this loop is \"Get(ting) the SHA-1 of the commits\"\nto me, though. It is setting up ->util to be usable as a back-pointer\ninto the list.\n\n> +\n> +\ttodo_list_release(&todo_list);\n\nBut then the todo-list is released? The util field we have set, if\nany, in the previous loop are now dangling, no?\n\n> +\tstrbuf_addstr(&todo_file, \".backup\");\n> +\tfd = open(todo_file.buf, O_RDONLY);\n> +\tif (fd < 0) {\n> +\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n> +\t\tgoto leave_check;\n> +\t}\n> +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> +\t\tclose(fd);\n> +\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n> +\t\tgoto leave_check;\n> +\t}\n> +\tclose(fd);\n> +\tstrbuf_release(&todo_file);\n> +\tres = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);\n\nThen we read from .backup; failure to do so does not result in the\n\"you need to --edit-todo\" warning.\n\n> +\t/* Find commits that are missing after editing */\n> +\tfor (i = 0; i < todo_list.nr; i++) {\n> +\t\tstruct commit *commit = todo_list.items[i].commit;\n> +\t\tif (commit && !commit->util) {\n> +\t\t\tcommit_list_insert(commit, &missing);\n> +\t\t\tcommit->util = todo_list.items + i;\n> +\t\t}\n> +\t}\n\nAnd we check the commits mentioned in the backup; any commit whose\nutil is not marked in the previous loop is noticed and thrown into\nthe missing list.\n\nThe loop we later have does \"while (missing)\" and does not look at\ncommit->util for commits that are *not* missing, i.e. the ones that\nare marked in the previous loop, so it does not matter that their\nutil field have dangling pointers. In that sense, it may not be\nbuggy, but it is misleading. The only thing these two loops care\nabout is that the commits found in the earlier loop get their util\nfield set to non-NULL, so instead of using \"todo_list.items+i\",\nperhaps doing this\n\n\tif (commit)\n\t\tcommit->util = (void *)1; /* mark as seen */\n\nin the earlier loop instead would be much less confusing.\n\n> +\t/* Warn about missing commits */\n> +\tif (!missing)\n> +\t\tgoto leave_check;\n\nIf there is no missing one, we may still return error about\nunacceptable backup file. But if we read backup fine and didn't\nfind anything missing, we'll return silently and with success. OK.\n\n> +\tif (check_level == CHECK_ERROR)\n> +\t\traise_error = res = 1;\n\nOtherwise, we found missing ones and we want to report here.\n\nThe reason why I started reading this function aloud was because I\nfound two variables (raise_error and res) somewhat confusing. I\nthink what the code does makes sense, but I still find the way how\nthe code expresses the logic with these two variables confusing.\nPerhaps somebody else can hopefully offer possible improvements, as\nI do not offhand think of a way better than what is currently in\nthis patch myself.\n\n> +\tfprintf(stderr,\n> +\t\t_(\"Warning: some commits may have been dropped accidentally.\\n\"\n> +\t\t\"Dropped commits (newer to older):\\n\"));\n> +\n> +\t/* Make the list user-friendly and display */\n> +\twhile (missing) {\n> +\t\tstruct commit *commit = pop_commit(&missing);\n> +\t\tstruct todo_item *item = commit->util;\n> +\n> +\t\tfprintf(stderr, \" - %s %.*s\\n\", short_commit_name(commit),\n> +\t\t\titem->arg_len, item->arg);\n> +\t}\n> +\tfree_commit_list(missing);\n> +\n> +\tfprintf(stderr, _(\"To avoid this message, use \\\"drop\\\" to \"\n> +\t\t\"explicitly remove a commit.\\n\\n\"\n> +\t\t\"Use 'git config rebase.missingCommitsCheck' to change \"\n> +\t\t\"the level of warnings.\\n\"\n> +\t\t\"The possible behaviours are: ignore, warn, error.\\n\\n\"));\n> +\n> +leave_check:\n> +\tstrbuf_release(&todo_file);\n> +\ttodo_list_release(&todo_list);\n> +\n> +\tif (raise_error)\n> +\t\tfprintf(stderr,\n> +\t\t\t_(\"You can fix this with 'git rebase --edit-todo' \"\n> +\t\t\t \"and then run 'git rebase --continue'.\\n\"\n> +\t\t\t \"Or you can abort the rebase with 'git rebase\"\n> +\t\t\t \" --abort'.\\n\"));\n> +\n> +\treturn res;\n> +}\n> diff --git a/sequencer.h b/sequencer.h\n> index 47a81034e76..4978a61b83b 100644\n> --- a/sequencer.h\n> +++ b/sequencer.h\n> @@ -49,6 +49,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n> \t\tint argc, const char **argv);\n> \n> int transform_todo_ids(int shorten_sha1s);\n> +int check_todo_list(void);\n> \n> extern const char sign_off_header[];\n"},{"id":"318058","messageId":"xmqqfugumjnp.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"xmqqr30eo36n.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-27T06:47:22Z","receivedAt":"2017-04-27T06:47:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Johannes Schindelin writes:\n>\n>> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n>> index 214af0372ba..52a19e0bdb3 100644\n>> --- a/git-rebase--interactive.sh\n>> +++ b/git-rebase--interactive.sh\n>> @@ -774,11 +774,11 @@ transform_todo_ids () {\n>> }\n>> \n>> expand_todo_ids() {\n>> -\ttransform_todo_ids\n>> +\tgit rebase--helper --expand-sha1s\n>> }\n>> \n>> collapse_todo_ids() {\n>> -\ttransform_todo_ids --short\n>> +\tgit rebase--helper --shorten-sha1s\n>> }\n>\n> Obviously correct ;-) But doesn't this make transform_todo_ids ()\n> helper unused and removable?\n\nEhh, in case it was notclear, I meant the helper function in shell,\nnot the one you added below to C code.\n>\n>> +int transform_todo_ids(int shorten_sha1s)\n>> +{\n\n"},{"id":"318069","messageId":"alpine.DEB.2.20.1704271607550.3480@virtualbox","threadId":"43983","inReplyTo":"xmqqvapqo4i8.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-27T14:18:44Z","receivedAt":"2017-04-27T14:19:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Wed, 26 Apr 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > diff --git a/sequencer.c b/sequencer.c\n> > index 77afecaebf0..e858a976279 100644\n> > --- a/sequencer.c\n> > +++ b/sequencer.c\n> > @@ -2388,3 +2388,48 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> > \n> > \tstrbuf_release(&sob);\n> > }\n> > +\n> > +int sequencer_make_script(int keep_empty, FILE *out,\n> > +\t\tint argc, const char **argv)\n> > +{\n> > +\tchar *format = xstrdup(\"%s\");\n> > +\tstruct pretty_print_context pp = {0};\n> > +\tstruct strbuf buf = STRBUF_INIT;\n> > +\tstruct rev_info revs;\n> > +\tstruct commit *commit;\n> > +\n> > +\tinit_revisions(&revs, NULL);\n> > +\trevs.verbose_header = 1;\n> > +\trevs.max_parents = 1;\n> > +\trevs.cherry_pick = 1;\n> > +\trevs.limited = 1;\n> > +\trevs.reverse = 1;\n> > +\trevs.right_only = 1;\n> > +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> > +\trevs.topo_order = 1;\n> > +\n> > +\trevs.pretty_given = 1;\n> > +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n> > +\tget_commit_format(format, &revs);\n> > +\tfree(format);\n> > +\tpp.fmt = revs.commit_format;\n> > +\tpp.output_encoding = get_log_output_encoding();\n> \n> All of the above feels like inviting unnecessary future breakages by\n> knowing too much about the implementation the current version of\n> revision.c happens to use.\n\nYou mean that the `--reverse` option gets translated into the `reverse`\nbit, and the other settings?\n\n:-)\n\n> A more careful implementation would be to allocate our own av[] and\n> prepare \"--reverse\", \"--left-right\", \"--cherry-pick\", etc. to be parsed\n> by setup_revisions() call we see below.\n\nOh, so you were not joking.\n\nPart of why I think we should stay away from shell scripts has nothing to\ndo with performance (which would already be worth it) nor portability\nissues (which also would already be worth it) nor requiring contributors to\nknow more than C (which also would already be worth it), but static\ntyping.\n\nWhat you are asking is to do away with the strong, static typing (which\nwould show a breakage pretty quickly if that part of revision.c's API was\nchanged, therefore I think your concern is a little curious) in favor of\nloose typing which would demonstrate breakages only upon use.\n\nThat is the exact opposite direction of where I want to go.\n\n> The parsing is not an expensive part of the operation anyway,\n\n... but why, oh why make things more complicated than they need to be? The\nrevision API is an API, yes, an internal one, but an API, for crying out\nloud.\n\n> and that way we do not have to worry about one less thing.\n\nNot that I don't mind no double or triple negations, but no, not this one.\n\n> > +\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n> > +\t\treturn error(_(\"make_script: unhandled options\"));\n> > +\n> > +\tif (prepare_revision_walk(&revs) < 0)\n> > +\t\treturn error(_(\"make_script: error preparing revisions\"));\n> > +\n> > +\twhile ((commit = get_revision(&revs))) {\n> > +\t\tstrbuf_reset(&buf);\n> > +\t\tif (!keep_empty && is_original_commit_empty(commit))\n> > +\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n> \n> Presumably callers of this function (which does not exist yet at\n> this step) are expected to have done the configuration dance to\n> prepare comment_line_char to whatever the end-user specified?\n\nYes. Just like they have to take care of discovering the .git/ directory.\n\nI guess I kind of fail to see your point. Of course the configuration has\nto be read at this point... This is an internal API function that has the\nsame contract as all the other internal API functions: you have to set up\nand configure everything needed to run the API function beforehand.\n\nBut maybe what you really wanted to ask is: How do we know that\ncomment_line_char is initialized correctly at this point?\n\nIf that is the question, I understand your puzzlement, and it is easy to\ndispell: comment_line_char is configured as part of\ngit_default_core_config(), and initialized to '#' before Git even starts\nto run.\n\nSo we're safe here, as long as the default config handling runs. The\nintended user is obviously the rebase--helper, which runs git_config()\neven before parsing the options.\n\nMeaning: the code is safe.\n\nCiao,\nDscho\n"},{"id":"318095","messageId":"alpine.DEB.2.20.1704272331190.3480@virtualbox","threadId":"43983","inReplyTo":"xmqqr30eo36n.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-27T21:44:35Z","receivedAt":"2017-04-27T21:44:55Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Wed, 26 Apr 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n> > index 214af0372ba..52a19e0bdb3 100644\n> > --- a/git-rebase--interactive.sh\n> > +++ b/git-rebase--interactive.sh\n> > @@ -774,11 +774,11 @@ transform_todo_ids () {\n> > }\n> > \n> > expand_todo_ids() {\n> > -\ttransform_todo_ids\n> > +\tgit rebase--helper --expand-sha1s\n> > }\n> > \n> > collapse_todo_ids() {\n> > -\ttransform_todo_ids --short\n> > +\tgit rebase--helper --shorten-sha1s\n> > }\n> \n> Obviously correct ;-) But doesn't this make transform_todo_ids ()\n> helper unused and removable?\n\nBut of course it is now unused! Will fix.\n\n> > +int transform_todo_ids(int shorten_sha1s)\n> > +{\n> > +\tconst char *todo_file = rebase_path_todo();\n> > +\tstruct todo_list todo_list = TODO_LIST_INIT;\n> > +\tint fd, res, i;\n> > +\tFILE *out;\n> > +\n> > +\tstrbuf_reset(&todo_list.buf);\n> > +\tfd = open(todo_file, O_RDONLY);\n> > +\tif (fd < 0)\n> > +\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n> > +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> > +\t\tclose(fd);\n> > +\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n> > +\t}\n> > +\tclose(fd);\n> > +\n> > +\tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n> > +\tif (res) {\n> > +\t\ttodo_list_release(&todo_list);\n> > +\t\treturn error(_(\"unusable instruction sheet: '%s'\"), todo_file);\n> > +\t}\n> > +\n> > +\tout = fopen(todo_file, \"w\");\n> \n> The usual \"open lockfile, write to it and then rename\" dance is not\n> necessary for the purpose of preventing other people from reading\n> this file while we are writing to it. But if we fail inside this\n> function before we fclose(3) \"out\", the user will lose the todo\n> list. It probably is not a big deal, though.\n\nI guess you're right. It is bug-for-bug equivalent to the previous shell\nfunction, though.\n\n> > +\tif (!out) {\n> > +\t\ttodo_list_release(&todo_list);\n> > +\t\treturn error(_(\"unable to open '%s' for writing\"), todo_file);\n> > +\t}\n> > +\tfor (i = 0; i < todo_list.nr; i++) {\n> > +\t\tstruct todo_item *item = todo_list.items + i;\n> > +\t\tint bol = item->offset_in_buf;\n> > +\t\tconst char *p = todo_list.buf.buf + bol;\n> > +\t\tint eol = i + 1 < todo_list.nr ?\n> > +\t\t\ttodo_list.items[i + 1].offset_in_buf :\n> > +\t\t\ttodo_list.buf.len;\n> > +\n> > +\t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n> > +\t\t\tfwrite(p, eol - bol, 1, out);\n> > +\t\telse {\n> > +\t\t\tint eoc = strcspn(p, \" \\t\");\n> > +\t\t\tconst char *sha1 = shorten_sha1s ?\n> > +\t\t\t\tshort_commit_name(item->commit) :\n> > +\t\t\t\toid_to_hex(&item->commit->object.oid);\n> > +\n> > +\t\t\tif (!eoc) {\n> > +\t\t\t\tp += strspn(p, \" \\t\");\n> > +\t\t\t\teoc = strcspn(p, \" \\t\");\n> > +\t\t\t}\n> \n> It would be much easier to follow the logic if \"int eoc\" above were\n> a mere declaration without initialization and \"skip to the\n> whitespaces\" is done immediately before this if() statement. It's\n> not like the initialized value of eoc is needed there because it\n> participates in the computation of sha1, and also having the\n> assignment followed by \"oops, the line begins with a whitespace\"\n> recovery that is done here.\n> \n> Wouldn't it be simpler to do:\n> \n> \telse {\n> \t\tint eoc;\n> \t\tconst char *sha1 = ...\n> \t\tp += strspn(p, \" \\t\"); /* skip optional indent */\n> \t\teoc = strcspn(p, \" \\t\"); /* grab the command word */\n> \n> without conditional?\n\nSure, will fix.\n\nCiao,\nDscho\n\n"},{"id":"318097","messageId":"alpine.DEB.2.20.1704272345370.3480@virtualbox","threadId":"43983","inReplyTo":"xmqqmvb2o2x7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 5/9] t3404: relax rebase.missingCommitsCheck tests","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-27T22:01:22Z","receivedAt":"2017-04-27T22:01:41Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Wed, 26 Apr 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > These tests were a bit anal about the *exact* warning/error message\n> > printed by git rebase. But those messages are intended for the *end\n> > user*, therefore it does not make sense to test so rigidly for the\n> > *exact* wording.\n> >\n> > In the following, we will reimplement the missing commits check in\n> > the sequencer, with slightly different words.\n> \n> Up to this point I thought your strategy was to mimic the original\n> as closely as possible, and changes to update (and/or improve) the\n> end user experience (like rewording messages) are left as a separate\n> step. Changes to the test can be used to demonstrate the improved\n> end-user experience that way, and I found it a good way to structure\n> your series. Is there a technical reason why you deviate from that\n> pattern here? \n> \n> Just being curious, as I do not particularly mind seeing things done\n> differently (especially if there is a good reason).\n\nYes, I remember the qualms I had about this patch. It was simply too\ncumbersome to keep the exact error message, as it would have meant to\ndeviate purposefully from the way we do things in C.\n\nIn our C code, we error out using the error() function, with translateable\nmessages. That is what the todo list parsing code in sequencer.c does:\n\n\terror: invalid line 4: badcmd 0547e3f1350d D\n\nIt even prints out the line number, which I think is a nice touch.\n\nCompare that to the non-standard way the rebase -i *shell* code reported\nthe failure:\n\n\tWarning: the command isn't recognized in the following line:\n\t - badcmd 0547e3f1350d D\n\nFirst of all, it reported it as a \"Warning\". Which is wrong, as it is a\nfatal error, and it was always treated as such.\n\nSecond, it uses a capitalized prefix, which not even our warning()\nfunction does.\n\nThird, it uses two lines, and then indents the offending line with a\nleading dash (as if we were in the middle of an enumeration).\n\nI would have added that the previous message also imitated Lore in that it\nuses a contraction, but I was shocked to learn through a simple git grep\nthat plenty of our error messages use that style, too.\n\nI could have simply re-written the error() as an `fprintf(stderr,\n\"Warning: ...\");` but that was just too inconsistent for my taste.\n\nHence I settled for relaxing the error message, which I had already done\nmuch earlier so that running the test with a `set -x` inserted somewhere\ninto git-rebase--interacive.sh would still work.\n\nCiao,\nDscho\n"},{"id":"318108","messageId":"xmqqbmrhmlsw.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"alpine.DEB.2.20.1704271607550.3480@virtualbox","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-28T00:13:19Z","receivedAt":"2017-04-28T00:13:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> Hi Junio,\n>\n> On Wed, 26 Apr 2017, Junio C Hamano wrote:\n>\n>> Johannes Schindelin writes:\n>> \n>> > diff --git a/sequencer.c b/sequencer.c\n>> > index 77afecaebf0..e858a976279 100644\n>> > --- a/sequencer.c\n>> > +++ b/sequencer.c\n>> > @@ -2388,3 +2388,48 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n>> > \n>> > \tstrbuf_release(&sob);\n>> > }\n>> > +\n>> > +int sequencer_make_script(int keep_empty, FILE *out,\n>> > +\t\tint argc, const char **argv)\n>> > +{\n>> > +\tchar *format = xstrdup(\"%s\");\n>> > +\tstruct pretty_print_context pp = {0};\n>> > +\tstruct strbuf buf = STRBUF_INIT;\n>> > +\tstruct rev_info revs;\n>> > +\tstruct commit *commit;\n>> > +\n>> > +\tinit_revisions(&revs, NULL);\n>> > +\trevs.verbose_header = 1;\n>> > +\trevs.max_parents = 1;\n>> > +\trevs.cherry_pick = 1;\n>> > +\trevs.limited = 1;\n>> > +\trevs.reverse = 1;\n>> > +\trevs.right_only = 1;\n>> > +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n>> > +\trevs.topo_order = 1;\n>> > +\n>> > +\trevs.pretty_given = 1;\n>> > +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n>> > +\tget_commit_format(format, &revs);\n>> > +\tfree(format);\n>> > +\tpp.fmt = revs.commit_format;\n>> > +\tpp.output_encoding = get_log_output_encoding();\n>> \n>> All of the above feels like inviting unnecessary future breakages by\n>> knowing too much about the implementation the current version of\n>> revision.c happens to use.\n>\n> You mean that the `--reverse` option gets translated into the `reverse`\n> bit, and the other settings?\n\nYes. The \"pretty_given\" trick is one example that the underlying\nimplementation can change over time. If you wrote this patch before\n66b2ed09 (\"Fix \"log\" family not to be too agressive about showing\nnotes\", 2010-01-20) happened, you wouldn't have known to flip this\nbit on to emulate the command line parsing of \"--pretty\" and\nfriends, and you would have required the author of that change to\nknow that you have this cut & pasted duplicated code here when the\ncommit is primarily about updating revision.c\n\nSo I am very serious when I say that this is adding an unnecessary\nmaintenance burden.\n\n"},{"id":"318109","messageId":"xmqq7f25mlpc.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"alpine.DEB.2.20.1704272331190.3480@virtualbox","subject":"Re: [PATCH v3 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-28T00:15:27Z","receivedAt":"2017-04-28T00:15:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n>> > +\tout = fopen(todo_file, \"w\");\n>> \n>> The usual \"open lockfile, write to it and then rename\" dance is not\n>> necessary for the purpose of preventing other people from reading\n>> this file while we are writing to it. But if we fail inside this\n>> function before we fclose(3) \"out\", the user will lose the todo\n>> list. It probably is not a big deal, though.\n>\n> I guess you're right. It is bug-for-bug equivalent to the previous shell\n> function, though.\n\nI think the scripted version uses the \"write to $todo.new and mv\n$todo.new to $todo\" pattern so you'd at least have something to go\nback to when the loopfails.\n"},{"id":"318114","messageId":"xmqqy3ull0m9.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"xmqqbmrhmlsw.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-04-28T02:36:14Z","receivedAt":"2017-04-28T02:36:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Yes. The \"pretty_given\" trick is one example that the underlying\n> implementation can change over time. If you wrote this patch before\n> 66b2ed09 (\"Fix \"log\" family not to be too agressive about showing\n> notes\", 2010-01-20) happened, you wouldn't have known to flip this\n> bit on to emulate the command line parsing of \"--pretty\" and\n> friends, and you would have required the author of that change to\n> know that you have this cut & pasted duplicated code here when the\n> commit is primarily about updating revision.c\n>\n> So I am very serious when I say that this is adding an unnecessary\n> maintenance burden.\n\nI _am_ sympathetic to your wish to have the compiler catch a\nmisspelt \"revs.verboes_header = 1\". A misspelt \"--formta=...\" \nwould not be caught until the execution time.\n\nBut the compiler's static name checking helps only one time while\nyou are writing _this_ patch, and it does not help at all to protect\nthis duplicated code from future breakages. The way \"rev-list\" and\nfriends _internally_ implement \"--format=...\" or any other options\nsed by the rev-list command whose behaviour you are recreating here\ncan (and will) change in the future, just like it already did change\nin early 2010. We didn't have pretty_given field for several years\nafter \"--pretty\" etc. that currently set the field were originally\nintroduced.\n\nIn an ideal world, we would probably have specific methods to\nmanipulate \"struct rev_info\" and set_format_string() method, which\nwould be called when the command line parser is reacting to\n\"--format=...\" in setup_revisions(), may encapsulate the\nimplementation detail of setting verbose_header and pretty_given\nfields in addition to calling get_commit_format() method on the\nrev_info object, and your new code may be calling that method,\nwithout having to know the implementation detail.\n\nWe do not live in that ideal world, and it is _not_ the theme of\nyour topic to bring us closer to the ideal world. Under that\nconstraint, a future-proof way to set up the revision machinery is\nto have setup_revisions() parse an av[] array. What will be done to\nyour copy of revs will stay compatible with what rev-list would do\nafter the implementation detail of setup_revisions() changes that\nway. It is true that a misspelt \"--formta=...\" would not be caught\nuntil the execution time, but once the code in this part is written,\nit is less likely to get broken by a change coming from needs by\nother parts of the system (e.g. the addition of pretty_given came\nnot because we wanted to enhance how --format or --pretty worked; it\ncame because we wanted to make sure they are not affected by changes\nto another option).\n\nSo after being forced by your response to rethink about it, I feel\neven firmer about this than I felt when I sent my first review\ncomment.\n\nThanks.\n\n\n\n"},{"id":"318138","messageId":"8c1f3519-0768-69d9-4d15-782da0be8390@talktalk.net","threadId":"43983","inReplyTo":"c44a15ed1f1015d7e9377e18610a0c428786995b.1493207864.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Phillip Wood","fromEmail":"phillip.wood@talktalk.net","sentAt":"2017-04-28T10:08:12Z","receivedAt":"2017-04-28T10:16:35Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 26/04/17 12:59, Johannes Schindelin wrote:\n> The first step of an interactive rebase is to generate the so-called \"todo\n> script\", to be stored in the state directory as \"git-rebase-todo\" and to\n> be edited by the user.\n> \n> Originally, we adjusted the output of `git log ` using a simple\n> sed script. Over the course of the years, the code became more\n> complicated. We now use shell scripting to edit the output of `git log`\n> conditionally, depending whether to keep \"empty\" commits (i.e. commits\n> that do not change any files).\n> \n> On platforms where shell scripting is not native, this can be a serious\n> drag. And it opens the door for incompatibilities between platforms when\n> it comes to shell scripting or to Unix-y commands.\n> \n> Let's just re-implement the todo script generation in plain C, using the\n> revision machinery directly.\n> \n> This is substantially faster, improving the speed relative to the\n> shell script version of the interactive rebase from 2x to 3x on Windows.\n> \n> Note that the rearrange_squash() function in git-rebase--interactive\n> relied on the fact that we set the \"format\" variable to the config setting\n> rebase.instructionFormat. Relying on a side effect like this is no good,\n> hence we explicitly perform that assignment (possibly again) in\n> rearrange_squash().\n> \n> Signed-off-by: Johannes Schindelin \n> ---\n> builtin/rebase--helper.c | 8 +++++++-\n> git-rebase--interactive.sh | 44 +++++++++++++++++++++++---------------------\n> sequencer.c | 45 +++++++++++++++++++++++++++++++++++++++++++++\n> sequencer.h | 3 +++\n> 4 files changed, 78 insertions(+), 22 deletions(-)\n> \n> diff --git a/sequencer.c b/sequencer.c\n> index 77afecaebf0..e858a976279 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -2388,3 +2388,48 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> \n> \tstrbuf_release(&sob);\n> }\n> +\n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv)\n> +{\n> +\tchar *format = xstrdup(\"%s\");\n> +\tstruct pretty_print_context pp = {0};\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tstruct rev_info revs;\n> +\tstruct commit *commit;\n> +\n> +\tinit_revisions(&revs, NULL);\n> +\trevs.verbose_header = 1;\n> +\trevs.max_parents = 1;\n> +\trevs.cherry_pick = 1;\n> +\trevs.limited = 1;\n> +\trevs.reverse = 1;\n> +\trevs.right_only = 1;\n> +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> +\trevs.topo_order = 1;\n> +\n> +\trevs.pretty_given = 1;\n> +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n\nFirstly thanks for all your work on speeding up rebase -i, it definitely\nfeels faster.\n\nThis changes the behaviour of\ngit -c rebase.instructionFormat= rebase -i\nThe shell version treats the rebase.instructionFormat being unset or set\nto the empty string as equivalent. This version generates a todo list\nwith lines like 'pick ' rather than 'pick \n'\n\nI only picked this up because I have a script that does 'git -c\nrebase.instructionFormat= rebase -i' with a custom sequence editor. I\ncan easily add '%s' in the appropriate place but I thought I'd point it\nout in case other people are affected by the change.\n\nPlease CC me in any replies as I'm not subscribed to this list\n\nBest Wishes\n\nPhillip\n\n> +\tget_commit_format(format, &revs);\n> +\tfree(format);\n> +\tpp.fmt = revs.commit_format;\n> +\tpp.output_encoding = get_log_output_encoding();\n> +\n> +\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n> +\t\treturn error(_(\"make_script: unhandled options\"));\n> +\n> +\tif (prepare_revision_walk(&revs) < 0)\n> +\t\treturn error(_(\"make_script: error preparing revisions\"));\n> +\n> +\twhile ((commit = get_revision(&revs))) {\n> +\t\tstrbuf_reset(&buf);\n> +\t\tif (!keep_empty && is_original_commit_empty(commit))\n> +\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n> +\t\tstrbuf_addf(&buf, \"pick %s \", oid_to_hex(&commit->object.oid));\n> +\t\tpretty_print_commit(&pp, commit, &buf);\n> +\t\tstrbuf_addch(&buf, '\\n');\n> +\t\tfputs(buf.buf, out);\n> +\t}\n> +\tstrbuf_release(&buf);\n> +\treturn 0;\n> +}\n> diff --git a/sequencer.h b/sequencer.h\n> index f885b68395f..83f2943b7a9 100644\n> --- a/sequencer.h\n> +++ b/sequencer.h\n> @@ -45,6 +45,9 @@ int sequencer_continue(struct replay_opts *opts);\n> int sequencer_rollback(struct replay_opts *opts);\n> int sequencer_remove_state(struct replay_opts *opts);\n> \n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv);\n> +\n> extern const char sign_off_header[];\n> \n> void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n> \n\n"},{"id":"318182","messageId":"alpine.DEB.2.20.1704281659010.3480@virtualbox","threadId":"43983","inReplyTo":"xmqqh91ao1of.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 6/9] rebase -i: check for missing commits in the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T15:10:52Z","receivedAt":"2017-04-28T15:11:11Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\n\nOn Wed, 26 Apr 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > -check_todo_list\n> > +git rebase--helper --check-todo-list || {\n> > +\tret=$?\n> > +\tcheckout_onto\n> > +\texit $ret\n> > +}\n> \n> I find this a better division of labor between \"check_todo_list\" and\n> its caller. Compared to the original that did the \"recover and exit\n> with failure\" inside the helper, this is much easier to see what is\n> going on.\n\nYes. My first attempt did not even checkout , and it was\nsurprisingly difficult to pin that one down. I would never have expected\ncheck_todo_list to have that side effect.\n\n> > +/*\n> > + * Check if the user dropped some commits by mistake\n> > + * Behaviour determined by rebase.missingCommitsCheck.\n> > + * Check if there is an unrecognized command or a\n> > + * bad SHA-1 in a command.\n> > + */\n> > +int check_todo_list(void)\n> > +{\n> > +\tenum check_level check_level = get_missing_commit_check_level();\n> > +\tstruct strbuf todo_file = STRBUF_INIT;\n> > +\tstruct todo_list todo_list = TODO_LIST_INIT;\n> > +\tstruct commit_list *missing = NULL;\n> > +\tint raise_error = 0, res = 0, fd, i;\n> > +\n> > +\tstrbuf_addstr(&todo_file, rebase_path_todo());\n> > +\tfd = open(todo_file.buf, O_RDONLY);\n> > +\tif (fd < 0) {\n> > +\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n> > +\t\tgoto leave_check;\n> > +\t}\n> > +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> > +\t\tclose(fd);\n> > +\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n> > +\t\tgoto leave_check;\n> > +\t}\n> > +\tclose(fd);\n> > +\traise_error = res =\n> > +\t\tparse_insn_buffer(todo_list.buf.buf, &todo_list);\n> > +\n> > +\tif (check_level == CHECK_IGNORE)\n> > +\t\tgoto leave_check;\n> \n> OK, so even it is set to ignore, unreadable todo list will be shown\n> with a loud error message that tells the user to use --edit-todo.\n> \n> What should happen when it is not set to ignore and we found the\n> todo list unacceptable, I wonder?\n\nWhoops. In case of a parse error, it does not make sense to check, does\nit. Fixed.\n\n> > +\t/* Get the SHA-1 of the commits */\n> > +\tfor (i = 0; i < todo_list.nr; i++) {\n> > +\t\tstruct commit *commit = todo_list.items[i].commit;\n> > +\t\tif (commit)\n> > +\t\t\tcommit->util = todo_list.items + i;\n> > +\t}\n> \n> It does not look like this loop is \"Get(ting) the SHA-1 of the commits\"\n> to me, though. It is setting up ->util to be usable as a back-pointer\n> into the list.\n\nRight, and that is not even necessary. It is even incorrect, as we release\nthe todo_list and read git-rebase-todo.backup into the same data\nstructure, possibly reallocating the array, therefore the pointers may\nbecome stale. So I went with your suggestion further down to use (void *)1\ninstead.\n\nAlso, the comment is actively wrong, I agree. I changed it to\n\n\t/* Mark the commits in git-rebase-todo as seen */\n\n> > +\ttodo_list_release(&todo_list);\n> \n> But then the todo-list is released? The util field we have set, if\n> any, in the previous loop are now dangling, no?\n\nRight.\n\n> > +\tstrbuf_addstr(&todo_file, \".backup\");\n> > +\tfd = open(todo_file.buf, O_RDONLY);\n> > +\tif (fd < 0) {\n> > +\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n> > +\t\tgoto leave_check;\n> > +\t}\n> > +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> > +\t\tclose(fd);\n> > +\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n> > +\t\tgoto leave_check;\n> > +\t}\n> > +\tclose(fd);\n> > +\tstrbuf_release(&todo_file);\n> > +\tres = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);\n> \n> Then we read from .backup; failure to do so does not result in the\n> \"you need to --edit-todo\" warning.\n\nCorrect. At this point, we could even add\n\n\tif (res)\n\t\tdie(\"BUG: cannot read '%s'\", todo_file.buf);\n\n(moving the strbuf_release(&todo_file) below, of course), as the .backup\nfile is not intended to be edited by the user, i.e. it is the original\ntodo which should *never* be unparseable.\n\n> > +\t/* Find commits that are missing after editing */\n> > +\tfor (i = 0; i < todo_list.nr; i++) {\n> > +\t\tstruct commit *commit = todo_list.items[i].commit;\n> > +\t\tif (commit && !commit->util) {\n> > +\t\t\tcommit_list_insert(commit, &missing);\n> > +\t\t\tcommit->util = todo_list.items + i;\n> > +\t\t}\n> > +\t}\n> \n> And we check the commits mentioned in the backup; any commit whose\n> util is not marked in the previous loop is noticed and thrown into\n> the missing list.\n> \n> The loop we later have does \"while (missing)\" and does not look at\n> commit->util for commits that are *not* missing, i.e. the ones that\n> are marked in the previous loop, so it does not matter that their\n> util field have dangling pointers. In that sense, it may not be\n> buggy, but it is misleading. The only thing these two loops care\n> about is that the commits found in the earlier loop get their util\n> field set to non-NULL, so instead of using \"todo_list.items+i\",\n> perhaps doing this\n> \n> \tif (commit)\n> \t\tcommit->util = (void *)1; /* mark as seen */\n> \n> in the earlier loop instead would be much less confusing.\n\n... and doing the same in this loop. I agree, that's exactly what I\nchanged it to.\n\n> > +\t/* Warn about missing commits */\n> > +\tif (!missing)\n> > +\t\tgoto leave_check;\n> \n> If there is no missing one, we may still return error about\n> unacceptable backup file. But if we read backup fine and didn't\n> find anything missing, we'll return silently and with success. OK.\n> \n> > +\tif (check_level == CHECK_ERROR)\n> > +\t\traise_error = res = 1;\n> \n> Otherwise, we found missing ones and we want to report here.\n> \n> The reason why I started reading this function aloud was because I\n> found two variables (raise_error and res) somewhat confusing. I\n> think what the code does makes sense, but I still find the way how\n> the code expresses the logic with these two variables confusing.\n> Perhaps somebody else can hopefully offer possible improvements, as\n> I do not offhand think of a way better than what is currently in\n> this patch myself.\n\nI renamed the `raise_error` variable to `advise_to_edit_todo`. The `res`\ndoes not need renaming, methinks, as it is used everywhere else in that\nfile to indicate the return value.\n\nCiao,\nDscho\n"},{"id":"318183","messageId":"alpine.DEB.2.20.1704281711540.3480@virtualbox","threadId":"43983","inReplyTo":"xmqqbmrhmlsw.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T15:13:37Z","receivedAt":"2017-04-28T15:13:54Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Thu, 27 Apr 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > On Wed, 26 Apr 2017, Junio C Hamano wrote:\n> >\n> >> Johannes Schindelin writes:\n> >> \n> >> > diff --git a/sequencer.c b/sequencer.c\n> >> > index 77afecaebf0..e858a976279 100644\n> >> > --- a/sequencer.c\n> >> > +++ b/sequencer.c\n> >> > @@ -2388,3 +2388,48 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> >> > \n> >> > \tstrbuf_release(&sob);\n> >> > }\n> >> > +\n> >> > +int sequencer_make_script(int keep_empty, FILE *out,\n> >> > +\t\tint argc, const char **argv)\n> >> > +{\n> >> > +\tchar *format = xstrdup(\"%s\");\n> >> > +\tstruct pretty_print_context pp = {0};\n> >> > +\tstruct strbuf buf = STRBUF_INIT;\n> >> > +\tstruct rev_info revs;\n> >> > +\tstruct commit *commit;\n> >> > +\n> >> > +\tinit_revisions(&revs, NULL);\n> >> > +\trevs.verbose_header = 1;\n> >> > +\trevs.max_parents = 1;\n> >> > +\trevs.cherry_pick = 1;\n> >> > +\trevs.limited = 1;\n> >> > +\trevs.reverse = 1;\n> >> > +\trevs.right_only = 1;\n> >> > +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> >> > +\trevs.topo_order = 1;\n> >> > +\n> >> > +\trevs.pretty_given = 1;\n> >> > +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n> >> > +\tget_commit_format(format, &revs);\n> >> > +\tfree(format);\n> >> > +\tpp.fmt = revs.commit_format;\n> >> > +\tpp.output_encoding = get_log_output_encoding();\n> >> \n> >> All of the above feels like inviting unnecessary future breakages by\n> >> knowing too much about the implementation the current version of\n> >> revision.c happens to use.\n> >\n> > You mean that the `--reverse` option gets translated into the `reverse`\n> > bit, and the other settings?\n> \n> Yes. The \"pretty_given\" trick is one example that the underlying\n> implementation can change over time. If you wrote this patch before\n> 66b2ed09 (\"Fix \"log\" family not to be too agressive about showing\n> notes\", 2010-01-20) happened, you wouldn't have known to flip this\n> bit on to emulate the command line parsing of \"--pretty\" and\n> friends, and you would have required the author of that change to\n> know that you have this cut & pasted duplicated code here when the\n> commit is primarily about updating revision.c\n> \n> So I am very serious when I say that this is adding an unnecessary\n> maintenance burden.\n\nIn that case, I would strongly advise to consider redesigning the API. It\nis just no good to ask for a change in stringent code that would delay\ncompile errors to runtime errors, that's just poor form.\n\nAnd if the API allows settings that do something unintentional without at\nleast a runtime warning, the API is no good.\n\nCiao,\nDscho\n"},{"id":"318184","messageId":"alpine.DEB.2.20.1704281714480.3480@virtualbox","threadId":"43983","inReplyTo":"xmqq7f25mlpc.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 4/9] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T15:15:25Z","receivedAt":"2017-04-28T15:15:40Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Thu, 27 Apr 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> >> > +\tout = fopen(todo_file, \"w\");\n> >> \n> >> The usual \"open lockfile, write to it and then rename\" dance is not\n> >> necessary for the purpose of preventing other people from reading\n> >> this file while we are writing to it. But if we fail inside this\n> >> function before we fclose(3) \"out\", the user will lose the todo\n> >> list. It probably is not a big deal, though.\n> >\n> > I guess you're right. It is bug-for-bug equivalent to the previous shell\n> > function, though.\n> \n> I think the scripted version uses the \"write to $todo.new and mv\n> $todo.new to $todo\" pattern so you'd at least have something to go\n> back to when the loopfails.\n\nMy mistake.\n\nSorry,\nDscho\n"},{"id":"318197","messageId":"alpine.DEB.2.20.1704282059100.3480@virtualbox","threadId":"43983","inReplyTo":"8c1f3519-0768-69d9-4d15-782da0be8390@talktalk.net","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T19:22:30Z","receivedAt":"2017-04-28T19:22:58Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Philip,\n\nOn Fri, 28 Apr 2017, Phillip Wood wrote:\n\n> On 26/04/17 12:59, Johannes Schindelin wrote:\n>\n> > The first step of an interactive rebase is to generate the so-called\n> > \"todo script\", to be stored in the state directory as\n> > \"git-rebase-todo\" and to be edited by the user.\n> > \n> > Originally, we adjusted the output of `git log ` using a\n> > simple sed script. Over the course of the years, the code became more\n> > complicated. We now use shell scripting to edit the output of `git\n> > log` conditionally, depending whether to keep \"empty\" commits (i.e.\n> > commits that do not change any files).\n> > \n> > On platforms where shell scripting is not native, this can be a\n> > serious drag. And it opens the door for incompatibilities between\n> > platforms when it comes to shell scripting or to Unix-y commands.\n> > \n> > Let's just re-implement the todo script generation in plain C, using\n> > the revision machinery directly.\n> > \n> > This is substantially faster, improving the speed relative to the\n> > shell script version of the interactive rebase from 2x to 3x on\n> > Windows.\n> \n> This changes the behaviour of git -c rebase.instructionFormat= rebase -i\n> The shell version treats the rebase.instructionFormat being unset or set\n> to the empty string as equivalent. This version generates a todo list\n> with lines like 'pick ' rather than 'pick \n> '\n> \n> I only picked this up because I have a script that does 'git -c\n> rebase.instructionFormat= rebase -i' with a custom sequence editor. I\n> can easily add '%s' in the appropriate place but I thought I'd point it\n> out in case other people are affected by the change.\n\nWhile I would argue that the C version is more correct, it would be\nbackwards-incompatible.\n\nSo I changed it.\n\nBTW in the future you could help me a *lot* by providing a patch that adds\na test case to our test suite that not only demonstrates what exactly goes\nwrong, but also will help prevent future regressions.\n\nCiao,\nJohannes\n"},{"id":"318200","messageId":"cover.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493207864.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:30:42Z","receivedAt":"2017-04-28T21:31:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This patch series reimplements the expensive pre- and post-processing of\nthe todo script in C.\n\nAnd it concludes the work I did to accelerate rebase -i.\n\nChanges since v3:\n\n- removed the no-longer-used transform_todo_ids shell function\n\n- simplified transform_todo_ids()'s command parsing\n\n- fixed two commits in check_todo_list(), renamed the unclear\n `raise_error` variable to `advise_to_edit_todo`, build the message\n about missing commits directly (without the detour to building a\n commit_list) and instead of assigning an unused pointer to commit->util\n the code now uses (void *)1.\n\n- return early from check_todo_list() when parsing failed, even if the\n check level is something else than CHECK_IGNORE\n\n- the todo list is generated is again generated in the same way as\n before when rebase.instructionFormat is empty: it was interpreted as\n if it had not been set\n\n- added a test for empty rebase.instructionFormat settings\n\n\nJohannes Schindelin (10):\n t3415: verify that an empty instructionFormat is handled as before\n rebase -i: generate the script via rebase--helper\n rebase -i: remove useless indentation\n rebase -i: do not invent onelines when expanding/collapsing SHA-1s\n rebase -i: also expand/collapse the SHA-1s via the rebase--helper\n t3404: relax rebase.missingCommitsCheck tests\n rebase -i: check for missing commits in the rebase--helper\n rebase -i: skip unnecessary picks using the rebase--helper\n t3415: test fixup with wrapped oneline\n rebase -i: rearrange fixup/squash lines using the rebase--helper\n\n Documentation/git-rebase.txt | 16 +-\n builtin/rebase--helper.c | 29 ++-\n git-rebase--interactive.sh | 373 ++++-------------------------\n sequencer.c | 530 ++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 8 +\n t/t3404-rebase-interactive.sh | 22 +-\n t/t3415-rebase-autosquash.sh | 28 ++-\n 7 files changed, 646 insertions(+), 360 deletions(-)\n\n\nbase-commit: 027a3b943b444a3e3a76f9a89803fc10245b858f\nBased-On: rebase--helper at https://github.com/dscho/git\nFetch-Base-Via: git fetch https://github.com/dscho/git rebase--helper\nPublished-As: https://github.com/dscho/git/releases/tag/rebase-i-extra-v4\nFetch-It-Via: git fetch https://github.com/dscho/git rebase-i-extra-v4\n\nInterdiff vs v3:\n\n diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n index d39fe4f5fb7..84c6e62518f 100644\n --- a/git-rebase--interactive.sh\n +++ b/git-rebase--interactive.sh\n @@ -713,29 +713,6 @@ do_rest () {\n \tdone\n }\n \n -transform_todo_ids () {\n -\twhile read -r command rest\n -\tdo\n -\t\tcase \"$command\" in\n -\t\t\"$comment_char\"* | exec)\n -\t\t\t# Be careful for oddball commands like 'exec'\n -\t\t\t# that do not have a SHA-1 at the beginning of $rest.\n -\t\t\t;;\n -\t\t*)\n -\t\t\tsha1=$(git rev-parse --verify --quiet \"$@\" ${rest%%[\t ]*}) &&\n -\t\t\tif test \"a$rest\" = \"a${rest#*[\t ]}\"\n -\t\t\tthen\n -\t\t\t\trest=$sha1\n -\t\t\telse\n -\t\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n -\t\t\tfi\n -\t\t\t;;\n -\t\tesac\n -\t\tprintf '%s\\n' \"$command${rest:+ }$rest\"\n -\tdone <\"$todo\" >\"$todo.new\" &&\n -\tmv -f \"$todo.new\" \"$todo\"\n -}\n -\n expand_todo_ids() {\n \tgit rebase--helper --expand-sha1s\n }\n diff --git a/sequencer.c b/sequencer.c\n index 84f8e366761..63a588f0916 100644\n --- a/sequencer.c\n +++ b/sequencer.c\n @@ -2393,7 +2393,7 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv)\n {\n -\tchar *format = xstrdup(\"%s\");\n +\tchar *format = NULL;\n \tstruct pretty_print_context pp = {0};\n \tstruct strbuf buf = STRBUF_INIT;\n \tstruct rev_info revs;\n @@ -2411,6 +2411,10 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \n \trevs.pretty_given = 1;\n \tgit_config_get_string(\"rebase.instructionFormat\", &format);\n +\tif (!format || !*format) {\n +\t\tfree(format);\n +\t\tformat = xstrdup(\"%s\");\n +\t}\n \tget_commit_format(format, &revs);\n \tfree(format);\n \tpp.fmt = revs.commit_format;\n @@ -2475,18 +2479,16 @@ int transform_todo_ids(int shorten_sha1s)\n \t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n \t\t\tfwrite(p, eol - bol, 1, out);\n \t\telse {\n -\t\t\tint eoc = strcspn(p, \" \\t\");\n \t\t\tconst char *sha1 = shorten_sha1s ?\n \t\t\t\tshort_commit_name(item->commit) :\n \t\t\t\toid_to_hex(&item->commit->object.oid);\n +\t\t\tint len;\n \n -\t\t\tif (!eoc) {\n -\t\t\t\tp += strspn(p, \" \\t\");\n -\t\t\t\teoc = strcspn(p, \" \\t\");\n -\t\t\t}\n +\t\t\tp += strspn(p, \" \\t\"); /* left-trim command */\n +\t\t\tlen = strcspn(p, \" \\t\"); /* length of command */\n \n \t\t\tfprintf(out, \"%.*s %s %.*s\\n\",\n -\t\t\t\teoc, p, sha1, item->arg_len, item->arg);\n +\t\t\t\tlen, p, sha1, item->arg_len, item->arg);\n \t\t}\n \t}\n \tfclose(out);\n @@ -2525,8 +2527,8 @@ int check_todo_list(void)\n \tenum check_level check_level = get_missing_commit_check_level();\n \tstruct strbuf todo_file = STRBUF_INIT;\n \tstruct todo_list todo_list = TODO_LIST_INIT;\n -\tstruct commit_list *missing = NULL;\n -\tint raise_error = 0, res = 0, fd, i;\n +\tstruct strbuf missing = STRBUF_INIT;\n +\tint advise_to_edit_todo = 0, res = 0, fd, i;\n \n \tstrbuf_addstr(&todo_file, rebase_path_todo());\n \tfd = open(todo_file.buf, O_RDONLY);\n @@ -2540,17 +2542,17 @@ int check_todo_list(void)\n \t\tgoto leave_check;\n \t}\n \tclose(fd);\n -\traise_error = res =\n +\tadvise_to_edit_todo = res =\n \t\tparse_insn_buffer(todo_list.buf.buf, &todo_list);\n \n -\tif (check_level == CHECK_IGNORE)\n +\tif (res || check_level == CHECK_IGNORE)\n \t\tgoto leave_check;\n \n -\t/* Get the SHA-1 of the commits */\n +\t/* Mark the commits in git-rebase-todo as seen */\n \tfor (i = 0; i < todo_list.nr; i++) {\n \t\tstruct commit *commit = todo_list.items[i].commit;\n \t\tif (commit)\n -\t\t\tcommit->util = todo_list.items + i;\n +\t\t\tcommit->util = (void *)1;\n \t}\n \n \ttodo_list_release(&todo_list);\n @@ -2569,35 +2571,32 @@ int check_todo_list(void)\n \tstrbuf_release(&todo_file);\n \tres = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);\n \n -\t/* Find commits that are missing after editing */\n -\tfor (i = 0; i < todo_list.nr; i++) {\n -\t\tstruct commit *commit = todo_list.items[i].commit;\n +\t/* Find commits in git-rebase-todo.backup yet unseen */\n +\tfor (i = todo_list.nr - 1; i >= 0; i--) {\n +\t\tstruct todo_item *item = todo_list.items + i;\n +\t\tstruct commit *commit = item->commit;\n \t\tif (commit && !commit->util) {\n -\t\t\tcommit_list_insert(commit, &missing);\n -\t\t\tcommit->util = todo_list.items + i;\n +\t\t\tstrbuf_addf(&missing, \" - %s %.*s\\n\",\n +\t\t\t\t short_commit_name(commit),\n +\t\t\t\t item->arg_len, item->arg);\n +\t\t\tcommit->util = (void *)1;\n \t\t}\n \t}\n \n \t/* Warn about missing commits */\n -\tif (!missing)\n +\tif (!missing.len)\n \t\tgoto leave_check;\n \n \tif (check_level == CHECK_ERROR)\n -\t\traise_error = res = 1;\n +\t\tadvise_to_edit_todo = res = 1;\n \n \tfprintf(stderr,\n \t\t_(\"Warning: some commits may have been dropped accidentally.\\n\"\n \t\t\"Dropped commits (newer to older):\\n\"));\n \n \t/* Make the list user-friendly and display */\n -\twhile (missing) {\n -\t\tstruct commit *commit = pop_commit(&missing);\n -\t\tstruct todo_item *item = commit->util;\n -\n -\t\tfprintf(stderr, \" - %s %.*s\\n\", short_commit_name(commit),\n -\t\t\titem->arg_len, item->arg);\n -\t}\n -\tfree_commit_list(missing);\n +\tfputs(missing.buf, stderr);\n +\tstrbuf_release(&missing);\n \n \tfprintf(stderr, _(\"To avoid this message, use \\\"drop\\\" to \"\n \t\t\"explicitly remove a commit.\\n\\n\"\n @@ -2609,7 +2608,7 @@ int check_todo_list(void)\n \tstrbuf_release(&todo_file);\n \ttodo_list_release(&todo_list);\n \n -\tif (raise_error)\n +\tif (advise_to_edit_todo)\n \t\tfprintf(stderr,\n \t\t\t_(\"You can fix this with 'git rebase --edit-todo' \"\n \t\t\t \"and then run 'git rebase --continue'.\\n\"\n diff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\n index b9e26008a79..2f88f50c057 100755\n --- a/t/t3415-rebase-autosquash.sh\n +++ b/t/t3415-rebase-autosquash.sh\n @@ -271,6 +271,18 @@ test_expect_success 'autosquash with custom inst format' '\n \ttest 2 = $(git cat-file commit HEAD^ | grep squash | wc -l)\n '\n \n +test_expect_success 'autosquash with empty custom instructionFormat' '\n +\tgit reset --hard base &&\n +\ttest_commit empty-instructionFormat-test &&\n +\t(\n +\t\tset_cat_todo_editor &&\n +\t\ttest_must_fail git -c rebase.instructionFormat= \\\n +\t\t\trebase --autosquash --force -i HEAD^ >actual &&\n +\t\tgit log -1 --format=\"pick %h %s\" >expect &&\n +\t\ttest_cmp expect actual\n +\t)\n +'\n +\n set_backup_editor () {\n \twrite_script backup-editor.sh <<-\\EOF\n \tcp \"$1\" .git/backup-\"$(basename \"$1\")\"\n\n-- \n2.12.2.windows.2.800.gede8f145e06\n\n"},{"id":"318201","messageId":"210c26f91c8dc8a4bd05fda2c46eb0c28bfb2da6.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 01/10] t3415: verify that an empty instructionFormat is handled as before","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:31:30Z","receivedAt":"2017-04-28T21:31:50Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"An upcoming patch will move the todo list generation into the\nrebase--helper. An early version of that patch regressed on an empty\nrebase.instructionFormat value (the shell version could not discern\nbetween an empty one and a non-existing one, but the C version used the\nempty one as if that was intended to skip the oneline from the `pick\n` lines).\n\nLet's verify that this still works as before.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3415-rebase-autosquash.sh | 12 ++++++++++++\n 1 file changed, 12 insertions(+)\n\ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 48346f1cc0c..6c37ebdff87 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -271,6 +271,18 @@ test_expect_success 'autosquash with custom inst format' '\n \ttest 2 = $(git cat-file commit HEAD^ | grep squash | wc -l)\n '\n \n+test_expect_success 'autosquash with empty custom instructionFormat' '\n+\tgit reset --hard base &&\n+\ttest_commit empty-instructionFormat-test &&\n+\t(\n+\t\tset_cat_todo_editor &&\n+\t\ttest_must_fail git -c rebase.instructionFormat= \\\n+\t\t\trebase --autosquash --force -i HEAD^ >actual &&\n+\t\tgit log -1 --format=\"pick %h %s\" >expect &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n set_backup_editor () {\n \twrite_script backup-editor.sh <<-\\EOF\n \tcp \"$1\" .git/backup-\"$(basename \"$1\")\"\n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318202","messageId":"e173445d9a6321f80053967e50375400c0a61764.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:31:37Z","receivedAt":"2017-04-28T21:31:56Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The first step of an interactive rebase is to generate the so-called \"todo\nscript\", to be stored in the state directory as \"git-rebase-todo\" and to\nbe edited by the user.\n\nOriginally, we adjusted the output of `git log ` using a simple\nsed script. Over the course of the years, the code became more\ncomplicated. We now use shell scripting to edit the output of `git log`\nconditionally, depending whether to keep \"empty\" commits (i.e. commits\nthat do not change any files).\n\nOn platforms where shell scripting is not native, this can be a serious\ndrag. And it opens the door for incompatibilities between platforms when\nit comes to shell scripting or to Unix-y commands.\n\nLet's just re-implement the todo script generation in plain C, using the\nrevision machinery directly.\n\nThis is substantially faster, improving the speed relative to the\nshell script version of the interactive rebase from 2x to 3x on Windows.\n\nNote that the rearrange_squash() function in git-rebase--interactive\nrelied on the fact that we set the \"format\" variable to the config setting\nrebase.instructionFormat. Relying on a side effect like this is no good,\nhence we explicitly perform that assignment (possibly again) in\nrearrange_squash().\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 8 +++++++-\n git-rebase--interactive.sh | 44 +++++++++++++++++++++--------------------\n sequencer.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 3 +++\n 4 files changed, 82 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex ca1ebb2fa18..821058d452d 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -11,15 +11,19 @@ static const char * const builtin_rebase_helper_usage[] = {\n int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n {\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n+\tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n+\t\tOPT_BOOL(0, \"keep-empty\", &keep_empty, N_(\"keep empty commits\")),\n \t\tOPT_CMDMODE(0, \"continue\", &command, N_(\"continue rebase\"),\n \t\t\t\tCONTINUE),\n \t\tOPT_CMDMODE(0, \"abort\", &command, N_(\"abort rebase\"),\n \t\t\t\tABORT),\n+\t\tOPT_CMDMODE(0, \"make-script\", &command,\n+\t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n \t\tOPT_END()\n \t};\n \n@@ -36,5 +40,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!sequencer_continue(&opts);\n \tif (command == ABORT && argc == 1)\n \t\treturn !!sequencer_remove_state(&opts);\n+\tif (command == MAKE_SCRIPT && argc > 1)\n+\t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 2c9c0165b5a..609e150d38f 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -785,6 +785,7 @@ collapse_todo_ids() {\n # each log message will be re-retrieved in order to normalize the\n # autosquash arrangement\n rearrange_squash () {\n+\tformat=$(git config --get rebase.instructionFormat)\n \t# extract fixup!/squash! lines and resolve any referenced sha1's\n \twhile read -r pick sha1 message\n \tdo\n@@ -1210,26 +1211,27 @@ else\n \trevisions=$onto...$orig_head\n \tshortrevisions=$shorthead\n fi\n-format=$(git config --get rebase.instructionFormat)\n-# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n-git rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n-\t--reverse --left-right --topo-order \\\n-\t$revisions ${restrict_revision+^$restrict_revision} | \\\n-\tsed -n \"s/^>//p\" |\n-while read -r sha1 rest\n-do\n-\n-\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n-\tthen\n-\t\tcomment_out=\"$comment_char \"\n-\telse\n-\t\tcomment_out=\n-\tfi\n+if test t != \"$preserve_merges\"\n+then\n+\tgit rebase--helper --make-script ${keep_empty:+--keep-empty} \\\n+\t\t$revisions ${restrict_revision+^$restrict_revision} >\"$todo\"\n+else\n+\tformat=$(git config --get rebase.instructionFormat)\n+\t# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n+\tgit rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n+\t\t--reverse --left-right --topo-order \\\n+\t\t$revisions ${restrict_revision+^$restrict_revision} | \\\n+\t\tsed -n \"s/^>//p\" |\n+\twhile read -r sha1 rest\n+\tdo\n+\n+\t\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n+\t\tthen\n+\t\t\tcomment_out=\"$comment_char \"\n+\t\telse\n+\t\t\tcomment_out=\n+\t\tfi\n \n-\tif test t != \"$preserve_merges\"\n-\tthen\n-\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n-\telse\n \t\tif test -z \"$rebase_root\"\n \t\tthen\n \t\t\tpreserve=t\n@@ -1248,8 +1250,8 @@ do\n \t\t\ttouch \"$rewritten\"/$sha1\n \t\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n \t\tfi\n-\tfi\n-done\n+\tdone\n+fi\n \n # Watch for commits that been dropped by --cherry-pick\n if test t = \"$preserve_merges\"\ndiff --git a/sequencer.c b/sequencer.c\nindex 130cc868e51..88819a1a2a9 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2388,3 +2388,52 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n \n \tstrbuf_release(&sob);\n }\n+\n+int sequencer_make_script(int keep_empty, FILE *out,\n+\t\tint argc, const char **argv)\n+{\n+\tchar *format = NULL;\n+\tstruct pretty_print_context pp = {0};\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct rev_info revs;\n+\tstruct commit *commit;\n+\n+\tinit_revisions(&revs, NULL);\n+\trevs.verbose_header = 1;\n+\trevs.max_parents = 1;\n+\trevs.cherry_pick = 1;\n+\trevs.limited = 1;\n+\trevs.reverse = 1;\n+\trevs.right_only = 1;\n+\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n+\trevs.topo_order = 1;\n+\n+\trevs.pretty_given = 1;\n+\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n+\tif (!format || !*format) {\n+\t\tfree(format);\n+\t\tformat = xstrdup(\"%s\");\n+\t}\n+\tget_commit_format(format, &revs);\n+\tfree(format);\n+\tpp.fmt = revs.commit_format;\n+\tpp.output_encoding = get_log_output_encoding();\n+\n+\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n+\t\treturn error(_(\"make_script: unhandled options\"));\n+\n+\tif (prepare_revision_walk(&revs) < 0)\n+\t\treturn error(_(\"make_script: error preparing revisions\"));\n+\n+\twhile ((commit = get_revision(&revs))) {\n+\t\tstrbuf_reset(&buf);\n+\t\tif (!keep_empty && is_original_commit_empty(commit))\n+\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n+\t\tstrbuf_addf(&buf, \"pick %s \", oid_to_hex(&commit->object.oid));\n+\t\tpretty_print_commit(&pp, commit, &buf);\n+\t\tstrbuf_addch(&buf, '\\n');\n+\t\tfputs(buf.buf, out);\n+\t}\n+\tstrbuf_release(&buf);\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex f885b68395f..83f2943b7a9 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -45,6 +45,9 @@ int sequencer_continue(struct replay_opts *opts);\n int sequencer_rollback(struct replay_opts *opts);\n int sequencer_remove_state(struct replay_opts *opts);\n \n+int sequencer_make_script(int keep_empty, FILE *out,\n+\t\tint argc, const char **argv);\n+\n extern const char sign_off_header[];\n \n void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318203","messageId":"fefd59fb626a9a0ddd6c9bc3a7a42d6d1016b4c3.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 03/10] rebase -i: remove useless indentation","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:31:42Z","receivedAt":"2017-04-28T21:32:10Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The commands used to be indented, and it is nice to look at, but when we\ntransform the SHA-1s, the indentation is removed. So let's do away with it.\n\nFor the moment, at least: when we will use the upcoming rebase--helper\nto transform the SHA-1s, we *will* keep the indentation and can\nreintroduce it. Yet, to be able to validate the rebase--helper against\nthe output of the current shell script version, we need to remove the\nextra indentation.\n\nSigned-off-by: Johannes Schindelin \n---\n git-rebase--interactive.sh | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 609e150d38f..c40b1fd1d2e 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -155,13 +155,13 @@ reschedule_last_action () {\n append_todo_help () {\n \tgettext \"\n Commands:\n- p, pick = use commit\n- r, reword = use commit, but edit the commit message\n- e, edit = use commit, but stop for amending\n- s, squash = use commit, but meld into previous commit\n- f, fixup = like \\\"squash\\\", but discard this commit's log message\n- x, exec = run command (the rest of the line) using shell\n- d, drop = remove commit\n+p, pick = use commit\n+r, reword = use commit, but edit the commit message\n+e, edit = use commit, but stop for amending\n+s, squash = use commit, but meld into previous commit\n+f, fixup = like \\\"squash\\\", but discard this commit's log message\n+x, exec = run command (the rest of the line) using shell\n+d, drop = remove commit\n \n These lines can be re-ordered; they are executed from top to bottom.\n \" | git stripspace --comment-lines >>\"$todo\"\n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318204","messageId":"82f415322e786479f9da2f882a2cbcef05bbe74b.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 04/10] rebase -i: do not invent onelines when expanding/collapsing SHA-1s","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:32:14Z","receivedAt":"2017-04-28T21:32:32Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To avoid problems with short SHA-1s that become non-unique during the\nrebase, we rewrite the todo script with short/long SHA-1s before and\nafter letting the user edit the script. Since SHA-1s are not intuitive\nfor humans, rebase -i also provides the onelines (commit message\nsubjects) in the script, purely for the user's convenience.\n\nIt is very possible to generate a todo script via different means than\nrebase -i and then to let rebase -i run with it; In this case, these\nonelines are not required.\n\nAnd this is where the expand/collapse machinery has a bug: it *expects*\nthat oneline, and failing to find one reuses the previous SHA-1 as\n\"oneline\".\n\nIt was most likely an oversight, and made implementation in the (quite\nlimiting) shell script language less convoluted. However, we are about\nto reimplement performance-critical parts in C (and due to spawning a\ngit.exe process for every single line of the todo script, the\nexpansion/collapsing of the SHA-1s *is* performance-hampering on\nWindows), therefore let's fix this bug to make cross-validation with the\nC version of that functionality possible.\n\nSigned-off-by: Johannes Schindelin \n---\n git-rebase--interactive.sh | 7 ++++++-\n 1 file changed, 6 insertions(+), 1 deletion(-)\n\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex c40b1fd1d2e..214af0372ba 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -760,7 +760,12 @@ transform_todo_ids () {\n \t\t\t;;\n \t\t*)\n \t\t\tsha1=$(git rev-parse --verify --quiet \"$@\" ${rest%%[\t ]*}) &&\n-\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n+\t\t\tif test \"a$rest\" = \"a${rest#*[\t ]}\"\n+\t\t\tthen\n+\t\t\t\trest=$sha1\n+\t\t\telse\n+\t\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n+\t\t\tfi\n \t\t\t;;\n \t\tesac\n \t\tprintf '%s\\n' \"$command${rest:+ }$rest\"\n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318205","messageId":"6cf1bcd39672474230b179a3d12c5eeab353ae0c.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 05/10] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:32:33Z","receivedAt":"2017-04-28T21:32:52Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This is crucial to improve performance on Windows, as the speed is now\nmostly dominated by the SHA-1 transformation (because it spawns a new\nrev-parse process for *every* line, and spawning processes is pretty\nslow from Git for Windows' MSYS2 Bash).\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 10 +++++++-\n git-rebase--interactive.sh | 27 ++--------------------\n sequencer.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 2 ++\n 4 files changed, 70 insertions(+), 26 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex 821058d452d..9444c8d6c60 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n \tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -24,6 +24,10 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\t\tABORT),\n \t\tOPT_CMDMODE(0, \"make-script\", &command,\n \t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n+\t\tOPT_CMDMODE(0, \"shorten-sha1s\", &command,\n+\t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n+\t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n+\t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n \t\tOPT_END()\n \t};\n \n@@ -42,5 +46,9 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!sequencer_remove_state(&opts);\n \tif (command == MAKE_SCRIPT && argc > 1)\n \t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n+\tif (command == SHORTEN_SHA1S && argc == 1)\n+\t\treturn !!transform_todo_ids(1);\n+\tif (command == EXPAND_SHA1S && argc == 1)\n+\t\treturn !!transform_todo_ids(0);\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 214af0372ba..82a1941c42c 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -750,35 +750,12 @@ skip_unnecessary_picks () {\n \t\tdie \"$(gettext \"Could not skip unnecessary pick commands\")\"\n }\n \n-transform_todo_ids () {\n-\twhile read -r command rest\n-\tdo\n-\t\tcase \"$command\" in\n-\t\t\"$comment_char\"* | exec)\n-\t\t\t# Be careful for oddball commands like 'exec'\n-\t\t\t# that do not have a SHA-1 at the beginning of $rest.\n-\t\t\t;;\n-\t\t*)\n-\t\t\tsha1=$(git rev-parse --verify --quiet \"$@\" ${rest%%[\t ]*}) &&\n-\t\t\tif test \"a$rest\" = \"a${rest#*[\t ]}\"\n-\t\t\tthen\n-\t\t\t\trest=$sha1\n-\t\t\telse\n-\t\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n-\t\t\tfi\n-\t\t\t;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$command${rest:+ }$rest\"\n-\tdone <\"$todo\" >\"$todo.new\" &&\n-\tmv -f \"$todo.new\" \"$todo\"\n-}\n-\n expand_todo_ids() {\n-\ttransform_todo_ids\n+\tgit rebase--helper --expand-sha1s\n }\n \n collapse_todo_ids() {\n-\ttransform_todo_ids --short\n+\tgit rebase--helper --shorten-sha1s\n }\n \n # Rearrange the todo list that has both \"pick sha1 msg\" and\ndiff --git a/sequencer.c b/sequencer.c\nindex 88819a1a2a9..201d45b1677 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2437,3 +2437,60 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \tstrbuf_release(&buf);\n \treturn 0;\n }\n+\n+\n+int transform_todo_ids(int shorten_sha1s)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tint fd, res, i;\n+\tFILE *out;\n+\n+\tstrbuf_reset(&todo_list.buf);\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0)\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\n+\tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\tif (res) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn error(_(\"unusable instruction sheet: '%s'\"), todo_file);\n+\t}\n+\n+\tout = fopen(todo_file, \"w\");\n+\tif (!out) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn error(_(\"unable to open '%s' for writing\"), todo_file);\n+\t}\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tint bol = item->offset_in_buf;\n+\t\tconst char *p = todo_list.buf.buf + bol;\n+\t\tint eol = i + 1 < todo_list.nr ?\n+\t\t\ttodo_list.items[i + 1].offset_in_buf :\n+\t\t\ttodo_list.buf.len;\n+\n+\t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n+\t\t\tfwrite(p, eol - bol, 1, out);\n+\t\telse {\n+\t\t\tconst char *sha1 = shorten_sha1s ?\n+\t\t\t\tshort_commit_name(item->commit) :\n+\t\t\t\toid_to_hex(&item->commit->object.oid);\n+\t\t\tint len;\n+\n+\t\t\tp += strspn(p, \" \\t\"); /* left-trim command */\n+\t\t\tlen = strcspn(p, \" \\t\"); /* length of command */\n+\n+\t\t\tfprintf(out, \"%.*s %s %.*s\\n\",\n+\t\t\t\tlen, p, sha1, item->arg_len, item->arg);\n+\t\t}\n+\t}\n+\tfclose(out);\n+\ttodo_list_release(&todo_list);\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 83f2943b7a9..47a81034e76 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -48,6 +48,8 @@ int sequencer_remove_state(struct replay_opts *opts);\n int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv);\n \n+int transform_todo_ids(int shorten_sha1s);\n+\n extern const char sign_off_header[];\n \n void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318206","messageId":"f43af06e50eaeb93289692ca64d706a0be769bb9.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 06/10] t3404: relax rebase.missingCommitsCheck tests","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:32:38Z","receivedAt":"2017-04-28T21:32:57Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"These tests were a bit anal about the *exact* warning/error message\nprinted by git rebase. But those messages are intended for the *end\nuser*, therefore it does not make sense to test so rigidly for the\n*exact* wording.\n\nIn the following, we will reimplement the missing commits check in\nthe sequencer, with slightly different words.\n\nSo let's just test for the parts in the warning/error message that\nwe *really* care about, nothing more, nothing less.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3404-rebase-interactive.sh | 22 ++++------------------\n 1 file changed, 4 insertions(+), 18 deletions(-)\n\ndiff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh\nindex 33d392ba112..61113be08a4 100755\n--- a/t/t3404-rebase-interactive.sh\n+++ b/t/t3404-rebase-interactive.sh\n@@ -1242,20 +1242,13 @@ test_expect_success 'rebase -i respects rebase.missingCommitsCheck = error' '\n \ttest B = $(git cat-file commit HEAD^ | sed -ne \\$p)\n '\n \n-cat >expect <actual &&\n-\ttest_i18ncmp expect actual &&\n+\ttest_i18ngrep \"badcmd $(git rev-list --oneline -1 master~1)\" actual &&\n+\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n \tFAKE_LINES=\"1 2 3 drop 4 5\" git rebase --edit-todo &&\n \tgit rebase --continue &&\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p) &&\n@@ -1277,20 +1270,13 @@ test_expect_success 'tabs and spaces are accepted in the todolist' '\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n '\n \n-cat >expect <actual &&\n-\ttest_i18ncmp expect actual &&\n+\ttest_i18ngrep \"edit XXXXXXX False commit\" actual &&\n+\ttest_i18ngrep \"You can fix this with .git rebase --edit-todo..\" actual &&\n \tFAKE_LINES=\"1 2 4 5 6\" git rebase --edit-todo &&\n \tgit rebase --continue &&\n \ttest E = $(git cat-file commit HEAD | sed -ne \\$p)\n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318207","messageId":"feceef9a82dbb198e965074b518db6170f5c8892.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 07/10] rebase -i: check for missing commits in the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:32:45Z","receivedAt":"2017-04-28T21:33:03Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"In particular on Windows, where shell scripts are even more expensive\nthan on MacOSX or Linux, it makes sense to move a loop that forks\nGit at least once for every line in the todo list into a builtin.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 7 +-\n git-rebase--interactive.sh | 164 ++-------------------------------------------\n sequencer.c | 122 +++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n 4 files changed, 134 insertions(+), 160 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex 9444c8d6c60..e706eac710d 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -13,7 +13,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n \tint keep_empty = 0;\n \tenum {\n-\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n+\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n+\t\tCHECK_TODO_LIST\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -28,6 +29,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n \t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n \t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n+\t\tOPT_CMDMODE(0, \"check-todo-list\", &command,\n+\t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n \t\tOPT_END()\n \t};\n \n@@ -50,5 +53,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!transform_todo_ids(1);\n \tif (command == EXPAND_SHA1S && argc == 1)\n \t\treturn !!transform_todo_ids(0);\n+\tif (command == CHECK_TODO_LIST && argc == 1)\n+\t\treturn !!check_todo_list();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 82a1941c42c..08168a0d46b 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -867,96 +867,6 @@ add_exec_commands () {\n \tmv \"$1.new\" \"$1\"\n }\n \n-# Check if the SHA-1 passed as an argument is a\n-# correct one, if not then print $2 in \"$todo\".badsha\n-# $1: the SHA-1 to test\n-# $2: the line number of the input\n-# $3: the input filename\n-check_commit_sha () {\n-\tbadsha=0\n-\tif test -z \"$1\"\n-\tthen\n-\t\tbadsha=1\n-\telse\n-\t\tsha1_verif=\"$(git rev-parse --verify --quiet $1^{commit})\"\n-\t\tif test -z \"$sha1_verif\"\n-\t\tthen\n-\t\t\tbadsha=1\n-\t\tfi\n-\tfi\n-\n-\tif test $badsha -ne 0\n-\tthen\n-\t\tline=\"$(sed -n -e \"${2}p\" \"$3\")\"\n-\t\twarn \"$(eval_gettext \"\\\n-Warning: the SHA-1 is missing or isn't a commit in the following line:\n- - \\$line\")\"\n-\t\twarn\n-\tfi\n-\n-\treturn $badsha\n-}\n-\n-# prints the bad commits and bad commands\n-# from the todolist in stdin\n-check_bad_cmd_and_sha () {\n-\tretval=0\n-\tlineno=0\n-\twhile read -r command rest\n-\tdo\n-\t\tlineno=$(( $lineno + 1 ))\n-\t\tcase $command in\n-\t\t\"$comment_char\"*|''|noop|x|exec)\n-\t\t\t# Doesn't expect a SHA-1\n-\t\t\t;;\n-\t\t\"$cr\")\n-\t\t\t# Work around CR left by \"read\" (e.g. with Git for\n-\t\t\t# Windows' Bash).\n-\t\t\t;;\n-\t\tpick|p|drop|d|reword|r|edit|e|squash|s|fixup|f)\n-\t\t\tif ! check_commit_sha \"${rest%%[ \t]*}\" \"$lineno\" \"$1\"\n-\t\t\tthen\n-\t\t\t\tretval=1\n-\t\t\tfi\n-\t\t\t;;\n-\t\t*)\n-\t\t\tline=\"$(sed -n -e \"${lineno}p\" \"$1\")\"\n-\t\t\twarn \"$(eval_gettext \"\\\n-Warning: the command isn't recognized in the following line:\n- - \\$line\")\"\n-\t\t\twarn\n-\t\t\tretval=1\n-\t\t\t;;\n-\t\tesac\n-\tdone <\"$1\"\n-\treturn $retval\n-}\n-\n-# Print the list of the SHA-1 of the commits\n-# from stdin to stdout\n-todo_list_to_sha_list () {\n-\tgit stripspace --strip-comments |\n-\twhile read -r command sha1 rest\n-\tdo\n-\t\tcase $command in\n-\t\t\"$comment_char\"*|''|noop|x|\"exec\")\n-\t\t\t;;\n-\t\t*)\n-\t\t\tlong_sha=$(git rev-list --no-walk \"$sha1\" 2>/dev/null)\n-\t\t\tprintf \"%s\\n\" \"$long_sha\"\n-\t\t\t;;\n-\t\tesac\n-\tdone\n-}\n-\n-# Use warn for each line in stdin\n-warn_lines () {\n-\twhile read -r line\n-\tdo\n-\t\twarn \" - $line\"\n-\tdone\n-}\n-\n # Switch to the branch in $into and notify it in the reflog\n checkout_onto () {\n \tGIT_REFLOG_ACTION=\"$GIT_REFLOG_ACTION: checkout $onto_name\"\n@@ -971,74 +881,6 @@ get_missing_commit_check_level () {\n \tprintf '%s' \"$check_level\" | tr 'A-Z' 'a-z'\n }\n \n-# Check if the user dropped some commits by mistake\n-# Behaviour determined by rebase.missingCommitsCheck.\n-# Check if there is an unrecognized command or a\n-# bad SHA-1 in a command.\n-check_todo_list () {\n-\traise_error=f\n-\n-\tcheck_level=$(get_missing_commit_check_level)\n-\n-\tcase \"$check_level\" in\n-\twarn|error)\n-\t\t# Get the SHA-1 of the commits\n-\t\ttodo_list_to_sha_list <\"$todo\".backup >\"$todo\".oldsha1\n-\t\ttodo_list_to_sha_list <\"$todo\" >\"$todo\".newsha1\n-\n-\t\t# Sort the SHA-1 and compare them\n-\t\tsort -u \"$todo\".oldsha1 >\"$todo\".oldsha1+\n-\t\tmv \"$todo\".oldsha1+ \"$todo\".oldsha1\n-\t\tsort -u \"$todo\".newsha1 >\"$todo\".newsha1+\n-\t\tmv \"$todo\".newsha1+ \"$todo\".newsha1\n-\t\tcomm -2 -3 \"$todo\".oldsha1 \"$todo\".newsha1 >\"$todo\".miss\n-\n-\t\t# Warn about missing commits\n-\t\tif test -s \"$todo\".miss\n-\t\tthen\n-\t\t\ttest \"$check_level\" = error && raise_error=t\n-\n-\t\t\twarn \"$(gettext \"\\\n-Warning: some commits may have been dropped accidentally.\n-Dropped commits (newer to older):\")\"\n-\n-\t\t\t# Make the list user-friendly and display\n-\t\t\topt=\"--no-walk=sorted --format=oneline --abbrev-commit --stdin\"\n-\t\t\tgit rev-list $opt <\"$todo\".miss | warn_lines\n-\n-\t\t\twarn \"$(gettext \"\\\n-To avoid this message, use \\\"drop\\\" to explicitly remove a commit.\n-\n-Use 'git config rebase.missingCommitsCheck' to change the level of warnings.\n-The possible behaviours are: ignore, warn, error.\")\"\n-\t\t\twarn\n-\t\tfi\n-\t\t;;\n-\tignore)\n-\t\t;;\n-\t*)\n-\t\twarn \"$(eval_gettext \"Unrecognized setting \\$check_level for option rebase.missingCommitsCheck. Ignoring.\")\"\n-\t\t;;\n-\tesac\n-\n-\tif ! check_bad_cmd_and_sha \"$todo\"\n-\tthen\n-\t\traise_error=t\n-\tfi\n-\n-\tif test $raise_error = t\n-\tthen\n-\t\t# Checkout before the first commit of the\n-\t\t# rebase: this way git rebase --continue\n-\t\t# will work correctly as it expects HEAD to be\n-\t\t# placed before the commit of the next action\n-\t\tcheckout_onto\n-\n-\t\twarn \"$(gettext \"You can fix this with 'git rebase --edit-todo' and then run 'git rebase --continue'.\")\"\n-\t\tdie \"$(gettext \"Or you can abort the rebase with 'git rebase --abort'.\")\"\n-\tfi\n-}\n-\n # The whole contents of this file is run by dot-sourcing it from\n # inside a shell function. It used to be that \"return\"s we see\n # below were not inside any function, and expected to return\n@@ -1299,7 +1141,11 @@ git_sequence_editor \"$todo\" ||\n has_action \"$todo\" ||\n \treturn 2\n \n-check_todo_list\n+git rebase--helper --check-todo-list || {\n+\tret=$?\n+\tcheckout_onto\n+\texit $ret\n+}\n \n expand_todo_ids\n \ndiff --git a/sequencer.c b/sequencer.c\nindex 201d45b1677..4535ba9d12f 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2494,3 +2494,125 @@ int transform_todo_ids(int shorten_sha1s)\n \ttodo_list_release(&todo_list);\n \treturn 0;\n }\n+\n+enum check_level {\n+\tCHECK_IGNORE = 0, CHECK_WARN, CHECK_ERROR\n+};\n+\n+static enum check_level get_missing_commit_check_level(void)\n+{\n+\tconst char *value;\n+\n+\tif (git_config_get_value(\"rebase.missingcommitscheck\", &value) ||\n+\t\t\t!strcasecmp(\"ignore\", value))\n+\t\treturn CHECK_IGNORE;\n+\tif (!strcasecmp(\"warn\", value))\n+\t\treturn CHECK_WARN;\n+\tif (!strcasecmp(\"error\", value))\n+\t\treturn CHECK_ERROR;\n+\twarning(_(\"unrecognized setting %s for option\"\n+\t\t \"rebase.missingCommitsCheck. Ignoring.\"), value);\n+\treturn CHECK_IGNORE;\n+}\n+\n+/*\n+ * Check if the user dropped some commits by mistake\n+ * Behaviour determined by rebase.missingCommitsCheck.\n+ * Check if there is an unrecognized command or a\n+ * bad SHA-1 in a command.\n+ */\n+int check_todo_list(void)\n+{\n+\tenum check_level check_level = get_missing_commit_check_level();\n+\tstruct strbuf todo_file = STRBUF_INIT;\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct strbuf missing = STRBUF_INIT;\n+\tint advise_to_edit_todo = 0, res = 0, fd, i;\n+\n+\tstrbuf_addstr(&todo_file, rebase_path_todo());\n+\tfd = open(todo_file.buf, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tclose(fd);\n+\tadvise_to_edit_todo = res =\n+\t\tparse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\n+\tif (res || check_level == CHECK_IGNORE)\n+\t\tgoto leave_check;\n+\n+\t/* Mark the commits in git-rebase-todo as seen */\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct commit *commit = todo_list.items[i].commit;\n+\t\tif (commit)\n+\t\t\tcommit->util = (void *)1;\n+\t}\n+\n+\ttodo_list_release(&todo_list);\n+\tstrbuf_addstr(&todo_file, \".backup\");\n+\tfd = open(todo_file.buf, O_RDONLY);\n+\tif (fd < 0) {\n+\t\tres = error_errno(_(\"could not open '%s'\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\tres = error(_(\"could not read '%s'.\"), todo_file.buf);\n+\t\tgoto leave_check;\n+\t}\n+\tclose(fd);\n+\tstrbuf_release(&todo_file);\n+\tres = !!parse_insn_buffer(todo_list.buf.buf, &todo_list);\n+\n+\t/* Find commits in git-rebase-todo.backup yet unseen */\n+\tfor (i = todo_list.nr - 1; i >= 0; i--) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tstruct commit *commit = item->commit;\n+\t\tif (commit && !commit->util) {\n+\t\t\tstrbuf_addf(&missing, \" - %s %.*s\\n\",\n+\t\t\t\t short_commit_name(commit),\n+\t\t\t\t item->arg_len, item->arg);\n+\t\t\tcommit->util = (void *)1;\n+\t\t}\n+\t}\n+\n+\t/* Warn about missing commits */\n+\tif (!missing.len)\n+\t\tgoto leave_check;\n+\n+\tif (check_level == CHECK_ERROR)\n+\t\tadvise_to_edit_todo = res = 1;\n+\n+\tfprintf(stderr,\n+\t\t_(\"Warning: some commits may have been dropped accidentally.\\n\"\n+\t\t\"Dropped commits (newer to older):\\n\"));\n+\n+\t/* Make the list user-friendly and display */\n+\tfputs(missing.buf, stderr);\n+\tstrbuf_release(&missing);\n+\n+\tfprintf(stderr, _(\"To avoid this message, use \\\"drop\\\" to \"\n+\t\t\"explicitly remove a commit.\\n\\n\"\n+\t\t\"Use 'git config rebase.missingCommitsCheck' to change \"\n+\t\t\"the level of warnings.\\n\"\n+\t\t\"The possible behaviours are: ignore, warn, error.\\n\\n\"));\n+\n+leave_check:\n+\tstrbuf_release(&todo_file);\n+\ttodo_list_release(&todo_list);\n+\n+\tif (advise_to_edit_todo)\n+\t\tfprintf(stderr,\n+\t\t\t_(\"You can fix this with 'git rebase --edit-todo' \"\n+\t\t\t \"and then run 'git rebase --continue'.\\n\"\n+\t\t\t \"Or you can abort the rebase with 'git rebase\"\n+\t\t\t \" --abort'.\\n\"));\n+\n+\treturn res;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 47a81034e76..4978a61b83b 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -49,6 +49,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \t\tint argc, const char **argv);\n \n int transform_todo_ids(int shorten_sha1s);\n+int check_todo_list(void);\n \n extern const char sign_off_header[];\n \n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318208","messageId":"3da0bb88f6a4c34d36d457d4118082f7ac6cfbc6.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 08/10] rebase -i: skip unnecessary picks using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:32:49Z","receivedAt":"2017-04-28T21:33:07Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"In particular on Windows, where shell scripts are even more expensive\nthan on MacOSX or Linux, it makes sense to move a loop that forks\nGit at least once for every line in the todo list into a builtin.\n\nNote: The original code did not try to skip unnecessary picks of root\ncommits but punts instead (probably --root was not considered common\nenough of a use case to bother optimizing). We do the same, for now.\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/rebase--helper.c | 6 ++-\n git-rebase--interactive.sh | 41 ++---------------\n sequencer.c | 107 +++++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n 4 files changed, 116 insertions(+), 39 deletions(-)\n\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex e706eac710d..de3ccd9bfbc 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tint keep_empty = 0;\n \tenum {\n \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n-\t\tCHECK_TODO_LIST\n+\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -31,6 +31,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n \t\tOPT_CMDMODE(0, \"check-todo-list\", &command,\n \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n+\t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n+\t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n \t\tOPT_END()\n \t};\n \n@@ -55,5 +57,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!transform_todo_ids(0);\n \tif (command == CHECK_TODO_LIST && argc == 1)\n \t\treturn !!check_todo_list();\n+\tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n+\t\treturn !!skip_unnecessary_picks();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 08168a0d46b..92e3ca1de3b 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -713,43 +713,6 @@ do_rest () {\n \tdone\n }\n \n-# skip picking commits whose parents are unchanged\n-skip_unnecessary_picks () {\n-\tfd=3\n-\twhile read -r command rest\n-\tdo\n-\t\t# fd=3 means we skip the command\n-\t\tcase \"$fd,$command\" in\n-\t\t3,pick|3,p)\n-\t\t\t# pick a commit whose parent is current $onto -> skip\n-\t\t\tsha1=${rest%% *}\n-\t\t\tcase \"$(git rev-parse --verify --quiet \"$sha1\"^)\" in\n-\t\t\t\"$onto\"*)\n-\t\t\t\tonto=$sha1\n-\t\t\t\t;;\n-\t\t\t*)\n-\t\t\t\tfd=1\n-\t\t\t\t;;\n-\t\t\tesac\n-\t\t\t;;\n-\t\t3,\"$comment_char\"*|3,)\n-\t\t\t# copy comments\n-\t\t\t;;\n-\t\t*)\n-\t\t\tfd=1\n-\t\t\t;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$command${rest:+ }$rest\" >&$fd\n-\tdone <\"$todo\" >\"$todo.new\" 3>>\"$done\" &&\n-\tmv -f \"$todo\".new \"$todo\" &&\n-\tcase \"$(peek_next_command)\" in\n-\tsquash|s|fixup|f)\n-\t\trecord_in_rewritten \"$onto\"\n-\t\t;;\n-\tesac ||\n-\t\tdie \"$(gettext \"Could not skip unnecessary pick commands\")\"\n-}\n-\n expand_todo_ids() {\n \tgit rebase--helper --expand-sha1s\n }\n@@ -1149,7 +1112,9 @@ git rebase--helper --check-todo-list || {\n \n expand_todo_ids\n \n-test -d \"$rewritten\" || test -n \"$force_rebase\" || skip_unnecessary_picks\n+test -d \"$rewritten\" || test -n \"$force_rebase\" ||\n+onto=\"$(git rebase--helper --skip-unnecessary-picks)\" ||\n+die \"Could not skip unnecessary pick commands\"\n \n checkout_onto\n if test -z \"$rebase_root\" && test ! -d \"$rewritten\"\ndiff --git a/sequencer.c b/sequencer.c\nindex 4535ba9d12f..72e3ad8d145 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -2616,3 +2616,110 @@ int check_todo_list(void)\n \n \treturn res;\n }\n+\n+/* skip picking commits whose parents are unchanged */\n+int skip_unnecessary_picks(void)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct object_id onto_oid, *oid = &onto_oid, *parent_oid;\n+\tint fd, i;\n+\n+\tif (!read_oneliner(&buf, rebase_path_onto(), 0))\n+\t\treturn error(_(\"could not read 'onto'\"));\n+\tif (get_sha1(buf.buf, onto_oid.hash)) {\n+\t\tstrbuf_release(&buf);\n+\t\treturn error(_(\"need a HEAD to fixup\"));\n+\t}\n+\tstrbuf_release(&buf);\n+\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0) {\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\t}\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn -1;\n+\t}\n+\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\n+\t\tif (item->command >= TODO_NOOP)\n+\t\t\tcontinue;\n+\t\tif (item->command != TODO_PICK)\n+\t\t\tbreak;\n+\t\tif (parse_commit(item->commit)) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error(_(\"could not parse commit '%s'\"),\n+\t\t\t\toid_to_hex(&item->commit->object.oid));\n+\t\t}\n+\t\tif (!item->commit->parents)\n+\t\t\tbreak; /* root commit */\n+\t\tif (item->commit->parents->next)\n+\t\t\tbreak; /* merge commit */\n+\t\tparent_oid = &item->commit->parents->item->object.oid;\n+\t\tif (hashcmp(parent_oid->hash, oid->hash))\n+\t\t\tbreak;\n+\t\toid = &item->commit->object.oid;\n+\t}\n+\tif (i > 0) {\n+\t\tint offset = i < todo_list.nr ?\n+\t\t\ttodo_list.items[i].offset_in_buf : todo_list.buf.len;\n+\t\tconst char *done_path = rebase_path_done();\n+\n+\t\tfd = open(done_path, O_CREAT | O_WRONLY | O_APPEND, 0666);\n+\t\tif (fd < 0) {\n+\t\t\terror_errno(_(\"could not open '%s' for writing\"),\n+\t\t\t\t done_path);\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tif (write_in_full(fd, todo_list.buf.buf, offset) < 0) {\n+\t\t\terror_errno(_(\"could not write to '%s'\"), done_path);\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\tclose(fd);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tclose(fd);\n+\n+\t\tfd = open(rebase_path_todo(), O_WRONLY, 0666);\n+\t\tif (fd < 0) {\n+\t\t\terror_errno(_(\"could not open '%s' for writing\"),\n+\t\t\t\t rebase_path_todo());\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tif (write_in_full(fd, todo_list.buf.buf + offset,\n+\t\t\t\ttodo_list.buf.len - offset) < 0) {\n+\t\t\terror_errno(_(\"could not write to '%s'\"),\n+\t\t\t\t rebase_path_todo());\n+\t\t\tclose(fd);\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tif (ftruncate(fd, todo_list.buf.len - offset) < 0) {\n+\t\t\terror_errno(_(\"could not truncate '%s'\"),\n+\t\t\t\t rebase_path_todo());\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\tclose(fd);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tclose(fd);\n+\n+\t\ttodo_list.current = i;\n+\t\tif (is_fixup(peek_command(&todo_list, 0)))\n+\t\t\trecord_in_rewritten(oid, peek_command(&todo_list, 0));\n+\t}\n+\n+\ttodo_list_release(&todo_list);\n+\tprintf(\"%s\\n\", oid_to_hex(oid));\n+\n+\treturn 0;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 4978a61b83b..28e1fc1e9bb 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -50,6 +50,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n \n int transform_todo_ids(int shorten_sha1s);\n int check_todo_list(void);\n+int skip_unnecessary_picks(void);\n \n extern const char sign_off_header[];\n \n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318209","messageId":"b28721e0c7875e88e15457aa532597e5d43e7279.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 09/10] t3415: test fixup with wrapped oneline","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:32:55Z","receivedAt":"2017-04-28T21:33:17Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"The `git commit --fixup` command unwraps wrapped onelines when\nconstructing the commit message, without wrapping the result.\n\nWe need to make sure that `git rebase --autosquash` keeps handling such\ncases correctly, in particular since we are about to move the autosquash\nhandling into the rebase--helper.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t3415-rebase-autosquash.sh | 14 ++++++++++++++\n 1 file changed, 14 insertions(+)\n\ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 6c37ebdff87..926bb3da788 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -316,4 +316,18 @@ test_expect_success 'extra spaces after fixup!' '\n \ttest $base = $parent\n '\n \n+test_expect_success 'wrapped original subject' '\n+\tif test -d .git/rebase-merge; then git rebase --abort; fi &&\n+\tbase=$(git rev-parse HEAD) &&\n+\techo \"wrapped subject\" >wrapped &&\n+\tgit add wrapped &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m \"$(printf \"To\\nfixup\")\" &&\n+\ttest_tick &&\n+\tgit commit --allow-empty -m \"fixup! To fixup\" &&\n+\tgit rebase -i --autosquash --keep-empty HEAD~2 &&\n+\tparent=$(git rev-parse HEAD^) &&\n+\ttest $base = $parent\n+'\n+\n test_done\n-- \n2.12.2.windows.2.800.gede8f145e06\n\n\n"},{"id":"318210","messageId":"6825a2bb46f43687af0565e4d38224d472932203.1493414945.git.johannes.schindelin@gmx.de","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 10/10] rebase -i: rearrange fixup/squash lines using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-04-28T21:33:07Z","receivedAt":"2017-04-28T21:33:26Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"This operation has quadratic complexity, which is especially painful\non Windows, where shell scripts are *already* slow (mainly due to the\noverhead of the POSIX emulation layer).\n\nLet's reimplement this with linear complexity (using a hash map to\nmatch the commits' subject lines) for the common case; Sadly, the\nfixup/squash feature's design neglected performance considerations,\nallowing arbitrary prefixes (read: `fixup! hell` will match the\ncommit subject `hello world`), which means that we are stuck with\nquadratic performance in the worst case.\n\nThe reimplemented logic also happens to fix a bug where commented-out\nlines (representing empty patches) were dropped by the previous code.\n\nWhile at it, clarify how the fixup/squash feature works in `git rebase\n-i`'s man page.\n\nSigned-off-by: Johannes Schindelin \n---\n Documentation/git-rebase.txt | 16 ++--\n builtin/rebase--helper.c | 6 +-\n git-rebase--interactive.sh | 90 +-------------------\n sequencer.c | 195 +++++++++++++++++++++++++++++++++++++++++++\n sequencer.h | 1 +\n t/t3415-rebase-autosquash.sh | 2 +-\n 6 files changed, 212 insertions(+), 98 deletions(-)\n\ndiff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt\nindex 53f4e144444..c5464aa5365 100644\n--- a/Documentation/git-rebase.txt\n+++ b/Documentation/git-rebase.txt\n@@ -430,13 +430,15 @@ without an explicit `--interactive`.\n --autosquash::\n --no-autosquash::\n \tWhen the commit log message begins with \"squash! ...\" (or\n-\t\"fixup! ...\"), and there is a commit whose title begins with\n-\tthe same ..., automatically modify the todo list of rebase -i\n-\tso that the commit marked for squashing comes right after the\n-\tcommit to be modified, and change the action of the moved\n-\tcommit from `pick` to `squash` (or `fixup`). Ignores subsequent\n-\t\"fixup! \" or \"squash! \" after the first, in case you referred to an\n-\tearlier fixup/squash with `git commit --fixup/--squash`.\n+\t\"fixup! ...\"), and there is already a commit in the todo list that\n+\tmatches the same `...`, automatically modify the todo list of rebase\n+\t-i so that the commit marked for squashing comes right after the\n+\tcommit to be modified, and change the action of the moved commit\n+\tfrom `pick` to `squash` (or `fixup`). A commit matches the `...` if\n+\tthe commit subject matches, or if the `...` refers to the commit's\n+\thash. As a fall-back, partial matches of the commit subject work,\n+\ttoo. The recommended way to create fixup/squash commits is by using\n+\tthe `--fixup`/`--squash` options of linkgit:git-commit[1].\n +\n This option is only valid when the `--interactive` option is used.\n +\ndiff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\nindex de3ccd9bfbc..e6591f01112 100644\n--- a/builtin/rebase--helper.c\n+++ b/builtin/rebase--helper.c\n@@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \tint keep_empty = 0;\n \tenum {\n \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n-\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n+\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH\n \t} command = 0;\n \tstruct option options[] = {\n \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n@@ -33,6 +33,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n \t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n \t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n+\t\tOPT_CMDMODE(0, \"rearrange-squash\", &command,\n+\t\t\tN_(\"rearrange fixup/squash lines\"), REARRANGE_SQUASH),\n \t\tOPT_END()\n \t};\n \n@@ -59,5 +61,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n \t\treturn !!check_todo_list();\n \tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n \t\treturn !!skip_unnecessary_picks();\n+\tif (command == REARRANGE_SQUASH && argc == 1)\n+\t\treturn !!rearrange_squash();\n \tusage_with_options(builtin_rebase_helper_usage, options);\n }\ndiff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\nindex 92e3ca1de3b..84c6e62518f 100644\n--- a/git-rebase--interactive.sh\n+++ b/git-rebase--interactive.sh\n@@ -721,94 +721,6 @@ collapse_todo_ids() {\n \tgit rebase--helper --shorten-sha1s\n }\n \n-# Rearrange the todo list that has both \"pick sha1 msg\" and\n-# \"pick sha1 fixup!/squash! msg\" appears in it so that the latter\n-# comes immediately after the former, and change \"pick\" to\n-# \"fixup\"/\"squash\".\n-#\n-# Note that if the config has specified a custom instruction format\n-# each log message will be re-retrieved in order to normalize the\n-# autosquash arrangement\n-rearrange_squash () {\n-\tformat=$(git config --get rebase.instructionFormat)\n-\t# extract fixup!/squash! lines and resolve any referenced sha1's\n-\twhile read -r pick sha1 message\n-\tdo\n-\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n-\t\tcase \"$message\" in\n-\t\t\"squash! \"*|\"fixup! \"*)\n-\t\t\taction=\"${message%%!*}\"\n-\t\t\trest=$message\n-\t\t\tprefix=\n-\t\t\t# skip all squash! or fixup! (but save for later)\n-\t\t\twhile :\n-\t\t\tdo\n-\t\t\t\tcase \"$rest\" in\n-\t\t\t\t\"squash! \"*|\"fixup! \"*)\n-\t\t\t\t\tprefix=\"$prefix${rest%%!*},\"\n-\t\t\t\t\trest=\"${rest#*! }\"\n-\t\t\t\t\t;;\n-\t\t\t\t*)\n-\t\t\t\t\tbreak\n-\t\t\t\t\t;;\n-\t\t\t\tesac\n-\t\t\tdone\n-\t\t\tprintf '%s %s %s %s\\n' \"$sha1\" \"$action\" \"$prefix\" \"$rest\"\n-\t\t\t# if it's a single word, try to resolve to a full sha1 and\n-\t\t\t# emit a second copy. This allows us to match on both message\n-\t\t\t# and on sha1 prefix\n-\t\t\tif test \"${rest#* }\" = \"$rest\"; then\n-\t\t\t\tfullsha=\"$(git rev-parse -q --verify \"$rest\" 2>/dev/null)\"\n-\t\t\t\tif test -n \"$fullsha\"; then\n-\t\t\t\t\t# prefix the action to uniquely identify this line as\n-\t\t\t\t\t# intended for full sha1 match\n-\t\t\t\t\techo \"$sha1 +$action $prefix $fullsha\"\n-\t\t\t\tfi\n-\t\t\tfi\n-\t\tesac\n-\tdone >\"$1.sq\" <\"$1\"\n-\ttest -s \"$1.sq\" || return\n-\n-\tused=\n-\twhile read -r pick sha1 message\n-\tdo\n-\t\tcase \" $used\" in\n-\t\t*\" $sha1 \"*) continue ;;\n-\t\tesac\n-\t\tprintf '%s\\n' \"$pick $sha1 $message\"\n-\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n-\t\tused=\"$used$sha1 \"\n-\t\twhile read -r squash action msg_prefix msg_content\n-\t\tdo\n-\t\t\tcase \" $used\" in\n-\t\t\t*\" $squash \"*) continue ;;\n-\t\t\tesac\n-\t\t\temit=0\n-\t\t\tcase \"$action\" in\n-\t\t\t+*)\n-\t\t\t\taction=\"${action#+}\"\n-\t\t\t\t# full sha1 prefix test\n-\t\t\t\tcase \"$msg_content\" in \"$sha1\"*) emit=1;; esac ;;\n-\t\t\t*)\n-\t\t\t\t# message prefix test\n-\t\t\t\tcase \"$message\" in \"$msg_content\"*) emit=1;; esac ;;\n-\t\t\tesac\n-\t\t\tif test $emit = 1; then\n-\t\t\t\tif test -n \"${format}\"\n-\t\t\t\tthen\n-\t\t\t\t\tmsg_content=$(git log -n 1 --format=\"${format}\" ${squash})\n-\t\t\t\telse\n-\t\t\t\t\tmsg_content=\"$(echo \"$msg_prefix\" | sed \"s/,/! /g\")$msg_content\"\n-\t\t\t\tfi\n-\t\t\t\tprintf '%s\\n' \"$action $squash $msg_content\"\n-\t\t\t\tused=\"$used$squash \"\n-\t\t\tfi\n-\t\tdone <\"$1.sq\"\n-\tdone >\"$1.rearranged\" <\"$1\"\n-\tcat \"$1.rearranged\" >\"$1\"\n-\trm -f \"$1.sq\" \"$1.rearranged\"\n-}\n-\n # Add commands after a pick or after a squash/fixup serie\n # in the todo list.\n add_exec_commands () {\n@@ -1068,7 +980,7 @@ then\n fi\n \n test -s \"$todo\" || echo noop >> \"$todo\"\n-test -n \"$autosquash\" && rearrange_squash \"$todo\"\n+test -z \"$autosquash\" || git rebase--helper --rearrange-squash || exit\n test -n \"$cmd\" && add_exec_commands \"$todo\"\n \n todocount=$(git stripspace --strip-comments <\"$todo\" | wc -l)\ndiff --git a/sequencer.c b/sequencer.c\nindex 72e3ad8d145..63a588f0916 100644\n--- a/sequencer.c\n+++ b/sequencer.c\n@@ -19,6 +19,7 @@\n #include \"trailer.h\"\n #include \"log-tree.h\"\n #include \"wt-status.h\"\n+#include \"hashmap.h\"\n \n #define GIT_REFLOG_ACTION \"GIT_REFLOG_ACTION\"\n \n@@ -2723,3 +2724,197 @@ int skip_unnecessary_picks(void)\n \n \treturn 0;\n }\n+\n+struct subject2item_entry {\n+\tstruct hashmap_entry entry;\n+\tint i;\n+\tchar subject[FLEX_ARRAY];\n+};\n+\n+static int subject2item_cmp(const struct subject2item_entry *a,\n+\tconst struct subject2item_entry *b, const void *key)\n+{\n+\treturn key ? strcmp(a->subject, key) : strcmp(a->subject, b->subject);\n+}\n+\n+/*\n+ * Rearrange the todo list that has both \"pick sha1 msg\" and \"pick sha1\n+ * fixup!/squash! msg\" in it so that the latter is put immediately after the\n+ * former, and change \"pick\" to \"fixup\"/\"squash\".\n+ *\n+ * Note that if the config has specified a custom instruction format, each log\n+ * message will have to be retrieved from the commit (as the oneline in the\n+ * script cannot be trusted) in order to normalize the autosquash arrangement.\n+ */\n+int rearrange_squash(void)\n+{\n+\tconst char *todo_file = rebase_path_todo();\n+\tstruct todo_list todo_list = TODO_LIST_INIT;\n+\tstruct hashmap subject2item;\n+\tint res = 0, rearranged = 0, *next, *tail, fd, i;\n+\tchar **subjects;\n+\n+\tfd = open(todo_file, O_RDONLY);\n+\tif (fd < 0)\n+\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n+\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n+\t\tclose(fd);\n+\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n+\t}\n+\tclose(fd);\n+\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n+\t\ttodo_list_release(&todo_list);\n+\t\treturn -1;\n+\t}\n+\n+\t/*\n+\t * The hashmap maps onelines to the respective todo list index.\n+\t *\n+\t * If any items need to be rearranged, the next[i] value will indicate\n+\t * which item was moved directly after the i'th.\n+\t *\n+\t * In that case, last[i] will indicate the index of the latest item to\n+\t * be moved to appear after the i'th.\n+\t */\n+\thashmap_init(&subject2item, (hashmap_cmp_fn) subject2item_cmp,\n+\t\t todo_list.nr);\n+\tALLOC_ARRAY(next, todo_list.nr);\n+\tALLOC_ARRAY(tail, todo_list.nr);\n+\tALLOC_ARRAY(subjects, todo_list.nr);\n+\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\t\tstruct todo_item *item = todo_list.items + i;\n+\t\tconst char *commit_buffer, *subject, *p;\n+\t\tsize_t subject_len;\n+\t\tint i2 = -1;\n+\t\tstruct subject2item_entry *entry;\n+\n+\t\tnext[i] = tail[i] = -1;\n+\t\tif (item->command >= TODO_EXEC) {\n+\t\t\tsubjects[i] = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (is_fixup(item->command)) {\n+\t\t\ttodo_list_release(&todo_list);\n+\t\t\treturn error(_(\"the script was already rearranged.\"));\n+\t\t}\n+\n+\t\titem->commit->util = item;\n+\n+\t\tparse_commit(item->commit);\n+\t\tcommit_buffer = get_commit_buffer(item->commit, NULL);\n+\t\tfind_commit_subject(commit_buffer, &subject);\n+\t\tformat_subject(&buf, subject, \" \");\n+\t\tsubject = subjects[i] = strbuf_detach(&buf, &subject_len);\n+\t\tunuse_commit_buffer(item->commit, commit_buffer);\n+\t\tif ((skip_prefix(subject, \"fixup! \", &p) ||\n+\t\t skip_prefix(subject, \"squash! \", &p))) {\n+\t\t\tstruct commit *commit2;\n+\n+\t\t\tfor (;;) {\n+\t\t\t\twhile (isspace(*p))\n+\t\t\t\t\tp++;\n+\t\t\t\tif (!skip_prefix(p, \"fixup! \", &p) &&\n+\t\t\t\t !skip_prefix(p, \"squash! \", &p))\n+\t\t\t\t\tbreak;\n+\t\t\t}\n+\n+\t\t\tif ((entry = hashmap_get_from_hash(&subject2item,\n+\t\t\t\t\t\t\t strhash(p), p)))\n+\t\t\t\t/* found by title */\n+\t\t\t\ti2 = entry->i;\n+\t\t\telse if (!strchr(p, ' ') &&\n+\t\t\t\t (commit2 =\n+\t\t\t\t lookup_commit_reference_by_name(p)) &&\n+\t\t\t\t commit2->util)\n+\t\t\t\t/* found by commit name */\n+\t\t\t\ti2 = (struct todo_item *)commit2->util\n+\t\t\t\t\t- todo_list.items;\n+\t\t\telse {\n+\t\t\t\t/* copy can be a prefix of the commit subject */\n+\t\t\t\tfor (i2 = 0; i2 < i; i2++)\n+\t\t\t\t\tif (subjects[i2] &&\n+\t\t\t\t\t starts_with(subjects[i2], p))\n+\t\t\t\t\t\tbreak;\n+\t\t\t\tif (i2 == i)\n+\t\t\t\t\ti2 = -1;\n+\t\t\t}\n+\t\t}\n+\t\tif (i2 >= 0) {\n+\t\t\trearranged = 1;\n+\t\t\ttodo_list.items[i].command =\n+\t\t\t\tstarts_with(subject, \"fixup!\") ?\n+\t\t\t\tTODO_FIXUP : TODO_SQUASH;\n+\t\t\tif (next[i2] < 0)\n+\t\t\t\tnext[i2] = i;\n+\t\t\telse\n+\t\t\t\tnext[tail[i2]] = i;\n+\t\t\ttail[i2] = i;\n+\t\t} else if (!hashmap_get_from_hash(&subject2item,\n+\t\t\t\t\t\tstrhash(subject), subject)) {\n+\t\t\tFLEX_ALLOC_MEM(entry, subject, subject, subject_len);\n+\t\t\tentry->i = i;\n+\t\t\thashmap_entry_init(entry, strhash(entry->subject));\n+\t\t\thashmap_put(&subject2item, entry);\n+\t\t}\n+\t}\n+\n+\tif (rearranged) {\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\n+\t\tfor (i = 0; i < todo_list.nr; i++) {\n+\t\t\tenum todo_command command = todo_list.items[i].command;\n+\t\t\tint cur = i;\n+\n+\t\t\t/*\n+\t\t\t * Initially, all commands are 'pick's. If it is a\n+\t\t\t * fixup or a squash now, we have rearranged it.\n+\t\t\t */\n+\t\t\tif (is_fixup(command))\n+\t\t\t\tcontinue;\n+\n+\t\t\twhile (cur >= 0) {\n+\t\t\t\tint offset = todo_list.items[cur].offset_in_buf;\n+\t\t\t\tint end_offset = cur + 1 < todo_list.nr ?\n+\t\t\t\t\ttodo_list.items[cur + 1].offset_in_buf :\n+\t\t\t\t\ttodo_list.buf.len;\n+\t\t\t\tchar *bol = todo_list.buf.buf + offset;\n+\t\t\t\tchar *eol = todo_list.buf.buf + end_offset;\n+\n+\t\t\t\t/* replace 'pick', by 'fixup' or 'squash' */\n+\t\t\t\tcommand = todo_list.items[cur].command;\n+\t\t\t\tif (is_fixup(command)) {\n+\t\t\t\t\tstrbuf_addstr(&buf,\n+\t\t\t\t\t\ttodo_command_info[command].str);\n+\t\t\t\t\tbol += strcspn(bol, \" \\t\");\n+\t\t\t\t}\n+\n+\t\t\t\tstrbuf_add(&buf, bol, eol - bol);\n+\n+\t\t\t\tcur = next[cur];\n+\t\t\t}\n+\t\t}\n+\n+\t\tfd = open(todo_file, O_WRONLY);\n+\t\tif (fd < 0)\n+\t\t\tres = error_errno(_(\"could not open '%s'\"), todo_file);\n+\t\telse if (write(fd, buf.buf, buf.len) < 0)\n+\t\t\tres = error_errno(_(\"could not read '%s'.\"), todo_file);\n+\t\telse if (ftruncate(fd, buf.len) < 0)\n+\t\t\tres = error_errno(_(\"could not finish '%s'\"),\n+\t\t\t\t\t todo_file);\n+\t\tclose(fd);\n+\t\tstrbuf_release(&buf);\n+\t}\n+\n+\tfree(next);\n+\tfree(tail);\n+\tfor (i = 0; i < todo_list.nr; i++)\n+\t\tfree(subjects[i]);\n+\tfree(subjects);\n+\thashmap_free(&subject2item, 1);\n+\ttodo_list_release(&todo_list);\n+\n+\treturn res;\n+}\ndiff --git a/sequencer.h b/sequencer.h\nindex 28e1fc1e9bb..1c94bec7622 100644\n--- a/sequencer.h\n+++ b/sequencer.h\n@@ -51,6 +51,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n int transform_todo_ids(int shorten_sha1s);\n int check_todo_list(void);\n int skip_unnecessary_picks(void);\n+int rearrange_squash(void);\n \n extern const char sign_off_header[];\n \ndiff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\nindex 926bb3da788..2f88f50c057 100755\n--- a/t/t3415-rebase-autosquash.sh\n+++ b/t/t3415-rebase-autosquash.sh\n@@ -290,7 +290,7 @@ set_backup_editor () {\n \ttest_set_editor \"$PWD/backup-editor.sh\"\n }\n \n-test_expect_failure 'autosquash with multiple empty patches' '\n+test_expect_success 'autosquash with multiple empty patches' '\n \ttest_tick &&\n \tgit commit --allow-empty -m \"empty\" &&\n \ttest_tick &&\n-- \n2.12.2.windows.2.800.gede8f145e06\n"},{"id":"318275","messageId":"xmqqtw55jta7.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"8c1f3519-0768-69d9-4d15-782da0be8390@talktalk.net","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-01T00:49:04Z","receivedAt":"2017-05-01T00:49:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood writes:\n\n> This changes the behaviour of\n> git -c rebase.instructionFormat= rebase -i\n> The shell version treats the rebase.instructionFormat being unset or set\n> to the empty string as equivalent. This version generates a todo list\n> with lines like 'pick ' rather than 'pick \n> '\n>\n> I only picked this up because I have a script that does 'git -c\n> rebase.instructionFormat= rebase -i' with a custom sequence editor.\n\nSorry to hear that. As there is no way to unset a configuration\nvariable from the command line, \"git -c var=\" like you did above is\nthe best we can do, and that why treating unset and empty variable\nthe same way is often necessary. It seems that Dscho gave an ack to\nyour message, so hopefully the final version would not have such a\nregression.\n\nThanks for an early warning.\n\n"},{"id":"318342","messageId":"xmqq4lx5i83q.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"alpine.DEB.2.20.1704281711540.3480@virtualbox","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-01T03:11:53Z","receivedAt":"2017-05-01T03:11:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> In that case, I would strongly advise to consider redesigning the API.\n\nThe API we currently have and is used by \"log\", \"rev-list\" and\nfriends is to have setup_revisions() parse the av[], i.e. the\ntextual API, and it is sufficient to hide from the caller the\nimplementation detail of what bit rev_info structure has and which\nbits are flipped when reacting to say \"--format=...\" option [*1*].\n\nAs the implementaiton detail of which bits are flipped when reacting\nto each options is _not_ the API, we are essentially left with two\nchoices: write this series to the current textual API, or invent an\nalternate API [*2*] and write this series to that new API.\n\nBesides, the original was already using the textual interface to\nset-up the revision traversal machinery (after all, it was a shell\nscript that invoked rev-list), and the series attempts a faithful\nrewrite of it in C; writing to the current textual API is a\nfuture-proof way to do so, and something you can do without waiting\nfor a new API to materialize (that is, assuming that we need an\nalternate API, favoured over the current textual API).\n\n\n[Footnote]\n\n*1* You'll notice that there already are (and were in 2010) users\n that cheated and peeked into the implementation detail by\n looking at unnecessary places the patch that added pretty_given\n bit; some of the places it needed to touch probably didn't have\n to be touched if they were writing to the API and had their av[]\n parsed.\n\n*2* Quite honestly, I do not get how much you would gain dumping the\n current API. It uses the same codepath \"git rev-list\" and\n friends use to parse the requests by the end-users and scripts,\n guaranteeing that it will stay stable, unlike the underlying\n implementation that may and will change. And the set-up of the\n machinery is not even the expensive part anyway.\n"},{"id":"318356","messageId":"42892d55-965d-af9d-fb80-518797d590c0@talktalk.net","threadId":"43983","inReplyTo":"alpine.DEB.2.20.1704282059100.3480@virtualbox","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Phillip Wood","fromEmail":"phillip.wood@talktalk.net","sentAt":"2017-05-01T10:06:19Z","receivedAt":"2017-05-01T10:06:28Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 28/04/17 20:22, Johannes Schindelin wrote:\n> Hi Philip,\n> \n> On Fri, 28 Apr 2017, Phillip Wood wrote:\n> \n>> On 26/04/17 12:59, Johannes Schindelin wrote:\n>>\n>>> The first step of an interactive rebase is to generate the so-called\n>>> \"todo script\", to be stored in the state directory as\n>>> \"git-rebase-todo\" and to be edited by the user.\n>>>\n>>> Originally, we adjusted the output of `git log ` using a\n>>> simple sed script. Over the course of the years, the code became more\n>>> complicated. We now use shell scripting to edit the output of `git\n>>> log` conditionally, depending whether to keep \"empty\" commits (i.e.\n>>> commits that do not change any files).\n>>>\n>>> On platforms where shell scripting is not native, this can be a\n>>> serious drag. And it opens the door for incompatibilities between\n>>> platforms when it comes to shell scripting or to Unix-y commands.\n>>>\n>>> Let's just re-implement the todo script generation in plain C, using\n>>> the revision machinery directly.\n>>>\n>>> This is substantially faster, improving the speed relative to the\n>>> shell script version of the interactive rebase from 2x to 3x on\n>>> Windows.\n>>\n>> This changes the behaviour of git -c rebase.instructionFormat= rebase -i\n>> The shell version treats the rebase.instructionFormat being unset or set\n>> to the empty string as equivalent. This version generates a todo list\n>> with lines like 'pick ' rather than 'pick \n>> '\n>>\n>> I only picked this up because I have a script that does 'git -c\n>> rebase.instructionFormat= rebase -i' with a custom sequence editor. I\n>> can easily add '%s' in the appropriate place but I thought I'd point it\n>> out in case other people are affected by the change.\n> \n> While I would argue that the C version is more correct, it would be\n> backwards-incompatible.\n\nI was going to make a point about resetting config variables to their\ndefault value on the command line but Junio beat me to it.\n\n> So I changed it.\n\nThat's great, thanks\n\n> BTW in the future you could help me a *lot* by providing a patch that adds\n> a test case to our test suite that not only demonstrates what exactly goes\n> wrong, but also will help prevent future regressions.\n\nI'll bear that in mind, it does assume that reporters have a good\nunderstanding of the test suite layout and helper functions though. Is\nthere a particular reason you put the test case in the autosquash tests?\nI wouldn't have thought of doing that.\n\nThanks again\n\nPhillip\n\n"},{"id":"318359","messageId":"alpine.DEB.2.20.1705011305570.3480@virtualbox","threadId":"43983","inReplyTo":"xmqqtw55jta7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-01T11:06:57Z","receivedAt":"2017-05-01T11:07:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Sun, 30 Apr 2017, Junio C Hamano wrote:\n\n> Phillip Wood writes:\n> \n> > This changes the behaviour of\n> > git -c rebase.instructionFormat= rebase -i\n> > The shell version treats the rebase.instructionFormat being unset or set\n> > to the empty string as equivalent. This version generates a todo list\n> > with lines like 'pick ' rather than 'pick \n> > '\n> >\n> > I only picked this up because I have a script that does 'git -c\n> > rebase.instructionFormat= rebase -i' with a custom sequence editor.\n> \n> Sorry to hear that. As there is no way to unset a configuration\n> variable from the command line, \"git -c var=\" like you did above is\n> the best we can do, and that why treating unset and empty variable\n> the same way is often necessary. It seems that Dscho gave an ack to\n> your message, so hopefully the final version would not have such a\n> regression.\n\nAs I mentioned in the cover letter of v4, unless you take v3 for the final\nversion, the final version won't have such a regression:\n\n> - the todo list is generated is again generated in the same way as\n> before when rebase.instructionFormat is empty: it was interpreted as\n> if it had not been set\n\nCiao,\nDscho\n"},{"id":"318363","messageId":"alpine.DEB.2.20.1705011307450.3480@virtualbox","threadId":"43983","inReplyTo":"xmqq4lx5i83q.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-01T11:47:57Z","receivedAt":"2017-05-01T11:49:30Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Sun, 30 Apr 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > In that case, I would strongly advise to consider redesigning the API.\n> \n> The API we currently have and is used by \"log\", \"rev-list\" and friends\n> is to have setup_revisions() parse the av[], i.e. the textual API, and\n> it is sufficient to hide from the caller the implementation detail of\n> what bit rev_info structure has and which bits are flipped when reacting\n> to say \"--format=...\" option [*1*].\n\nYes, this (parsing options passed in as strings, with the very real\npossibility of catching coding errors only at runtime) is the current way\nthe API is used.\n\nSometimes.\n\nAnd sometimes not.\n\nFor example, in builtin/bisect.c's show_diff_tree() function, we *do* call\nsetup_revisions(), but with argc = 0. We set all the options beforehand to\navoid parsing, to avoid runtime-instead-of-compile-time errors.\n\nThe same holds true for builtin/merge.c's squash_message() function.\n\n> As the implementaiton detail of which bits are flipped when reacting to\n> each options is _not_ the API, we are essentially left with two choices:\n> write this series to the current textual API, or invent an alternate API\n> [*2*] and write this series to that new API.\n\nYou make it sound as if my goal was to imitate slavishly what that option\ndid that was passed to the Git command in the git-rebase--interactive.sh\nscript.\n\nBut that is not what my goal is.\n\nMy goal is to imitate the *intent* of the shell script. Faithfully, of\ncourse. The fact that the shell script had no better way to access the\ninternal API than to call the Git command is just a red herring.\n\nWhat I really want to do *is* to access the revision machinery, as bare\nmetal as possible, because sequencer *is bare metal, too*.\n\nThe current code fulfills that goal rather excellently.\n\nYour suggested change to call the parser and pass plain options as plain\ntext really flies in the face of this goal.\n\nYour suggested alternative is actually not necessary here, as the code\ndoes exactly what it is supposed to do: it calls, from the internal\nlibgit.a, another part of libgit.a, and therefore it is totally legitimate\nto use implementation details.\n\nIf my code were bleeding implementation details to the user interface, I\nwould agree with you that there is an issue.\n\nBut that code does not do that. To the contrary, it hides those\nimplementation details behind a rather simple user interface.\n\nIn the long run, I think you are correct in your fear that bits may be set\nincorrectly.\n\nThe solution for that is, of course, not to rewrite the API. The solution\nis to make the API less fragile.\n\nTo be explicit about the fragility in question: the API should not require\nthe pretty_given bit at all, but it should use an initialized\npretty_print_context within the rev_info struct as indicator that the\npretty print machinery should be used to print out commit messages.\n\nThere is something even more fragile about the current concept of parsing\n--pretty: the fact that get_commit_format() sets a *file-local* variable\n`user_format`, and that that variable is then used for formatting when\npretty_given = 1, is just asking for trouble.\n\nThis fragile aspect of the API simply dooms the revision API to suffer\nside effects until fixed.\n\nAfter writing this, I really, really, really fail even more to see why you\nmake such a big deal out of the pretty_given bit. It is insignificant. If\nI were you, I would worry much, much, much, MUCH more about the fact that\n`user_format` in pretty.c is changed implicitly by sequencer_make_script()\n(not the fault of my patch, of course, but of the way get_commit_format()\noperates).\n\nObviously this latter issue (the `user_format` side effect) is what is a\nreal problem later on, when we try to make rebase a true builtin, as\nsequencer_make_script() will be called as part of a larger operation that\nwill subsequently run the rebase, and may very well use the revision\nmachinery to print other commit messages again, *possibly using that\n`user_format` by mistake*.\n\nNow, if your suggestion to undo the compile-time safety in favor of a\nruntime error, say, in case of a speling eror (which I really would like\nto avoid, as I find it a highly sloppy development style to turn compile\ntime errors into runtime errors for no good reason) would help avoid this\nproblem with the `user_format`, I would grudgingly bite my tongue,\nimplement what you suggested and move forward.\n\nBut it does not. All it does make the code less safe by pushing a possible\nproblem from the time of compilation to the time of running the code.\nMeaning that problems would be found by users, when developers could have\ncaught them without this change.\n\nI really wish we were on the same page that this is a really bad idea.\n\nCiao,\nDscho\n"},{"id":"318365","messageId":"alpine.DEB.2.20.1705011355040.3480@virtualbox","threadId":"43983","inReplyTo":"42892d55-965d-af9d-fb80-518797d590c0@talktalk.net","subject":"Re: [PATCH v3 1/9] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-01T11:58:00Z","receivedAt":"2017-05-01T12:00:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Phillip,\n\nOn Mon, 1 May 2017, Phillip Wood wrote:\n\n> On 28/04/17 20:22, Johannes Schindelin wrote:\n> \n> > BTW in the future you could help me a *lot* by providing a patch that\n> > adds a test case to our test suite that not only demonstrates what\n> > exactly goes wrong, but also will help prevent future regressions.\n> \n> I'll bear that in mind, it does assume that reporters have a good\n> understanding of the test suite layout and helper functions though.\n\nEven a shell script recreating the issue would be helpful, as it is easier\nto turn such a reproducer into a test case than to write the test case\nfrom scratch.\n\n> Is there a particular reason you put the test case in the autosquash\n> tests? I wouldn't have thought of doing that.\n\nYes. I looked for existing test cases setting rebase.instructionFormat.\nThat's where I put the new one.\n\n(I would also have avoided t3404, as it takes 5 minutes to run on Windows\ndue to its heavily-scripted nature: the shell script interpreter we use in\nGit for Windows jumps through all kinds of hoops to emulate POSIX\nfunctionality, and that costs time)\n\nCiao,\nJohannes\n"},{"id":"320784","messageId":"20170526031508.24838-1-liambeguin@gmail.com","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Liam Beguin","fromEmail":"liambeguin@gmail.com","sentAt":"2017-05-26T03:15:08Z","receivedAt":"2017-05-26T03:15:32Z","isPatch":true,"sender":{"key":"liambeguin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3811160?v=4"},"body":"Hi Johannes,\n\nJohannes Schindelin writes:\n> This patch series reimplements the expensive pre- and post-processing of\n> the todo script in C.\n>\n> And it concludes the work I did to accelerate rebase -i.\n>\n> Changes since v3:\n>\n> - removed the no-longer-used transform_todo_ids shell function\n>\n> - simplified transform_todo_ids()'s command parsing\n>\n> - fixed two commits in check_todo_list(), renamed the unclear\n> `raise_error` variable to `advise_to_edit_todo`, build the message\n> about missing commits directly (without the detour to building a\n> commit_list) and instead of assigning an unused pointer to commit->util\n> the code now uses (void *)1.\n>\n> - return early from check_todo_list() when parsing failed, even if the\n> check level is something else than CHECK_IGNORE\n>\n> - the todo list is generated is again generated in the same way as\n> before when rebase.instructionFormat is empty: it was interpreted as\n> if it had not been set\n>\n> - added a test for empty rebase.instructionFormat settings\n>\n>\n> Johannes Schindelin (10):\n> t3415: verify that an empty instructionFormat is handled as before\n> rebase -i: generate the script via rebase--helper\n> rebase -i: remove useless indentation\n> rebase -i: do not invent onelines when expanding/collapsing SHA-1s\n> rebase -i: also expand/collapse the SHA-1s via the rebase--helper\n> t3404: relax rebase.missingCommitsCheck tests\n> rebase -i: check for missing commits in the rebase--helper\n> rebase -i: skip unnecessary picks using the rebase--helper\n> t3415: test fixup with wrapped oneline\n> rebase -i: rearrange fixup/squash lines using the rebase--helper\n>\n> Documentation/git-rebase.txt | 16 +-\n> builtin/rebase--helper.c | 29 ++-\n> git-rebase--interactive.sh | 373 ++++-------------------------\n> sequencer.c | 530 ++++++++++++++++++++++++++++++++++++++++++\n> sequencer.h | 8 +\n> t/t3404-rebase-interactive.sh | 22 +-\n> t/t3415-rebase-autosquash.sh | 28 ++-\n> 7 files changed, 646 insertions(+), 360 deletions(-)\n>\n>\n> base-commit: 027a3b943b444a3e3a76f9a89803fc10245b858f\n> Based-On: rebase--helper at https://github.com/dscho/git\n> Fetch-Base-Via: git fetch https://github.com/dscho/git rebase--helper\n> Published-As: https://github.com/dscho/git/releases/tag/rebase-i-extra-v4\n> Fetch-It-Via: git fetch https://github.com/dscho/git rebase-i-extra-v4\n>\n\nThis is my first review so it's probably not the best you'll get, but\nhere it goes!\n\nI rebased the series ontop of v2.13.0 and run the whole `make test` on\nboth revisions.\nThe changes do not seem to have introduced any evident breakage as the\noutput of `make test` did not change.\n\nI tried to time the execution on an interactive rebase (on Linux) but\nI did not notice a significant change in speed.\nDo we have a way to measure performance / speed changes between version?\n\nLiam\n\n"},{"id":"320785","messageId":"20170526031516.26287-1-liambeguin@gmail.com","threadId":"43983","inReplyTo":"e173445d9a6321f80053967e50375400c0a61764.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Liam Beguin","fromEmail":"liambeguin@gmail.com","sentAt":"2017-05-26T03:15:16Z","receivedAt":"2017-05-26T03:15:45Z","isPatch":true,"sender":{"key":"liambeguin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3811160?v=4"},"body":"Hi Johannes,\n\nJohannes Schindelin writes:\n> The first step of an interactive rebase is to generate the so-called \"todo\n> script\", to be stored in the state directory as \"git-rebase-todo\" and to\n> be edited by the user.\n> \n> Originally, we adjusted the output of `git log ` using a simple\n> sed script. Over the course of the years, the code became more\n> complicated. We now use shell scripting to edit the output of `git log`\n> conditionally, depending whether to keep \"empty\" commits (i.e. commits\n> that do not change any files).\n> \n> On platforms where shell scripting is not native, this can be a serious\n> drag. And it opens the door for incompatibilities between platforms when\n> it comes to shell scripting or to Unix-y commands.\n> \n> Let's just re-implement the todo script generation in plain C, using the\n> revision machinery directly.\n> \n> This is substantially faster, improving the speed relative to the\n> shell script version of the interactive rebase from 2x to 3x on Windows.\n> \n> Note that the rearrange_squash() function in git-rebase--interactive\n> relied on the fact that we set the \"format\" variable to the config setting\n> rebase.instructionFormat. Relying on a side effect like this is no good,\n> hence we explicitly perform that assignment (possibly again) in\n> rearrange_squash().\n> \n> Signed-off-by: Johannes Schindelin \n> ---\n> builtin/rebase--helper.c | 8 +++++++-\n> git-rebase--interactive.sh | 44 +++++++++++++++++++++--------------------\n> sequencer.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++\n> sequencer.h | 3 +++\n> 4 files changed, 82 insertions(+), 22 deletions(-)\n> \n> diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\n> index ca1ebb2fa18..821058d452d 100644\n> --- a/builtin/rebase--helper.c\n> +++ b/builtin/rebase--helper.c\n> @@ -11,15 +11,19 @@ static const char * const builtin_rebase_helper_usage[] = {\n> int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> {\n> \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n> +\tint keep_empty = 0;\n> \tenum {\n> -\t\tCONTINUE = 1, ABORT\n> +\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n> \t} command = 0;\n> \tstruct option options[] = {\n> \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n> +\t\tOPT_BOOL(0, \"keep-empty\", &keep_empty, N_(\"keep empty commits\")),\n> \t\tOPT_CMDMODE(0, \"continue\", &command, N_(\"continue rebase\"),\n> \t\t\t\tCONTINUE),\n> \t\tOPT_CMDMODE(0, \"abort\", &command, N_(\"abort rebase\"),\n> \t\t\t\tABORT),\n> +\t\tOPT_CMDMODE(0, \"make-script\", &command,\n> +\t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n> \t\tOPT_END()\n> \t};\n> \n> @@ -36,5 +40,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> \t\treturn !!sequencer_continue(&opts);\n> \tif (command == ABORT && argc == 1)\n> \t\treturn !!sequencer_remove_state(&opts);\n> +\tif (command == MAKE_SCRIPT && argc > 1)\n> +\t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n> \tusage_with_options(builtin_rebase_helper_usage, options);\n> }\n> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n> index 2c9c0165b5a..609e150d38f 100644\n> --- a/git-rebase--interactive.sh\n> +++ b/git-rebase--interactive.sh\n> @@ -785,6 +785,7 @@ collapse_todo_ids() {\n> # each log message will be re-retrieved in order to normalize the\n> # autosquash arrangement\n> rearrange_squash () {\n> +\tformat=$(git config --get rebase.instructionFormat)\n> \t# extract fixup!/squash! lines and resolve any referenced sha1's\n> \twhile read -r pick sha1 message\n> \tdo\n> @@ -1210,26 +1211,27 @@ else\n> \trevisions=$onto...$orig_head\n> \tshortrevisions=$shorthead\n> fi\n> -format=$(git config --get rebase.instructionFormat)\n> -# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n> -git rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n> -\t--reverse --left-right --topo-order \\\n> -\t$revisions ${restrict_revision+^$restrict_revision} | \\\n> -\tsed -n \"s/^>//p\" |\n> -while read -r sha1 rest\n> -do\n> -\n> -\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n> -\tthen\n> -\t\tcomment_out=\"$comment_char \"\n> -\telse\n> -\t\tcomment_out=\n> -\tfi\n> +if test t != \"$preserve_merges\"\n> +then\n> +\tgit rebase--helper --make-script ${keep_empty:+--keep-empty} \\\n> +\t\t$revisions ${restrict_revision+^$restrict_revision} >\"$todo\"\n> +else\n> +\tformat=$(git config --get rebase.instructionFormat)\n> +\t# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n> +\tgit rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n> +\t\t--reverse --left-right --topo-order \\\n> +\t\t$revisions ${restrict_revision+^$restrict_revision} | \\\n> +\t\tsed -n \"s/^>//p\" |\n> +\twhile read -r sha1 rest\n> +\tdo\n> +\n> +\t\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n> +\t\tthen\n> +\t\t\tcomment_out=\"$comment_char \"\n> +\t\telse\n> +\t\t\tcomment_out=\n> +\t\tfi\n> \n> -\tif test t != \"$preserve_merges\"\n> -\tthen\n> -\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n> -\telse\n> \t\tif test -z \"$rebase_root\"\n> \t\tthen\n> \t\t\tpreserve=t\n> @@ -1248,8 +1250,8 @@ do\n> \t\t\ttouch \"$rewritten\"/$sha1\n> \t\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n> \t\tfi\n> -\tfi\n> -done\n> +\tdone\n> +fi\n> \n> # Watch for commits that been dropped by --cherry-pick\n> if test t = \"$preserve_merges\"\n> diff --git a/sequencer.c b/sequencer.c\n> index 130cc868e51..88819a1a2a9 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -2388,3 +2388,52 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> \n> \tstrbuf_release(&sob);\n> }\n> +\n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv)\n> +{\n> +\tchar *format = NULL;\n> +\tstruct pretty_print_context pp = {0};\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tstruct rev_info revs;\n> +\tstruct commit *commit;\n> +\n> +\tinit_revisions(&revs, NULL);\n> +\trevs.verbose_header = 1;\n> +\trevs.max_parents = 1;\n> +\trevs.cherry_pick = 1;\n> +\trevs.limited = 1;\n> +\trevs.reverse = 1;\n> +\trevs.right_only = 1;\n> +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> +\trevs.topo_order = 1;\n> +\n> +\trevs.pretty_given = 1;\n> +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n> +\tif (!format || !*format) {\n> +\t\tfree(format);\n> +\t\tformat = xstrdup(\"%s\");\n> +\t}\n> +\tget_commit_format(format, &revs);\n> +\tfree(format);\n> +\tpp.fmt = revs.commit_format;\n> +\tpp.output_encoding = get_log_output_encoding();\n> +\n> +\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n> +\t\treturn error(_(\"make_script: unhandled options\"));\n> +\n> +\tif (prepare_revision_walk(&revs) < 0)\n> +\t\treturn error(_(\"make_script: error preparing revisions\"));\n> +\n> +\twhile ((commit = get_revision(&revs))) {\n> +\t\tstrbuf_reset(&buf);\n> +\t\tif (!keep_empty && is_original_commit_empty(commit))\n> +\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n\nI've never had to use empty commits before, but while testing this, I\nnoticed that `git rebase -i --keep-empty` behaves differently if using\nthe --root option instead of a branch or something like 'HEAD~10'.\nI also tested this on v2.13.0 and the behavior is the same.\n\n> +\t\tstrbuf_addf(&buf, \"pick %s \", oid_to_hex(&commit->object.oid));\n> +\t\tpretty_print_commit(&pp, commit, &buf);\n> +\t\tstrbuf_addch(&buf, '\\n');\n> +\t\tfputs(buf.buf, out);\n> +\t}\n> +\tstrbuf_release(&buf);\n> +\treturn 0;\n> +}\n> diff --git a/sequencer.h b/sequencer.h\n> index f885b68395f..83f2943b7a9 100644\n> --- a/sequencer.h\n> +++ b/sequencer.h\n> @@ -45,6 +45,9 @@ int sequencer_continue(struct replay_opts *opts);\n> int sequencer_rollback(struct replay_opts *opts);\n> int sequencer_remove_state(struct replay_opts *opts);\n> \n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv);\n> +\n> extern const char sign_off_header[];\n> \n> void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n> -- \n> 2.12.2.windows.2.800.gede8f145e06\n\nLiam\n\n"},{"id":"320786","messageId":"20170526031524.27935-1-liambeguin@gmail.com","threadId":"43983","inReplyTo":"fefd59fb626a9a0ddd6c9bc3a7a42d6d1016b4c3.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 03/10] rebase -i: remove useless indentation","fromName":"Liam Beguin","fromEmail":"liambeguin@gmail.com","sentAt":"2017-05-26T03:15:24Z","receivedAt":"2017-05-26T03:15:50Z","isPatch":true,"sender":{"key":"liambeguin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3811160?v=4"},"body":"Hi Johannes,\n\nJohannes Schindelin writes:\n> The commands used to be indented, and it is nice to look at, but when we\n> transform the SHA-1s, the indentation is removed. So let's do away with it.\n> \n> For the moment, at least: when we will use the upcoming rebase--helper\n> to transform the SHA-1s, we *will* keep the indentation and can\n> reintroduce it. Yet, to be able to validate the rebase--helper against\n> the output of the current shell script version, we need to remove the\n> extra indentation.\n> \n> Signed-off-by: Johannes Schindelin \n> ---\n> git-rebase--interactive.sh | 14 +++++++-------\n> 1 file changed, 7 insertions(+), 7 deletions(-)\n> \n> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n> index 609e150d38f..c40b1fd1d2e 100644\n> --- a/git-rebase--interactive.sh\n> +++ b/git-rebase--interactive.sh\n> @@ -155,13 +155,13 @@ reschedule_last_action () {\n> append_todo_help () {\n> \tgettext \"\n> Commands:\n> - p, pick = use commit\n> - r, reword = use commit, but edit the commit message\n> - e, edit = use commit, but stop for amending\n> - s, squash = use commit, but meld into previous commit\n> - f, fixup = like \\\"squash\\\", but discard this commit's log message\n> - x, exec = run command (the rest of the line) using shell\n> - d, drop = remove commit\n> +p, pick = use commit\n> +r, reword = use commit, but edit the commit message\n> +e, edit = use commit, but stop for amending\n> +s, squash = use commit, but meld into previous commit\n> +f, fixup = like \\\"squash\\\", but discard this commit's log message\n> +x, exec = run command (the rest of the line) using shell\n> +d, drop = remove commit\n\ndo we also need to update all the translations since this is a `gettext`\nfunction?\n\n> \n> These lines can be re-ordered; they are executed from top to bottom.\n> \" | git stripspace --comment-lines >>\"$todo\"\n> -- \n> 2.12.2.windows.2.800.gede8f145e06\n\nLiam\n\n"},{"id":"320787","messageId":"20170526031530.29162-1-liambeguin@gmail.com","threadId":"43983","inReplyTo":"6cf1bcd39672474230b179a3d12c5eeab353ae0c.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 05/10] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Liam Beguin","fromEmail":"liambeguin@gmail.com","sentAt":"2017-05-26T03:15:30Z","receivedAt":"2017-05-26T03:15:55Z","isPatch":true,"sender":{"key":"liambeguin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3811160?v=4"},"body":"Hi Johannes,\n\nJohannes Schindelin writes:\n> This is crucial to improve performance on Windows, as the speed is now\n> mostly dominated by the SHA-1 transformation (because it spawns a new\n> rev-parse process for *every* line, and spawning processes is pretty\n> slow from Git for Windows' MSYS2 Bash).\n> \n> Signed-off-by: Johannes Schindelin \n> ---\n> builtin/rebase--helper.c | 10 +++++++-\n> git-rebase--interactive.sh | 27 ++--------------------\n> sequencer.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++\n> sequencer.h | 2 ++\n> 4 files changed, 70 insertions(+), 26 deletions(-)\n> \n> diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\n> index 821058d452d..9444c8d6c60 100644\n> --- a/builtin/rebase--helper.c\n> +++ b/builtin/rebase--helper.c\n> @@ -13,7 +13,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n> \tint keep_empty = 0;\n> \tenum {\n> -\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n> +\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S\n> \t} command = 0;\n> \tstruct option options[] = {\n> \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n> @@ -24,6 +24,10 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> \t\t\t\tABORT),\n> \t\tOPT_CMDMODE(0, \"make-script\", &command,\n> \t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n> +\t\tOPT_CMDMODE(0, \"shorten-sha1s\", &command,\n> +\t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n> +\t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n> +\t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n\nSince work is being done to convert to `struct object_id` would it\nnot be best to use a more generic name instead of 'sha1'?\nmaybe something like {shorten,expand}-hashs\n\n> \t\tOPT_END()\n> \t};\n> \n> @@ -42,5 +46,9 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> \t\treturn !!sequencer_remove_state(&opts);\n> \tif (command == MAKE_SCRIPT && argc > 1)\n> \t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n> +\tif (command == SHORTEN_SHA1S && argc == 1)\n> +\t\treturn !!transform_todo_ids(1);\n> +\tif (command == EXPAND_SHA1S && argc == 1)\n> +\t\treturn !!transform_todo_ids(0);\n> \tusage_with_options(builtin_rebase_helper_usage, options);\n> }\n> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n> index 214af0372ba..82a1941c42c 100644\n> --- a/git-rebase--interactive.sh\n> +++ b/git-rebase--interactive.sh\n> @@ -750,35 +750,12 @@ skip_unnecessary_picks () {\n> \t\tdie \"$(gettext \"Could not skip unnecessary pick commands\")\"\n> }\n> \n> -transform_todo_ids () {\n> -\twhile read -r command rest\n> -\tdo\n> -\t\tcase \"$command\" in\n> -\t\t\"$comment_char\"* | exec)\n> -\t\t\t# Be careful for oddball commands like 'exec'\n> -\t\t\t# that do not have a SHA-1 at the beginning of $rest.\n> -\t\t\t;;\n> -\t\t*)\n> -\t\t\tsha1=$(git rev-parse --verify --quiet \"$@\" ${rest%%[\t ]*}) &&\n> -\t\t\tif test \"a$rest\" = \"a${rest#*[\t ]}\"\n> -\t\t\tthen\n> -\t\t\t\trest=$sha1\n> -\t\t\telse\n> -\t\t\t\trest=\"$sha1 ${rest#*[\t ]}\"\n> -\t\t\tfi\n> -\t\t\t;;\n> -\t\tesac\n> -\t\tprintf '%s\\n' \"$command${rest:+ }$rest\"\n> -\tdone <\"$todo\" >\"$todo.new\" &&\n> -\tmv -f \"$todo.new\" \"$todo\"\n> -}\n> -\n> expand_todo_ids() {\n> -\ttransform_todo_ids\n> +\tgit rebase--helper --expand-sha1s\n> }\n> \n> collapse_todo_ids() {\n> -\ttransform_todo_ids --short\n> +\tgit rebase--helper --shorten-sha1s\n> }\n> \n> # Rearrange the todo list that has both \"pick sha1 msg\" and\n> diff --git a/sequencer.c b/sequencer.c\n> index 88819a1a2a9..201d45b1677 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -2437,3 +2437,60 @@ int sequencer_make_script(int keep_empty, FILE *out,\n> \tstrbuf_release(&buf);\n> \treturn 0;\n> }\n> +\n> +\n> +int transform_todo_ids(int shorten_sha1s)\n> +{\n> +\tconst char *todo_file = rebase_path_todo();\n> +\tstruct todo_list todo_list = TODO_LIST_INIT;\n> +\tint fd, res, i;\n> +\tFILE *out;\n> +\n> +\tstrbuf_reset(&todo_list.buf);\n> +\tfd = open(todo_file, O_RDONLY);\n> +\tif (fd < 0)\n> +\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n> +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> +\t\tclose(fd);\n> +\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n> +\t}\n> +\tclose(fd);\n> +\n> +\tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n> +\tif (res) {\n> +\t\ttodo_list_release(&todo_list);\n> +\t\treturn error(_(\"unusable instruction sheet: '%s'\"), todo_file);\n\nAs you pointed out last time, the name of the \"todo script\" can be a\nsource of confusion. The migration to C could be a good opportunity to\nclarify this.\nI don't know which is the preferred name but we could go with\n\"todo list\" as it is the most common across the code base.\n\n$ git grep 'todo[ -]list' | wc -l\n20\n$ git grep 'rebase[ -]script' | wc -l\n0\n$ git grep 'instruction[ -]list' | wc -l\n1\n$ git grep 'instruction[ -]sheet' | wc -l\n20\n$ git grep 'instruction[ -]sheet' | grep -v ^po | wc -l\n8\n\n> +\t}\n> +\n> +\tout = fopen(todo_file, \"w\");\n> +\tif (!out) {\n> +\t\ttodo_list_release(&todo_list);\n> +\t\treturn error(_(\"unable to open '%s' for writing\"), todo_file);\n> +\t}\n> +\tfor (i = 0; i < todo_list.nr; i++) {\n> +\t\tstruct todo_item *item = todo_list.items + i;\n> +\t\tint bol = item->offset_in_buf;\n> +\t\tconst char *p = todo_list.buf.buf + bol;\n> +\t\tint eol = i + 1 < todo_list.nr ?\n> +\t\t\ttodo_list.items[i + 1].offset_in_buf :\n> +\t\t\ttodo_list.buf.len;\n> +\n> +\t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n> +\t\t\tfwrite(p, eol - bol, 1, out);\n> +\t\telse {\n> +\t\t\tconst char *sha1 = shorten_sha1s ?\n> +\t\t\t\tshort_commit_name(item->commit) :\n> +\t\t\t\toid_to_hex(&item->commit->object.oid);\n\nWe could also use 'hash' or 'ids' here instead of 'sha1'.\n\n> +\t\t\tint len;\n> +\n> +\t\t\tp += strspn(p, \" \\t\"); /* left-trim command */\n> +\t\t\tlen = strcspn(p, \" \\t\"); /* length of command */\n> +\n> +\t\t\tfprintf(out, \"%.*s %s %.*s\\n\",\n> +\t\t\t\tlen, p, sha1, item->arg_len, item->arg);\n> +\t\t}\n> +\t}\n> +\tfclose(out);\n> +\ttodo_list_release(&todo_list);\n> +\treturn 0;\n> +}\n> diff --git a/sequencer.h b/sequencer.h\n> index 83f2943b7a9..47a81034e76 100644\n> --- a/sequencer.h\n> +++ b/sequencer.h\n> @@ -48,6 +48,8 @@ int sequencer_remove_state(struct replay_opts *opts);\n> int sequencer_make_script(int keep_empty, FILE *out,\n> \t\tint argc, const char **argv);\n> \n> +int transform_todo_ids(int shorten_sha1s);\n> +\n> extern const char sign_off_header[];\n> \n> void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n> -- \n> 2.12.2.windows.2.800.gede8f145e06\n\nLiam\n"},{"id":"320788","messageId":"20170526031639.10790-1-liambeguin@gmail.com","threadId":"43983","inReplyTo":"6825a2bb46f43687af0565e4d38224d472932203.1493414945.git.johannes.schindelin@gmx.de","subject":"[PATCH v4 10/10] rebase -i: rearrange fixup/squash lines using the rebase--helper","fromName":"Liam Beguin","fromEmail":"liambeguin@gmail.com","sentAt":"2017-05-26T03:16:39Z","receivedAt":"2017-05-26T03:16:56Z","isPatch":true,"sender":{"key":"liambeguin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3811160?v=4"},"body":"Hi Johannes,\n\nJohannes Schindelin writes:\n> This operation has quadratic complexity, which is especially painful\n> on Windows, where shell scripts are *already* slow (mainly due to the\n> overhead of the POSIX emulation layer).\n>\n> Let's reimplement this with linear complexity (using a hash map to\n> match the commits' subject lines) for the common case; Sadly, the\n> fixup/squash feature's design neglected performance considerations,\n> allowing arbitrary prefixes (read: `fixup! hell` will match the\n> commit subject `hello world`), which means that we are stuck with\n> quadratic performance in the worst case.\n>\n> The reimplemented logic also happens to fix a bug where commented-out\n> lines (representing empty patches) were dropped by the previous code.\n>\n> While at it, clarify how the fixup/squash feature works in `git rebase\n> -i`'s man page.\n>\n> Signed-off-by: Johannes Schindelin \n> ---\n> Documentation/git-rebase.txt | 16 ++--\n> builtin/rebase--helper.c | 6 +-\n> git-rebase--interactive.sh | 90 +-------------------\n> sequencer.c | 195 +++++++++++++++++++++++++++++++++++++++++++\n> sequencer.h | 1 +\n> t/t3415-rebase-autosquash.sh | 2 +-\n> 6 files changed, 212 insertions(+), 98 deletions(-)\n>\n> diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt\n> index 53f4e144444..c5464aa5365 100644\n> --- a/Documentation/git-rebase.txt\n> +++ b/Documentation/git-rebase.txt\n> @@ -430,13 +430,15 @@ without an explicit `--interactive`.\n> --autosquash::\n> --no-autosquash::\n> \tWhen the commit log message begins with \"squash! ...\" (or\n> -\t\"fixup! ...\"), and there is a commit whose title begins with\n> -\tthe same ..., automatically modify the todo list of rebase -i\n> -\tso that the commit marked for squashing comes right after the\n> -\tcommit to be modified, and change the action of the moved\n> -\tcommit from `pick` to `squash` (or `fixup`). Ignores subsequent\n> -\t\"fixup! \" or \"squash! \" after the first, in case you referred to an\n> -\tearlier fixup/squash with `git commit --fixup/--squash`.\n> +\t\"fixup! ...\"), and there is already a commit in the todo list that\n> +\tmatches the same `...`, automatically modify the todo list of rebase\n> +\t-i so that the commit marked for squashing comes right after the\n> +\tcommit to be modified, and change the action of the moved commit\n> +\tfrom `pick` to `squash` (or `fixup`). A commit matches the `...` if\n> +\tthe commit subject matches, or if the `...` refers to the commit's\n> +\thash. As a fall-back, partial matches of the commit subject work,\n> +\ttoo. The recommended way to create fixup/squash commits is by using\n> +\tthe `--fixup`/`--squash` options of linkgit:git-commit[1].\n> +\n> This option is only valid when the `--interactive` option is used.\n> +\n> diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\n> index de3ccd9bfbc..e6591f01112 100644\n> --- a/builtin/rebase--helper.c\n> +++ b/builtin/rebase--helper.c\n> @@ -14,7 +14,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> \tint keep_empty = 0;\n> \tenum {\n> \t\tCONTINUE = 1, ABORT, MAKE_SCRIPT, SHORTEN_SHA1S, EXPAND_SHA1S,\n> -\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS\n> +\t\tCHECK_TODO_LIST, SKIP_UNNECESSARY_PICKS, REARRANGE_SQUASH\n> \t} command = 0;\n> \tstruct option options[] = {\n> \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n> @@ -33,6 +33,8 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> \t\t\tN_(\"check the todo list\"), CHECK_TODO_LIST),\n> \t\tOPT_CMDMODE(0, \"skip-unnecessary-picks\", &command,\n> \t\t\tN_(\"skip unnecessary picks\"), SKIP_UNNECESSARY_PICKS),\n> +\t\tOPT_CMDMODE(0, \"rearrange-squash\", &command,\n> +\t\t\tN_(\"rearrange fixup/squash lines\"), REARRANGE_SQUASH),\n> \t\tOPT_END()\n> \t};\n> \n> @@ -59,5 +61,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> \t\treturn !!check_todo_list();\n> \tif (command == SKIP_UNNECESSARY_PICKS && argc == 1)\n> \t\treturn !!skip_unnecessary_picks();\n> +\tif (command == REARRANGE_SQUASH && argc == 1)\n> +\t\treturn !!rearrange_squash();\n> \tusage_with_options(builtin_rebase_helper_usage, options);\n> }\n> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n> index 92e3ca1de3b..84c6e62518f 100644\n> --- a/git-rebase--interactive.sh\n> +++ b/git-rebase--interactive.sh\n> @@ -721,94 +721,6 @@ collapse_todo_ids() {\n> \tgit rebase--helper --shorten-sha1s\n> }\n> \n> -# Rearrange the todo list that has both \"pick sha1 msg\" and\n> -# \"pick sha1 fixup!/squash! msg\" appears in it so that the latter\n> -# comes immediately after the former, and change \"pick\" to\n> -# \"fixup\"/\"squash\".\n> -#\n> -# Note that if the config has specified a custom instruction format\n> -# each log message will be re-retrieved in order to normalize the\n> -# autosquash arrangement\n> -rearrange_squash () {\n> -\tformat=$(git config --get rebase.instructionFormat)\n> -\t# extract fixup!/squash! lines and resolve any referenced sha1's\n> -\twhile read -r pick sha1 message\n> -\tdo\n> -\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n> -\t\tcase \"$message\" in\n> -\t\t\"squash! \"*|\"fixup! \"*)\n> -\t\t\taction=\"${message%%!*}\"\n> -\t\t\trest=$message\n> -\t\t\tprefix=\n> -\t\t\t# skip all squash! or fixup! (but save for later)\n> -\t\t\twhile :\n> -\t\t\tdo\n> -\t\t\t\tcase \"$rest\" in\n> -\t\t\t\t\"squash! \"*|\"fixup! \"*)\n> -\t\t\t\t\tprefix=\"$prefix${rest%%!*},\"\n> -\t\t\t\t\trest=\"${rest#*! }\"\n> -\t\t\t\t\t;;\n> -\t\t\t\t*)\n> -\t\t\t\t\tbreak\n> -\t\t\t\t\t;;\n> -\t\t\t\tesac\n> -\t\t\tdone\n> -\t\t\tprintf '%s %s %s %s\\n' \"$sha1\" \"$action\" \"$prefix\" \"$rest\"\n> -\t\t\t# if it's a single word, try to resolve to a full sha1 and\n> -\t\t\t# emit a second copy. This allows us to match on both message\n> -\t\t\t# and on sha1 prefix\n> -\t\t\tif test \"${rest#* }\" = \"$rest\"; then\n> -\t\t\t\tfullsha=\"$(git rev-parse -q --verify \"$rest\" 2>/dev/null)\"\n> -\t\t\t\tif test -n \"$fullsha\"; then\n> -\t\t\t\t\t# prefix the action to uniquely identify this line as\n> -\t\t\t\t\t# intended for full sha1 match\n> -\t\t\t\t\techo \"$sha1 +$action $prefix $fullsha\"\n> -\t\t\t\tfi\n> -\t\t\tfi\n> -\t\tesac\n> -\tdone >\"$1.sq\" <\"$1\"\n> -\ttest -s \"$1.sq\" || return\n> -\n> -\tused=\n> -\twhile read -r pick sha1 message\n> -\tdo\n> -\t\tcase \" $used\" in\n> -\t\t*\" $sha1 \"*) continue ;;\n> -\t\tesac\n> -\t\tprintf '%s\\n' \"$pick $sha1 $message\"\n> -\t\ttest -z \"${format}\" || message=$(git log -n 1 --format=\"%s\" ${sha1})\n> -\t\tused=\"$used$sha1 \"\n> -\t\twhile read -r squash action msg_prefix msg_content\n> -\t\tdo\n> -\t\t\tcase \" $used\" in\n> -\t\t\t*\" $squash \"*) continue ;;\n> -\t\t\tesac\n> -\t\t\temit=0\n> -\t\t\tcase \"$action\" in\n> -\t\t\t+*)\n> -\t\t\t\taction=\"${action#+}\"\n> -\t\t\t\t# full sha1 prefix test\n> -\t\t\t\tcase \"$msg_content\" in \"$sha1\"*) emit=1;; esac ;;\n> -\t\t\t*)\n> -\t\t\t\t# message prefix test\n> -\t\t\t\tcase \"$message\" in \"$msg_content\"*) emit=1;; esac ;;\n> -\t\t\tesac\n> -\t\t\tif test $emit = 1; then\n> -\t\t\t\tif test -n \"${format}\"\n> -\t\t\t\tthen\n> -\t\t\t\t\tmsg_content=$(git log -n 1 --format=\"${format}\" ${squash})\n> -\t\t\t\telse\n> -\t\t\t\t\tmsg_content=\"$(echo \"$msg_prefix\" | sed \"s/,/! /g\")$msg_content\"\n> -\t\t\t\tfi\n> -\t\t\t\tprintf '%s\\n' \"$action $squash $msg_content\"\n> -\t\t\t\tused=\"$used$squash \"\n> -\t\t\tfi\n> -\t\tdone <\"$1.sq\"\n> -\tdone >\"$1.rearranged\" <\"$1\"\n> -\tcat \"$1.rearranged\" >\"$1\"\n> -\trm -f \"$1.sq\" \"$1.rearranged\"\n> -}\n> -\n> # Add commands after a pick or after a squash/fixup serie\n> # in the todo list.\n> add_exec_commands () {\n> @@ -1068,7 +980,7 @@ then\n> fi\n> \n> test -s \"$todo\" || echo noop >> \"$todo\"\n> -test -n \"$autosquash\" && rearrange_squash \"$todo\"\n> +test -z \"$autosquash\" || git rebase--helper --rearrange-squash || exit\n> test -n \"$cmd\" && add_exec_commands \"$todo\"\n> \n> todocount=$(git stripspace --strip-comments <\"$todo\" | wc -l)\n> diff --git a/sequencer.c b/sequencer.c\n> index 72e3ad8d145..63a588f0916 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -19,6 +19,7 @@\n> #include \"trailer.h\"\n> #include \"log-tree.h\"\n> #include \"wt-status.h\"\n> +#include \"hashmap.h\"\n> \n> #define GIT_REFLOG_ACTION \"GIT_REFLOG_ACTION\"\n> \n> @@ -2723,3 +2724,197 @@ int skip_unnecessary_picks(void)\n> \n> \treturn 0;\n> }\n> +\n> +struct subject2item_entry {\n> +\tstruct hashmap_entry entry;\n> +\tint i;\n> +\tchar subject[FLEX_ARRAY];\n> +};\n> +\n> +static int subject2item_cmp(const struct subject2item_entry *a,\n> +\tconst struct subject2item_entry *b, const void *key)\n> +{\n> +\treturn key ? strcmp(a->subject, key) : strcmp(a->subject, b->subject);\n> +}\n> +\n> +/*\n> + * Rearrange the todo list that has both \"pick sha1 msg\" and \"pick sha1\n> + * fixup!/squash! msg\" in it so that the latter is put immediately after the\n> + * former, and change \"pick\" to \"fixup\"/\"squash\".\n> + *\n> + * Note that if the config has specified a custom instruction format, each log\n> + * message will have to be retrieved from the commit (as the oneline in the\n> + * script cannot be trusted) in order to normalize the autosquash arrangement.\n> + */\n> +int rearrange_squash(void)\n> +{\n> +\tconst char *todo_file = rebase_path_todo();\n> +\tstruct todo_list todo_list = TODO_LIST_INIT;\n> +\tstruct hashmap subject2item;\n> +\tint res = 0, rearranged = 0, *next, *tail, fd, i;\n> +\tchar **subjects;\n> +\n> +\tfd = open(todo_file, O_RDONLY);\n> +\tif (fd < 0)\n> +\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n> +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> +\t\tclose(fd);\n> +\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n> +\t}\n> +\tclose(fd);\n> +\tif (parse_insn_buffer(todo_list.buf.buf, &todo_list) < 0) {\n> +\t\ttodo_list_release(&todo_list);\n> +\t\treturn -1;\n> +\t}\n> +\n> +\t/*\n> +\t * The hashmap maps onelines to the respective todo list index.\n> +\t *\n> +\t * If any items need to be rearranged, the next[i] value will indicate\n> +\t * which item was moved directly after the i'th.\n> +\t *\n> +\t * In that case, last[i] will indicate the index of the latest item to\n> +\t * be moved to appear after the i'th.\n> +\t */\n> +\thashmap_init(&subject2item, (hashmap_cmp_fn) subject2item_cmp,\n> +\t\t todo_list.nr);\n> +\tALLOC_ARRAY(next, todo_list.nr);\n> +\tALLOC_ARRAY(tail, todo_list.nr);\n> +\tALLOC_ARRAY(subjects, todo_list.nr);\n> +\tfor (i = 0; i < todo_list.nr; i++) {\n> +\t\tstruct strbuf buf = STRBUF_INIT;\n> +\t\tstruct todo_item *item = todo_list.items + i;\n> +\t\tconst char *commit_buffer, *subject, *p;\n> +\t\tsize_t subject_len;\n> +\t\tint i2 = -1;\n> +\t\tstruct subject2item_entry *entry;\n> +\n> +\t\tnext[i] = tail[i] = -1;\n> +\t\tif (item->command >= TODO_EXEC) {\n> +\t\t\tsubjects[i] = NULL;\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\tif (is_fixup(item->command)) {\n> +\t\t\ttodo_list_release(&todo_list);\n> +\t\t\treturn error(_(\"the script was already rearranged.\"));\n> +\t\t}\n> +\n> +\t\titem->commit->util = item;\n> +\n> +\t\tparse_commit(item->commit);\n> +\t\tcommit_buffer = get_commit_buffer(item->commit, NULL);\n> +\t\tfind_commit_subject(commit_buffer, &subject);\n> +\t\tformat_subject(&buf, subject, \" \");\n> +\t\tsubject = subjects[i] = strbuf_detach(&buf, &subject_len);\n> +\t\tunuse_commit_buffer(item->commit, commit_buffer);\n> +\t\tif ((skip_prefix(subject, \"fixup! \", &p) ||\n> +\t\t skip_prefix(subject, \"squash! \", &p))) {\n> +\t\t\tstruct commit *commit2;\n> +\n> +\t\t\tfor (;;) {\n> +\t\t\t\twhile (isspace(*p))\n> +\t\t\t\t\tp++;\n> +\t\t\t\tif (!skip_prefix(p, \"fixup! \", &p) &&\n> +\t\t\t\t !skip_prefix(p, \"squash! \", &p))\n> +\t\t\t\t\tbreak;\n> +\t\t\t}\n> +\n> +\t\t\tif ((entry = hashmap_get_from_hash(&subject2item,\n> +\t\t\t\t\t\t\t strhash(p), p)))\n> +\t\t\t\t/* found by title */\n> +\t\t\t\ti2 = entry->i;\n> +\t\t\telse if (!strchr(p, ' ') &&\n> +\t\t\t\t (commit2 =\n> +\t\t\t\t lookup_commit_reference_by_name(p)) &&\n> +\t\t\t\t commit2->util)\n> +\t\t\t\t/* found by commit name */\n> +\t\t\t\ti2 = (struct todo_item *)commit2->util\n> +\t\t\t\t\t- todo_list.items;\n> +\t\t\telse {\n> +\t\t\t\t/* copy can be a prefix of the commit subject */\n> +\t\t\t\tfor (i2 = 0; i2 < i; i2++)\n> +\t\t\t\t\tif (subjects[i2] &&\n> +\t\t\t\t\t starts_with(subjects[i2], p))\n> +\t\t\t\t\t\tbreak;\n> +\t\t\t\tif (i2 == i)\n> +\t\t\t\t\ti2 = -1;\n> +\t\t\t}\n> +\t\t}\n> +\t\tif (i2 >= 0) {\n> +\t\t\trearranged = 1;\n> +\t\t\ttodo_list.items[i].command =\n> +\t\t\t\tstarts_with(subject, \"fixup!\") ?\n> +\t\t\t\tTODO_FIXUP : TODO_SQUASH;\n> +\t\t\tif (next[i2] < 0)\n> +\t\t\t\tnext[i2] = i;\n> +\t\t\telse\n> +\t\t\t\tnext[tail[i2]] = i;\n> +\t\t\ttail[i2] = i;\n> +\t\t} else if (!hashmap_get_from_hash(&subject2item,\n> +\t\t\t\t\t\tstrhash(subject), subject)) {\n> +\t\t\tFLEX_ALLOC_MEM(entry, subject, subject, subject_len);\n> +\t\t\tentry->i = i;\n> +\t\t\thashmap_entry_init(entry, strhash(entry->subject));\n> +\t\t\thashmap_put(&subject2item, entry);\n> +\t\t}\n> +\t}\n> +\n> +\tif (rearranged) {\n> +\t\tstruct strbuf buf = STRBUF_INIT;\n> +\n> +\t\tfor (i = 0; i < todo_list.nr; i++) {\n> +\t\t\tenum todo_command command = todo_list.items[i].command;\n> +\t\t\tint cur = i;\n> +\n> +\t\t\t/*\n> +\t\t\t * Initially, all commands are 'pick's. If it is a\n> +\t\t\t * fixup or a squash now, we have rearranged it.\n> +\t\t\t */\n> +\t\t\tif (is_fixup(command))\n> +\t\t\t\tcontinue;\n> +\n> +\t\t\twhile (cur >= 0) {\n> +\t\t\t\tint offset = todo_list.items[cur].offset_in_buf;\n> +\t\t\t\tint end_offset = cur + 1 < todo_list.nr ?\n> +\t\t\t\t\ttodo_list.items[cur + 1].offset_in_buf :\n> +\t\t\t\t\ttodo_list.buf.len;\n> +\t\t\t\tchar *bol = todo_list.buf.buf + offset;\n> +\t\t\t\tchar *eol = todo_list.buf.buf + end_offset;\n\nI got a little confused with these offsets. I know it was part of a\nprevious series, but maybe we could add a description to the fields\nof `struct todo_list` and `struct todo_item`.\nOther than that, I don't have any particular comments.\n\n> +\n> +\t\t\t\t/* replace 'pick', by 'fixup' or 'squash' */\n> +\t\t\t\tcommand = todo_list.items[cur].command;\n> +\t\t\t\tif (is_fixup(command)) {\n> +\t\t\t\t\tstrbuf_addstr(&buf,\n> +\t\t\t\t\t\ttodo_command_info[command].str);\n> +\t\t\t\t\tbol += strcspn(bol, \" \\t\");\n> +\t\t\t\t}\n> +\n> +\t\t\t\tstrbuf_add(&buf, bol, eol - bol);\n> +\n> +\t\t\t\tcur = next[cur];\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tfd = open(todo_file, O_WRONLY);\n> +\t\tif (fd < 0)\n> +\t\t\tres = error_errno(_(\"could not open '%s'\"), todo_file);\n> +\t\telse if (write(fd, buf.buf, buf.len) < 0)\n> +\t\t\tres = error_errno(_(\"could not read '%s'.\"), todo_file);\n> +\t\telse if (ftruncate(fd, buf.len) < 0)\n> +\t\t\tres = error_errno(_(\"could not finish '%s'\"),\n> +\t\t\t\t\t todo_file);\n> +\t\tclose(fd);\n> +\t\tstrbuf_release(&buf);\n> +\t}\n> +\n> +\tfree(next);\n> +\tfree(tail);\n> +\tfor (i = 0; i < todo_list.nr; i++)\n> +\t\tfree(subjects[i]);\n> +\tfree(subjects);\n> +\thashmap_free(&subject2item, 1);\n> +\ttodo_list_release(&todo_list);\n> +\n> +\treturn res;\n> +}\n> diff --git a/sequencer.h b/sequencer.h\n> index 28e1fc1e9bb..1c94bec7622 100644\n> --- a/sequencer.h\n> +++ b/sequencer.h\n> @@ -51,6 +51,7 @@ int sequencer_make_script(int keep_empty, FILE *out,\n> int transform_todo_ids(int shorten_sha1s);\n> int check_todo_list(void);\n> int skip_unnecessary_picks(void);\n> +int rearrange_squash(void);\n> \n> extern const char sign_off_header[];\n> \n> diff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh\n> index 926bb3da788..2f88f50c057 100755\n> --- a/t/t3415-rebase-autosquash.sh\n> +++ b/t/t3415-rebase-autosquash.sh\n> @@ -290,7 +290,7 @@ set_backup_editor () {\n> \ttest_set_editor \"$PWD/backup-editor.sh\"\n> }\n> \n> -test_expect_failure 'autosquash with multiple empty patches' '\n> +test_expect_success 'autosquash with multiple empty patches' '\n> \ttest_tick &&\n> \tgit commit --allow-empty -m \"empty\" &&\n> \ttest_tick &&\n> -- \n> 2.12.2.windows.2.800.gede8f145e06\n\nLiam\n"},{"id":"320829","messageId":"CAGZ79kYeeCog03NHqqzZeb4Ya_m+jpQfzfG9Aa=rM5pxHpRCKQ@mail.gmail.com","threadId":"43983","inReplyTo":"20170526031524.27935-1-liambeguin@gmail.com","subject":"Re: [PATCH v4 03/10] rebase -i: remove useless indentation","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-26T17:50:48Z","receivedAt":"2017-05-26T17:51:00Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, May 25, 2017 at 8:15 PM, Liam Beguin wrote:\n> Hi Johannes,\n>\n> Johannes Schindelin writes:\n>> The commands used to be indented, and it is nice to look at, but when we\n>> transform the SHA-1s, the indentation is removed. So let's do away with it.\n>>\n>> For the moment, at least: when we will use the upcoming rebase--helper\n>> to transform the SHA-1s, we *will* keep the indentation and can\n>> reintroduce it. Yet, to be able to validate the rebase--helper against\n>> the output of the current shell script version, we need to remove the\n>> extra indentation.\n>>\n>> Signed-off-by: Johannes Schindelin \n>> ---\n>> git-rebase--interactive.sh | 14 +++++++-------\n>> 1 file changed, 7 insertions(+), 7 deletions(-)\n>>\n>> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n>> index 609e150d38f..c40b1fd1d2e 100644\n>> --- a/git-rebase--interactive.sh\n>> +++ b/git-rebase--interactive.sh\n>> @@ -155,13 +155,13 @@ reschedule_last_action () {\n>> append_todo_help () {\n>> gettext \"\n>> Commands:\n>> - p, pick = use commit\n>> - r, reword = use commit, but edit the commit message\n>> - e, edit = use commit, but stop for amending\n>> - s, squash = use commit, but meld into previous commit\n>> - f, fixup = like \\\"squash\\\", but discard this commit's log message\n>> - x, exec = run command (the rest of the line) using shell\n>> - d, drop = remove commit\n>> +p, pick = use commit\n>> +r, reword = use commit, but edit the commit message\n>> +e, edit = use commit, but stop for amending\n>> +s, squash = use commit, but meld into previous commit\n>> +f, fixup = like \\\"squash\\\", but discard this commit's log message\n>> +x, exec = run command (the rest of the line) using shell\n>> +d, drop = remove commit\n>\n> do we also need to update all the translations since this is a `gettext`\n> function?\n\nTranslations are handled separately, later before a release is done.\nSeparation of skills. ;)\n\nAs programming is quite complicated and involved we'd rather ask\nJohannes to only focus on the code in such a patch here and then later\nthe translators will focus on getting the translation right. As the translation\ntools are sophisticated, they will likely give the previous translation such\nthat the translators see that there is only a white space change.\n\nBut as the commit message hints at a later patch that will reintroduce the\noriginal indentation, maybe the translators won't even see that change?\n\nFor more information on how the translations work in the git workflow,\nsee 951ea7656e (Merge tag 'l10n-2.13.0-rnd2.1' of\ngit://github.com/git-l10n/git-po, 2017-05-09) or see\nhttps://public-inbox.org/git/CANYiYbGfDXj4jJTcd3PpXqsDN-TwCC8Dm8B9Ov_3NaSzwsrCfg@mail.gmail.com/\n"},{"id":"320859","messageId":"fcd7f363-9c71-8aae-a298-7698285f479a@gmail.com","threadId":"43983","inReplyTo":"CAGZ79kYeeCog03NHqqzZeb4Ya_m+jpQfzfG9Aa=rM5pxHpRCKQ@mail.gmail.com","subject":"Re: [PATCH v4 03/10] rebase -i: remove useless indentation","fromName":"liam Beguin","fromEmail":"liambeguin@gmail.com","sentAt":"2017-05-27T03:15:47Z","receivedAt":"2017-05-27T03:24:35Z","isPatch":true,"sender":{"key":"liambeguin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3811160?v=4"},"body":"Hi Stefan,\n\nOn 26/05/17 01:50 PM, Stefan Beller wrote:\n> On Thu, May 25, 2017 at 8:15 PM, Liam Beguin wrote:\n>> Hi Johannes,\n>>\n>> Johannes Schindelin writes:\n>>> The commands used to be indented, and it is nice to look at, but when we\n>>> transform the SHA-1s, the indentation is removed. So let's do away with it.\n>>>\n>>> For the moment, at least: when we will use the upcoming rebase--helper\n>>> to transform the SHA-1s, we *will* keep the indentation and can\n>>> reintroduce it. Yet, to be able to validate the rebase--helper against\n>>> the output of the current shell script version, we need to remove the\n>>> extra indentation.\n>>>\n>>> Signed-off-by: Johannes Schindelin \n>>> ---\n>>> git-rebase--interactive.sh | 14 +++++++-------\n>>> 1 file changed, 7 insertions(+), 7 deletions(-)\n>>>\n>>> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n>>> index 609e150d38f..c40b1fd1d2e 100644\n>>> --- a/git-rebase--interactive.sh\n>>> +++ b/git-rebase--interactive.sh\n>>> @@ -155,13 +155,13 @@ reschedule_last_action () {\n>>> append_todo_help () {\n>>> gettext \"\n>>> Commands:\n>>> - p, pick = use commit\n>>> - r, reword = use commit, but edit the commit message\n>>> - e, edit = use commit, but stop for amending\n>>> - s, squash = use commit, but meld into previous commit\n>>> - f, fixup = like \\\"squash\\\", but discard this commit's log message\n>>> - x, exec = run command (the rest of the line) using shell\n>>> - d, drop = remove commit\n>>> +p, pick = use commit\n>>> +r, reword = use commit, but edit the commit message\n>>> +e, edit = use commit, but stop for amending\n>>> +s, squash = use commit, but meld into previous commit\n>>> +f, fixup = like \\\"squash\\\", but discard this commit's log message\n>>> +x, exec = run command (the rest of the line) using shell\n>>> +d, drop = remove commit\n>>\n>> do we also need to update all the translations since this is a `gettext`\n>> function?\n> \n> Translations are handled separately, later before a release is done.\n> Separation of skills. ;)\n> \n> As programming is quite complicated and involved we'd rather ask\n> Johannes to only focus on the code in such a patch here and then later\n> the translators will focus on getting the translation right. As the translation\n> tools are sophisticated, they will likely give the previous translation such\n> that the translators see that there is only a white space change.\n\nThanks for the clarification, I was wondering how this part was handled.\n\n> \n> But as the commit message hints at a later patch that will reintroduce the\n> original indentation, maybe the translators won't even see that change?\n> \n> For more information on how the translations work in the git workflow,\n> see 951ea7656e (Merge tag 'l10n-2.13.0-rnd2.1' of\n> git://github.com/git-l10n/git-po, 2017-05-09) or see\n> https://public-inbox.org/git/CANYiYbGfDXj4jJTcd3PpXqsDN-TwCC8Dm8B9Ov_3NaSzwsrCfg@mail.gmail.com/\n> \n\nLiam \n"},{"id":"320870","messageId":"6f0e176b-e638-c13c-6a6d-aa5b62297879@web.de","threadId":"43983","inReplyTo":"20170526031508.24838-1-liambeguin@gmail.com","subject":"Re: [PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2017-05-27T16:23:05Z","receivedAt":"2017-05-27T16:25:53Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 26.05.2017 um 05:15 schrieb Liam Beguin:\n> I tried to time the execution on an interactive rebase (on Linux) but\n> I did not notice a significant change in speed.\n> Do we have a way to measure performance / speed changes between version?\n\nWell, there's performance test script p3404-rebase-interactive.sh. You\ncould run it e.g. like this:\n\n\t$ (cd t/perf && ./run origin/master HEAD ./p3404*.sh)\n\nThis would compare the performance of master with the current branch\nyou're on. The results of p3404 are quite noisy for me on master,\nthough (saw 15% difference between runs without any code changes), so\ntake them with a bag of salt.\n\nThere's more info on performance tests in general in t/perf/README.\n\nRené\n"},{"id":"320927","messageId":"xmqq60gk8ad5.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"e173445d9a6321f80053967e50375400c0a61764.1493414945.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-29T06:07:02Z","receivedAt":"2017-05-29T06:07:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> diff --git a/sequencer.c b/sequencer.c\n> index 130cc868e51..88819a1a2a9 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -2388,3 +2388,52 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> \n> \tstrbuf_release(&sob);\n> }\n> +\n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv)\n> +{\n> +\tchar *format = NULL;\n> +\tstruct pretty_print_context pp = {0};\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tstruct rev_info revs;\n> +\tstruct commit *commit;\n> +\n> +\tinit_revisions(&revs, NULL);\n> +\trevs.verbose_header = 1;\n> +\trevs.max_parents = 1;\n> +\trevs.cherry_pick = 1;\n> +\trevs.limited = 1;\n> +\trevs.reverse = 1;\n> +\trevs.right_only = 1;\n> +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> +\trevs.topo_order = 1;\n\ncf. \n\nThis is still futzing below the API implementation detail\nunnecessarily.\n"},{"id":"320935","messageId":"xmqqefv86p4z.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"cover.1493414945.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-29T08:30:52Z","receivedAt":"2017-05-29T08:31:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> This patch series reimplements the expensive pre- and post-processing of\n> the todo script in C.\n>\n> And it concludes the work I did to accelerate rebase -i.\n>\n\nI took another look at the series (as \"What's cooking\" report was\nlisting this in the \"Needs review\" state). Except for an inssue\nthat I already pointed out in an earlier review, I didn't spot\nanything glaringly wrong in this v4 round.\n\nThanks.\n"},{"id":"320940","messageId":"alpine.DEB.2.21.1.1705291247000.3610@virtualbox","threadId":"43983","inReplyTo":"6f0e176b-e638-c13c-6a6d-aa5b62297879@web.de","subject":"Re: [PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-29T10:51:38Z","receivedAt":"2017-05-29T10:52:03Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi René,\n\nOn Sat, 27 May 2017, René Scharfe wrote:\n\n> Am 26.05.2017 um 05:15 schrieb Liam Beguin:\n> > I tried to time the execution on an interactive rebase (on Linux) but\n> > I did not notice a significant change in speed.\n> > Do we have a way to measure performance / speed changes between version?\n> \n> Well, there's performance test script p3404-rebase-interactive.sh. You\n> could run it e.g. like this:\n> \n> \t$ (cd t/perf && ./run origin/master HEAD ./p3404*.sh)\n> \n> This would compare the performance of master with the current branch\n> you're on. The results of p3404 are quite noisy for me on master,\n> though (saw 15% difference between runs without any code changes), so\n> take them with a bag of salt.\n\nIndeed. Our performance tests are simply not very meaningful.\n\nPart of it is the use of shell scripting (which defeats performance\ntesting pretty well), another part is that we have no performance testing\nexperts among us, and failed to attract any, so not only is the repeat\ncount ridiculously small, we also have no graphing worth speaking of (and\ntherefore it is impossible to even see trends, which is a rather important\nvisual way to verify sound performance testing).\n\nFrankly, I have no illusion about this getting fixed, ever.\n\nSo yes, in the meantime we need to use those numbers with a considerable\namount of skepticism.\n\nCiao,\nDscho"},{"id":"320941","messageId":"alpine.DEB.2.21.1.1705291252080.3610@virtualbox","threadId":"43983","inReplyTo":"xmqq60gk8ad5.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-29T10:54:01Z","receivedAt":"2017-05-29T10:54:16Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Mon, 29 May 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > diff --git a/sequencer.c b/sequencer.c\n> > index 130cc868e51..88819a1a2a9 100644\n> > --- a/sequencer.c\n> > +++ b/sequencer.c\n> > @@ -2388,3 +2388,52 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> > \n> > \tstrbuf_release(&sob);\n> > }\n> > +\n> > +int sequencer_make_script(int keep_empty, FILE *out,\n> > +\t\tint argc, const char **argv)\n> > +{\n> > +\tchar *format = NULL;\n> > +\tstruct pretty_print_context pp = {0};\n> > +\tstruct strbuf buf = STRBUF_INIT;\n> > +\tstruct rev_info revs;\n> > +\tstruct commit *commit;\n> > +\n> > +\tinit_revisions(&revs, NULL);\n> > +\trevs.verbose_header = 1;\n> > +\trevs.max_parents = 1;\n> > +\trevs.cherry_pick = 1;\n> > +\trevs.limited = 1;\n> > +\trevs.reverse = 1;\n> > +\trevs.right_only = 1;\n> > +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> > +\trevs.topo_order = 1;\n> \n> cf. \n> \n> This is still futzing below the API implementation detail\n> unnecessarily.\n\nYou still ask me to pass options in plain text that has to be parsed at\nrun-time, rather than compile-time-verifiable flags.\n\nI am quite puzzled that you keep asking me to make my code sloppy.\n\nCiao,\nDscho\n"},{"id":"320942","messageId":"alpine.DEB.2.21.1.1705291254400.3610@virtualbox","threadId":"43983","inReplyTo":"20170526031508.24838-1-liambeguin@gmail.com","subject":"Re: [PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-29T10:56:01Z","receivedAt":"2017-05-29T10:56:16Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Liam,\n\nOn Thu, 25 May 2017, Liam Beguin wrote:\n\n> Johannes Schindelin writes:\n> > This patch series reimplements the expensive pre- and post-processing of\n> > the todo script in C.\n> >\n> > [...]\n\nI see that you used git-send-email to send this. It did look a bit funny\nnot to have \"Re: \" prefixed subjects, so at first I thought you simply\nre-sent my patch series. But I guess we have no convenient way to perform\npatch review, therefore I don't fault you...\n\nThanks for reviewing!\nJohannes\n"},{"id":"320943","messageId":"alpine.DEB.2.21.1.1705291256430.3610@virtualbox","threadId":"43983","inReplyTo":"20170526031516.26287-1-liambeguin@gmail.com","subject":"Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-29T10:59:34Z","receivedAt":"2017-05-29T11:00:00Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Liam,\n\nOn Thu, 25 May 2017, Liam Beguin wrote:\n\n> Johannes Schindelin writes:\n>\n> > diff --git a/sequencer.c b/sequencer.c\n> > index 130cc868e51..88819a1a2a9 100644\n> > --- a/sequencer.c\n> > +++ b/sequencer.c\n> > @@ -2388,3 +2388,52 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> > \n> > \tstrbuf_release(&sob);\n> > }\n> > +\n> > +int sequencer_make_script(int keep_empty, FILE *out,\n> > +\t\tint argc, const char **argv)\n> > +{\n> > +\tchar *format = NULL;\n> > +\tstruct pretty_print_context pp = {0};\n> > +\tstruct strbuf buf = STRBUF_INIT;\n> > +\tstruct rev_info revs;\n> > +\tstruct commit *commit;\n> > +\n> > +\tinit_revisions(&revs, NULL);\n> > +\trevs.verbose_header = 1;\n> > +\trevs.max_parents = 1;\n> > +\trevs.cherry_pick = 1;\n> > +\trevs.limited = 1;\n> > +\trevs.reverse = 1;\n> > +\trevs.right_only = 1;\n> > +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> > +\trevs.topo_order = 1;\n> > +\n> > +\trevs.pretty_given = 1;\n> > +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n> > +\tif (!format || !*format) {\n> > +\t\tfree(format);\n> > +\t\tformat = xstrdup(\"%s\");\n> > +\t}\n> > +\tget_commit_format(format, &revs);\n> > +\tfree(format);\n> > +\tpp.fmt = revs.commit_format;\n> > +\tpp.output_encoding = get_log_output_encoding();\n> > +\n> > +\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n> > +\t\treturn error(_(\"make_script: unhandled options\"));\n> > +\n> > +\tif (prepare_revision_walk(&revs) < 0)\n> > +\t\treturn error(_(\"make_script: error preparing revisions\"));\n> > +\n> > +\twhile ((commit = get_revision(&revs))) {\n> > +\t\tstrbuf_reset(&buf);\n> > +\t\tif (!keep_empty && is_original_commit_empty(commit))\n> > +\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n> \n> I've never had to use empty commits before, but while testing this, I\n> noticed that `git rebase -i --keep-empty` behaves differently if using\n> the --root option instead of a branch or something like 'HEAD~10'.\n> I also tested this on v2.13.0 and the behavior is the same.\n\nFWIW the terminology \"empty commit\" is a pretty poor naming choice. This\nis totally not your fault at all. I just wish we had a much more intuitive\nname to describe a commit that does not introduce any changes to the tree.\n\nAnd yes, doing this with --root is a bit of a hack. That's because --root\nis a bit of a hack.\n\nI am curious, though, as to the exact differences you experienced. I mean,\nit could be buggy behavior that needs to be fixed (independently of this\npatch series, of course).\n\nCiao,\nJohannes\n"},{"id":"320948","messageId":"alpine.DEB.2.21.1.1705291302160.3610@virtualbox","threadId":"43983","inReplyTo":"20170526031530.29162-1-liambeguin@gmail.com","subject":"Re: [PATCH v4 05/10] rebase -i: also expand/collapse the SHA-1s via the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-29T11:20:30Z","receivedAt":"2017-05-29T11:20:55Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Liam,\n\nOn Thu, 25 May 2017, Liam Beguin wrote:\n\n> Johannes Schindelin writes:\n> \n> > diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\n> > index 821058d452d..9444c8d6c60 100644\n> > --- a/builtin/rebase--helper.c\n> > +++ b/builtin/rebase--helper.c\n> > @@ -24,6 +24,10 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> > \t\t\t\tABORT),\n> > \t\tOPT_CMDMODE(0, \"make-script\", &command,\n> > \t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n> > +\t\tOPT_CMDMODE(0, \"shorten-sha1s\", &command,\n> > +\t\t\tN_(\"shorten SHA-1s in the todo list\"), SHORTEN_SHA1S),\n> > +\t\tOPT_CMDMODE(0, \"expand-sha1s\", &command,\n> > +\t\t\tN_(\"expand SHA-1s in the todo list\"), EXPAND_SHA1S),\n> \n> Since work is being done to convert to `struct object_id` would it\n> not be best to use a more generic name instead of 'sha1'?\n> maybe something like {shorten,expand}-hashs\n\nGood point. You suggest the use of \"ids\" later, and I think that is an\neven better name: what we try to do here is to expand/reduce the commit\n*identifiers*. The fact that they are hexadecimal representations of\nhashes is an implementation detail.\n\n> > diff --git a/sequencer.c b/sequencer.c\n> > index 88819a1a2a9..201d45b1677 100644\n> > --- a/sequencer.c\n> > +++ b/sequencer.c\n> > @@ -2437,3 +2437,60 @@ int sequencer_make_script(int keep_empty, FILE *out,\n> > \tstrbuf_release(&buf);\n> > \treturn 0;\n> > }\n> > +\n> > +\n> > +int transform_todo_ids(int shorten_sha1s)\n> > +{\n> > +\tconst char *todo_file = rebase_path_todo();\n> > +\tstruct todo_list todo_list = TODO_LIST_INIT;\n> > +\tint fd, res, i;\n> > +\tFILE *out;\n> > +\n> > +\tstrbuf_reset(&todo_list.buf);\n> > +\tfd = open(todo_file, O_RDONLY);\n> > +\tif (fd < 0)\n> > +\t\treturn error_errno(_(\"could not open '%s'\"), todo_file);\n> > +\tif (strbuf_read(&todo_list.buf, fd, 0) < 0) {\n> > +\t\tclose(fd);\n> > +\t\treturn error(_(\"could not read '%s'.\"), todo_file);\n> > +\t}\n> > +\tclose(fd);\n> > +\n> > +\tres = parse_insn_buffer(todo_list.buf.buf, &todo_list);\n> > +\tif (res) {\n> > +\t\ttodo_list_release(&todo_list);\n> > +\t\treturn error(_(\"unusable instruction sheet: '%s'\"), todo_file);\n> \n> As you pointed out last time, the name of the \"todo script\" can be a\n> source of confusion. The migration to C could be a good opportunity to\n> clarify this.\n\nTrue. This was simply a copy-edited part, and I should have caught that.\n\n> I don't know which is the preferred name but we could go with\n> \"todo list\" as it is the most common across the code base.\n\nYep, my next iteration will use that term.\n\n> > +\t}\n> > +\n> > +\tout = fopen(todo_file, \"w\");\n> > +\tif (!out) {\n> > +\t\ttodo_list_release(&todo_list);\n> > +\t\treturn error(_(\"unable to open '%s' for writing\"), todo_file);\n> > +\t}\n> > +\tfor (i = 0; i < todo_list.nr; i++) {\n> > +\t\tstruct todo_item *item = todo_list.items + i;\n> > +\t\tint bol = item->offset_in_buf;\n> > +\t\tconst char *p = todo_list.buf.buf + bol;\n> > +\t\tint eol = i + 1 < todo_list.nr ?\n> > +\t\t\ttodo_list.items[i + 1].offset_in_buf :\n> > +\t\t\ttodo_list.buf.len;\n> > +\n> > +\t\tif (item->command >= TODO_EXEC && item->command != TODO_DROP)\n> > +\t\t\tfwrite(p, eol - bol, 1, out);\n> > +\t\telse {\n> > +\t\t\tconst char *sha1 = shorten_sha1s ?\n> > +\t\t\t\tshort_commit_name(item->commit) :\n> > +\t\t\t\toid_to_hex(&item->commit->object.oid);\n> \n> We could also use 'hash' or 'ids' here instead of 'sha1'.\n\nAbsolutely!\n\nThank you,\nJohannes\n"},{"id":"320950","messageId":"alpine.DEB.2.21.1.1705291322150.3610@virtualbox","threadId":"43983","inReplyTo":"20170526031639.10790-1-liambeguin@gmail.com","subject":"Re: [PATCH v4 10/10] rebase -i: rearrange fixup/squash lines using the rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-29T11:26:08Z","receivedAt":"2017-05-29T11:26:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Liam,\n\nOn Thu, 25 May 2017, Liam Beguin wrote:\n\n> Johannes Schindelin writes:\n> [...]\n> > +\tif (rearranged) {\n> > +\t\tstruct strbuf buf = STRBUF_INIT;\n> > +\n> > +\t\tfor (i = 0; i < todo_list.nr; i++) {\n> > +\t\t\tenum todo_command command = todo_list.items[i].command;\n> > +\t\t\tint cur = i;\n> > +\n> > +\t\t\t/*\n> > +\t\t\t * Initially, all commands are 'pick's. If it is a\n> > +\t\t\t * fixup or a squash now, we have rearranged it.\n> > +\t\t\t */\n> > +\t\t\tif (is_fixup(command))\n> > +\t\t\t\tcontinue;\n> > +\n> > +\t\t\twhile (cur >= 0) {\n> > +\t\t\t\tint offset = todo_list.items[cur].offset_in_buf;\n> > +\t\t\t\tint end_offset = cur + 1 < todo_list.nr ?\n> > +\t\t\t\t\ttodo_list.items[cur + 1].offset_in_buf :\n> > +\t\t\t\t\ttodo_list.buf.len;\n> > +\t\t\t\tchar *bol = todo_list.buf.buf + offset;\n> > +\t\t\t\tchar *eol = todo_list.buf.buf + end_offset;\n> \n> I got a little confused with these offsets. I know it was part of a\n> previous series, but maybe we could add a description to the fields\n> of `struct todo_list` and `struct todo_item`.\n\nYou mean \"offset_in_buf\"?\n\nSure, I can add a comment there. I would like to keep it out of this patch\nseries, though, as the purpose of it is to accelerate rebase -i by moving\nlogic from the (slow) shell script code to the (decently fast) C code.\n\nCiao,\nJohannes\n"},{"id":"320954","messageId":"CACBZZX4avOKJjWVSBSewNFMWyRj3FzHC2Onw3aWLf1F_MYi+Gg@mail.gmail.com","threadId":"43983","inReplyTo":"alpine.DEB.2.21.1.1705291247000.3610@virtualbox","subject":"Re: [PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-29T12:50:08Z","receivedAt":"2017-05-29T12:50:40Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Mon, May 29, 2017 at 12:51 PM, Johannes Schindelin\n wrote:\n> Hi René,\n>\n> On Sat, 27 May 2017, René Scharfe wrote:\n>\n>> Am 26.05.2017 um 05:15 schrieb Liam Beguin:\n>> > I tried to time the execution on an interactive rebase (on Linux) but\n>> > I did not notice a significant change in speed.\n>> > Do we have a way to measure performance / speed changes between version?\n>>\n>> Well, there's performance test script p3404-rebase-interactive.sh. You\n>> could run it e.g. like this:\n>>\n>> $ (cd t/perf && ./run origin/master HEAD ./p3404*.sh)\n>>\n>> This would compare the performance of master with the current branch\n>> you're on. The results of p3404 are quite noisy for me on master,\n>> though (saw 15% difference between runs without any code changes), so\n>> take them with a bag of salt.\n>\n> Indeed. Our performance tests are simply not very meaningful.\n>\n> Part of it is the use of shell scripting (which defeats performance\n> testing pretty well),\n\nDon't the performance tests take long enough that the shellscripting\noverhead gets lost in the noise? E.g. on Windows what do you get when\nyou run this in t/perf:\n\n $ GIT_PERF_REPEAT_COUNT=3 GIT_PERF_MAKE_OPTS=\"-j6 NO_OPENSSL=Y\nBLK_SHA1=Y CFLAGS=-O3\" ./run v2.10.0 v2.12.0 v2.13.0 p3400-rebase.sh\n\nI get split-index performance improving by 28% in 2.12 and 58% in\n2.13, small error bars even with just 3 runs. This is on Linux, but my\nsense of fork overhead on Windows is that it isn't so bad as to matter\nhere.\n\nI'd also be interested to see what sort of results you get for my\n\"grep: add support for the PCRE v1 JIT API\" patch which is in pu now,\nassuming you have a PCRE newer than 8.32 or so.\n\n> another part is that we have no performance testing\n> experts among us, and failed to attract any, so not only is the repeat\n> count ridiculously small, we also have no graphing worth speaking of (and\n> therefore it is impossible to even see trends, which is a rather important\n> visual way to verify sound performance testing).\n>\n> Frankly, I have no illusion about this getting fixed, ever.\n\nI have a project on my TODO that I've been meaning to get to which\nwould address this. I'd be interested to know what people think about\nthe design:\n\n* Run the perf tests in some more where the raw runtimes are saved away\n* Have some way to dump a static html page from that with graphs over\ntime (with gnuplot svg?)\n* Supply some config file to drive this, so you can e.g. run each\ntests N times against your repo X for the last 10 versions of git.\n* Since it's static HTML it would be trivial for anyone to share such\nresults, and e.g. setup running them in cron to regularly publish to\ngithub pages.\n\n> So yes, in the meantime we need to use those numbers with a considerable\n> amount of skepticism.\n\n...however, while the presentation could be improved, I've seen no\nreason to think that the underlying numbers are suspect, or that the\nperf framework needs to be rewritten as opposed to improved upon. If\nyou don't think so I'd like to know why.\n"},{"id":"320988","messageId":"xmqqfufnc56q.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"alpine.DEB.2.21.1.1705291252080.3610@virtualbox","subject":"Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-30T04:57:33Z","receivedAt":"2017-05-30T04:57:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> Hi Junio,\n>\n> On Mon, 29 May 2017, Junio C Hamano wrote:\n>\n>> Johannes Schindelin writes:\n>> \n>> > diff --git a/sequencer.c b/sequencer.c\n>> > index 130cc868e51..88819a1a2a9 100644\n>> > --- a/sequencer.c\n>> > +++ b/sequencer.c\n>> > @@ -2388,3 +2388,52 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n>> > \n>> > \tstrbuf_release(&sob);\n>> > }\n>> > +\n>> > +int sequencer_make_script(int keep_empty, FILE *out,\n>> > +\t\tint argc, const char **argv)\n>> > +{\n>> > +\tchar *format = NULL;\n>> > +\tstruct pretty_print_context pp = {0};\n>> > +\tstruct strbuf buf = STRBUF_INIT;\n>> > +\tstruct rev_info revs;\n>> > +\tstruct commit *commit;\n>> > +\n>> > +\tinit_revisions(&revs, NULL);\n>> > +\trevs.verbose_header = 1;\n>> > +\trevs.max_parents = 1;\n>> > +\trevs.cherry_pick = 1;\n>> > +\trevs.limited = 1;\n>> > +\trevs.reverse = 1;\n>> > +\trevs.right_only = 1;\n>> > +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n>> > +\trevs.topo_order = 1;\n>> \n>> cf. \n>> \n>> This is still futzing below the API implementation detail\n>> unnecessarily.\n>\n> You still ask me to pass options in plain text that has to be parsed at\n> run-time, rather than compile-time-verifiable flags.\n\nAbsolutely. \n\nWe do not want these implementation details to code that does not\nimplement command line parsing. This one is not parsing anybody's\nset of options and should not be mucking with the low level\nimplementation details.\n\nSee 66b2ed09 (\"Fix \"log\" family not to be too agressive about\nshowing notes\", 2010-01-20) and poinder. Back then, nobody outside\nbuiltin/log.c and revision.c (these are the two primary things that\nimplement command line parsing; \"log.c\" needs access to the low\nlevel details because it wants to establish custom default that is\napplied before it reads options given by the end-user) mucked\ndirectly with verbose_header, which allowed the addition of\n\"pretty_given\" to be limited only to these places that actually do\nthe parsing. If the above patch to sequencer.c existed before\n66b2ed09, it would have required unnecessary change to tweak\n\"pretty_given\" in there too when 66b2ed09 was done. That is forcing\na totally unnecesary work. And there is no reason to expect that\nthe kind of change 66b2ed09 made to the general command line parsing\nwill not happen in the future. Adding more code like the above that\nknows the implementation detail is unwarranted, when you can just\nask the existing command line parser to set them for you.\n\n"},{"id":"321024","messageId":"alpine.DEB.2.21.1.1705301659150.3610@virtualbox","threadId":"43983","inReplyTo":"xmqqfufnc56q.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-30T14:59:40Z","receivedAt":"2017-05-30T15:00:07Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Tue, 30 May 2017, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > You still ask me to pass options in plain text that has to be parsed at\n> > run-time, rather than compile-time-verifiable flags.\n> \n> Absolutely. \n\nIn other words, you want me to put my name to sloppy code.\n\nSorry, not interested,\nDscho\n"},{"id":"321026","messageId":"alpine.DEB.2.21.1.1705301700440.3610@virtualbox","threadId":"43983","inReplyTo":"xmqqfufnc56q.fsf@gitster.mtv.corp.google.com","subject":"revision API design, was Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-30T15:08:44Z","receivedAt":"2017-05-30T15:09:08Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Tue, 30 May 2017, Junio C Hamano wrote:\n\n> We do not want these [revision API] implementation details to code that\n> does not implement command line parsing. This one is not parsing\n> anybody's set of options and should not be mucking with the low level\n> implementation details.\n\nJust to make sure we are on the same level: you want the argc & argv to be\nthe free-form API of setup_revisions(), rather than code setting fields\nin struct rev_info whose names can be verified at compile time, and whose\nnames also suggest their intended use.\n\nI am still flabberghasted that any seasoned software developer sincerely\nwould suggest this.\n\n> See 66b2ed09 (\"Fix \"log\" family not to be too agressive about\n> showing notes\", 2010-01-20) and poinder. Back then, nobody outside\n> builtin/log.c and revision.c (these are the two primary things that\n> implement command line parsing; \"log.c\" needs access to the low\n> level details because it wants to establish custom default that is\n> applied before it reads options given by the end-user) mucked\n> directly with verbose_header, which allowed the addition of\n> \"pretty_given\" to be limited only to these places that actually do\n> the parsing. If the above patch to sequencer.c existed before\n> 66b2ed09, it would have required unnecessary change to tweak\n> \"pretty_given\" in there too when 66b2ed09 was done. That is forcing\n> a totally unnecesary work. And there is no reason to expect that\n> the kind of change 66b2ed09 made to the general command line parsing\n> will not happen in the future. Adding more code like the above that\n> knows the implementation detail is unwarranted, when you can just\n> ask the existing command line parser to set them for you.\n\nThis is a very eloquent description of a problem with the API.\n\nThe correct suggestion would be to fix the API, of course. Not to declare\nan interface intended for command-line parsing the internal API.\n\nMaybe the introduction of `pretty_given` was a strong hint at a design\nflaw to begin with, pointing to the fact that user_format is a singleton\n(yes, really, you can't have two different user_formats in the same Git\nprocess, what were we thinking)?\n\nCiao,\nDscho\n"},{"id":"321030","messageId":"alpine.DEB.2.21.1.1705301635440.3610@virtualbox","threadId":"43983","inReplyTo":"CACBZZX4avOKJjWVSBSewNFMWyRj3FzHC2Onw3aWLf1F_MYi+Gg@mail.gmail.com","subject":"Re: [PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-05-30T15:44:46Z","receivedAt":"2017-05-30T15:45:24Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ævar,\n\nOn Mon, 29 May 2017, Ævar Arnfjörð Bjarmason wrote:\n\n> On Mon, May 29, 2017 at 12:51 PM, Johannes Schindelin\n> wrote:\n> >\n> > On Sat, 27 May 2017, René Scharfe wrote:\n> >> Am 26.05.2017 um 05:15 schrieb Liam Beguin:\n> >> > I tried to time the execution on an interactive rebase (on Linux)\n> >> > but I did not notice a significant change in speed. Do we have a\n> >> > way to measure performance / speed changes between version?\n> >>\n> >> Well, there's performance test script p3404-rebase-interactive.sh.\n> >> You could run it e.g. like this:\n> >>\n> >> $ (cd t/perf && ./run origin/master HEAD ./p3404*.sh)\n> >>\n> >> This would compare the performance of master with the current branch\n> >> you're on. The results of p3404 are quite noisy for me on master,\n> >> though (saw 15% difference between runs without any code changes), so\n> >> take them with a bag of salt.\n> >\n> > Indeed. Our performance tests are simply not very meaningful.\n> >\n> > Part of it is the use of shell scripting (which defeats performance\n> > testing pretty well),\n> \n> Don't the performance tests take long enough that the shellscripting\n> overhead gets lost in the noise?\n\nOkay, here you go, my weekly (or so) clarification about the POSIX\nemulation layer called MSYS2 (which itself kind of a portable Cygwin).\n\nWhenever Git for Windows has to execute Unix shell scripts (which are not\nnative to Windows, as the \"Unix\" in \"Unix shell scripts\" so clearly\nsuggests), we resort to calling the Bash from the MSYS2 project, which\nspins up a POSIX emulation layer. Git for Windows' own .exe files\n(and in particular, git.exe) is *not* affected by the POSIX emulation\nlayer, as they are real Win32 programs.\n\nWhenever execution has to bridge into, or out of, the POSIX emulation\nlayer, a few things need to be done. To emulate signal handling, for\nexample, a completely new process has to be spun up that itself has the\nnon-MSYS2 process as a child. The environment has to be converted, to\nreflect the fact that some things are Unix-y paths (or path lists) inside\nthe POSIX emulation layer and Windows paths outside.\n\nEven when staying within the POSIX emulation layer, some operations are\nnot as cheap as \"Linux folks\" are used to. For example, to spawn a\nsubshell, due to the absence of a spawn syscall fork() is called, followed\nby exec(). However, fork() itself is not native to Windows and has to be\nemulated. The POSIX emulation layer spins up a new process, meticulously\ncopies the entire memory, tries to reopen the file descriptors, network\nconnections, etc (to emulate the fork() semantics).\n\nObviously, this is anything but cheap.\n\nAnd this is only a glimpse into the entire problem, as I am not aware of\nany thorough analysis what is going on in msys-2.0.dll when shell scripts\nrun. All I know is that it slows things down dramatically.\n\nAs a consequence, even the simple act of creating a repository, or\nspawning Win32 processes from within a shell, become quite the\ncontributing factors to the noise of the measurements.\n\n> E.g. on Windows what do you get when you run this in t/perf:\n> \n> $ GIT_PERF_REPEAT_COUNT=3 GIT_PERF_MAKE_OPTS=\"-j6 NO_OPENSSL=Y\n> BLK_SHA1=Y CFLAGS=-O3\" ./run v2.10.0 v2.12.0 v2.13.0 p3400-rebase.sh\n\nIn my hands, a repeat count of 3 always resulted in quite a bit of noise\npreviously.\n\nMind you, I am working my machine. It has to run two VMs in the\nbackground, has multiple browsers and dozens of tabs open, checks for\nmails and Tweets and RSS feeds and a couple of Skypes are open, too.\n\nSo yeah, obviously there is a bit of noise involved.\n\n> I get split-index performance improving by 28% in 2.12 and 58% in\n> 2.13, small error bars even with just 3 runs. This is on Linux, but my\n> sense of fork overhead on Windows is that it isn't so bad as to matter\n> here.\n\nTest\n------------------------------------------------------\n3400.2: rebase on top of a lot of unrelated changes\n\nv2.10.0 v2.12.0 v2.13.0\n------------------------------------------------------------------\n60.65(0.01+0.03) 55.75(0.01+0.07) -8.1% 55.97(0.04+0.09) -7.7%\n\n(wrapped myself, as the ./run output is a lot wider than the 80 columns\nallowed in plain text email format)\n\nAnd what does it tell you?\n\nNot much, right? You have no idea about the trend line of the three tests,\nnot even of the standard deviation (not that it would be meaningful for\nN=3). It is not immediately obvious whether the first run is always a tad\nslower (or faster), or whether there is no noticable difference between\nthe first and any subsequent runs.\n\nIn other words, there is no measure of confidence in those results. We\ncan't say how reliable those numbers are.\n\nAnd we certainly can't know how much the shell scripting hurts.\n\nAlthough... let's try something. Let's run the same command in a *Linux\nVM* on the same machine! Yes, that should give us an idea. So here goes:\n\nTest\n------------------------------------------------------\n3400.2: rebase on top of a lot of unrelated changes\n\nv2.10.0 v2.12.0 v2.13.0\n---------------------------------------------------------------\n2.08(1.76+0.15) 2.10(1.76+0.15) +1.0% 2.00(1.65+0.15) -3.8%\n\n\nA ha! Not only does this show a curious *increase* in v2.12.0 (but I'd not\nput much stock into that, again N=3 is way too low a repetition number),\nit also shows that the Linux VM runs the same thing roughly 30x faster.\n\nI did see a few speed differences between native git.exe on Windows and\nthe git executable on Linux, but it was barely in the two-digit\n*percentage* region. Nowhere near the four-digit percentage region.\n\nSo now you know how much shell scripting hurts performance testing.\n\nA lot.\n\nIt pretty much renders the entire endeavor of testing performance\ncompletely and utterly useless.\n\n> I'd also be interested to see what sort of results you get for my\n> \"grep: add support for the PCRE v1 JIT API\" patch which is in pu now,\n> assuming you have a PCRE newer than 8.32 or so.\n\npu does not build for me:\n\n2017-05-30T11:38:50.0089681Z libgit.a(grep.o): In function `pcre1match':\n2017-05-30T11:38:50.0289250Z .../grep.c:411: undefined reference to `__imp_pcre_jit_exec'\n2017-05-30T11:38:50.0329160Z collect2.exe: error: ld returned 1 exit\nstatus\n\n> > Frankly, I have no illusion about this getting fixed, ever.\n> \n> I have a project on my TODO that I've been meaning to get to which\n> would address this. I'd be interested to know what people think about\n> the design:\n> \n> * Run the perf tests in some more where the raw runtimes are saved away\n\nYou mean a test helper designed to do the timing and the setting up so as\nto time *just* the operations that should be timed?\n\nIf so: I am all for it.\n\n> * Have some way to dump a static html page from that with graphs over\n> time (with gnuplot svg?)\n\nIf you already go HTML, it would make much more sense to go d3.js. I would\neven prefer to go c3.js (which uses d3.js) right away. Would make\neverything so much easier.\n\nNot to mention more portable.\n\n> * Supply some config file to drive this, so you can e.g. run each\n> tests N times against your repo X for the last 10 versions of git.\n\nSure.\n\n> * Since it's static HTML it would be trivial for anyone to share such\n> results, and e.g. setup running them in cron to regularly publish to\n> github pages.\n\nIt does not need to be static. It can use, say, c3.js, for the added\nbenefit of being able to toggle multiple graphs in the same diagram.\n\nCiao,\nDscho"},{"id":"321031","messageId":"9baa8898-36c7-79a9-025b-799785c81284@gmail.com","threadId":"43983","inReplyTo":"alpine.DEB.2.21.1.1705291256430.3610@virtualbox","subject":"Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"liam Beguin","fromEmail":"liambeguin@gmail.com","sentAt":"2017-05-30T15:57:51Z","receivedAt":"2017-05-30T15:57:59Z","isPatch":true,"sender":{"key":"liambeguin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3811160?v=4"},"body":"Hi Johannes,\n\nOn 29/05/17 06:59 AM, Johannes Schindelin wrote:\n> Hi Liam,\n> \n> On Thu, 25 May 2017, Liam Beguin wrote:\n> \n>> Johannes Schindelin writes:\n>>\n>>> diff --git a/sequencer.c b/sequencer.c\n>>> index 130cc868e51..88819a1a2a9 100644\n>>> --- a/sequencer.c\n>>> +++ b/sequencer.c\n>>> @@ -2388,3 +2388,52 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n>>> \n>>> \tstrbuf_release(&sob);\n>>> }\n>>> +\n>>> +int sequencer_make_script(int keep_empty, FILE *out,\n>>> +\t\tint argc, const char **argv)\n>>> +{\n>>> +\tchar *format = NULL;\n>>> +\tstruct pretty_print_context pp = {0};\n>>> +\tstruct strbuf buf = STRBUF_INIT;\n>>> +\tstruct rev_info revs;\n>>> +\tstruct commit *commit;\n>>> +\n>>> +\tinit_revisions(&revs, NULL);\n>>> +\trevs.verbose_header = 1;\n>>> +\trevs.max_parents = 1;\n>>> +\trevs.cherry_pick = 1;\n>>> +\trevs.limited = 1;\n>>> +\trevs.reverse = 1;\n>>> +\trevs.right_only = 1;\n>>> +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n>>> +\trevs.topo_order = 1;\n>>> +\n>>> +\trevs.pretty_given = 1;\n>>> +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n>>> +\tif (!format || !*format) {\n>>> +\t\tfree(format);\n>>> +\t\tformat = xstrdup(\"%s\");\n>>> +\t}\n>>> +\tget_commit_format(format, &revs);\n>>> +\tfree(format);\n>>> +\tpp.fmt = revs.commit_format;\n>>> +\tpp.output_encoding = get_log_output_encoding();\n>>> +\n>>> +\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n>>> +\t\treturn error(_(\"make_script: unhandled options\"));\n>>> +\n>>> +\tif (prepare_revision_walk(&revs) < 0)\n>>> +\t\treturn error(_(\"make_script: error preparing revisions\"));\n>>> +\n>>> +\twhile ((commit = get_revision(&revs))) {\n>>> +\t\tstrbuf_reset(&buf);\n>>> +\t\tif (!keep_empty && is_original_commit_empty(commit))\n>>> +\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n>>\n>> I've never had to use empty commits before, but while testing this, I\n>> noticed that `git rebase -i --keep-empty` behaves differently if using\n>> the --root option instead of a branch or something like 'HEAD~10'.\n>> I also tested this on v2.13.0 and the behavior is the same.\n> \n> FWIW the terminology \"empty commit\" is a pretty poor naming choice. This\n> is totally not your fault at all. I just wish we had a much more intuitive\n> name to describe a commit that does not introduce any changes to the tree.\n> \n> And yes, doing this with --root is a bit of a hack. That's because --root\n> is a bit of a hack.\n> \n> I am curious, though, as to the exact differences you experienced. I mean,\n> it could be buggy behavior that needs to be fixed (independently of this\n> patch series, of course).\n> \n\nSorry, reading this I realized that I didn't give any details!\nWhen using --root, --keep-empty has no effect. The empty commits\ndo not appear in the todo list and they are removed.\nAlso, when using --root without --keep-empty, the empty commits\ndo not show up as comments in the list.\n\n> Ciao,\n> Johannes\n> \n\nLiam \n"},{"id":"321072","messageId":"b1377fcf-90a4-4ba8-d3ad-53be275b3b75@gmail.com","threadId":"43983","inReplyTo":"20170526031516.26287-1-liambeguin@gmail.com","subject":"Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"liam Beguin","fromEmail":"liambeguin@gmail.com","sentAt":"2017-05-30T18:19:33Z","receivedAt":"2017-05-30T18:19:41Z","isPatch":true,"sender":{"key":"liambeguin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3811160?v=4"},"body":"Hi Johannes,\n\nJohannes Schindelin writes:\n> The first step of an interactive rebase is to generate the so-called \"todo\n> script\", to be stored in the state directory as \"git-rebase-todo\" and to\n> be edited by the user.\n>\n> Originally, we adjusted the output of `git log ` using a simple\n> sed script. Over the course of the years, the code became more\n> complicated. We now use shell scripting to edit the output of `git log`\n> conditionally, depending whether to keep \"empty\" commits (i.e. commits\n> that do not change any files).\n>\n> On platforms where shell scripting is not native, this can be a serious\n> drag. And it opens the door for incompatibilities between platforms when\n> it comes to shell scripting or to Unix-y commands.\n>\n> Let's just re-implement the todo script generation in plain C, using the\n> revision machinery directly.\n>\n> This is substantially faster, improving the speed relative to the\n> shell script version of the interactive rebase from 2x to 3x on Windows.\n>\n> Note that the rearrange_squash() function in git-rebase--interactive\n> relied on the fact that we set the \"format\" variable to the config setting\n> rebase.instructionFormat. Relying on a side effect like this is no good,\n> hence we explicitly perform that assignment (possibly again) in\n> rearrange_squash().\n>\n> Signed-off-by: Johannes Schindelin \n> ---\n> builtin/rebase--helper.c | 8 +++++++-\n> git-rebase--interactive.sh | 44 +++++++++++++++++++++--------------------\n> sequencer.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++\n> sequencer.h | 3 +++\n> 4 files changed, 82 insertions(+), 22 deletions(-)\n>\n> diff --git a/builtin/rebase--helper.c b/builtin/rebase--helper.c\n> index ca1ebb2fa18..821058d452d 100644\n> --- a/builtin/rebase--helper.c\n> +++ b/builtin/rebase--helper.c\n> @@ -11,15 +11,19 @@ static const char * const builtin_rebase_helper_usage[] = {\n> int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> {\n> \tstruct replay_opts opts = REPLAY_OPTS_INIT;\n> +\tint keep_empty = 0;\n> \tenum {\n> -\t\tCONTINUE = 1, ABORT\n> +\t\tCONTINUE = 1, ABORT, MAKE_SCRIPT\n> \t} command = 0;\n> \tstruct option options[] = {\n> \t\tOPT_BOOL(0, \"ff\", &opts.allow_ff, N_(\"allow fast-forward\")),\n> +\t\tOPT_BOOL(0, \"keep-empty\", &keep_empty, N_(\"keep empty commits\")),\n> \t\tOPT_CMDMODE(0, \"continue\", &command, N_(\"continue rebase\"),\n> \t\t\t\tCONTINUE),\n> \t\tOPT_CMDMODE(0, \"abort\", &command, N_(\"abort rebase\"),\n> \t\t\t\tABORT),\n> +\t\tOPT_CMDMODE(0, \"make-script\", &command,\n> +\t\t\tN_(\"make rebase script\"), MAKE_SCRIPT),\n> \t\tOPT_END()\n> \t};\n\nThis is probably being picky, but we could also use a different name\nhere instead of 'rebase script'. This would help keep the name of this\nscript consistent as you already pointed out.\nmaybe something like 'make-todo-list' or just 'make-list'?\n\n> \n> @@ -36,5 +40,7 @@ int cmd_rebase__helper(int argc, const char **argv, const char *prefix)\n> \t\treturn !!sequencer_continue(&opts);\n> \tif (command == ABORT && argc == 1)\n> \t\treturn !!sequencer_remove_state(&opts);\n> +\tif (command == MAKE_SCRIPT && argc > 1)\n> +\t\treturn !!sequencer_make_script(keep_empty, stdout, argc, argv);\n\nsame here.\n\n> \tusage_with_options(builtin_rebase_helper_usage, options);\n> }\n> diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh\n> index 2c9c0165b5a..609e150d38f 100644\n> --- a/git-rebase--interactive.sh\n> +++ b/git-rebase--interactive.sh\n> @@ -785,6 +785,7 @@ collapse_todo_ids() {\n> # each log message will be re-retrieved in order to normalize the\n> # autosquash arrangement\n> rearrange_squash () {\n> +\tformat=$(git config --get rebase.instructionFormat)\n> \t# extract fixup!/squash! lines and resolve any referenced sha1's\n> \twhile read -r pick sha1 message\n> \tdo\n> @@ -1210,26 +1211,27 @@ else\n> \trevisions=$onto...$orig_head\n> \tshortrevisions=$shorthead\n> fi\n> -format=$(git config --get rebase.instructionFormat)\n> -# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n> -git rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n> -\t--reverse --left-right --topo-order \\\n> -\t$revisions ${restrict_revision+^$restrict_revision} | \\\n> -\tsed -n \"s/^>//p\" |\n> -while read -r sha1 rest\n> -do\n> -\n> -\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n> -\tthen\n> -\t\tcomment_out=\"$comment_char \"\n> -\telse\n> -\t\tcomment_out=\n> -\tfi\n> +if test t != \"$preserve_merges\"\n> +then\n> +\tgit rebase--helper --make-script ${keep_empty:+--keep-empty} \\\n> +\t\t$revisions ${restrict_revision+^$restrict_revision} >\"$todo\"\n> +else\n> +\tformat=$(git config --get rebase.instructionFormat)\n> +\t# the 'rev-list .. | sed' requires %m to parse; the instruction requires %H to parse\n> +\tgit rev-list $merges_option --format=\"%m%H ${format:-%s}\" \\\n> +\t\t--reverse --left-right --topo-order \\\n> +\t\t$revisions ${restrict_revision+^$restrict_revision} | \\\n> +\t\tsed -n \"s/^>//p\" |\n> +\twhile read -r sha1 rest\n> +\tdo\n> +\n> +\t\tif test -z \"$keep_empty\" && is_empty_commit $sha1 && ! is_merge_commit $sha1\n> +\t\tthen\n> +\t\t\tcomment_out=\"$comment_char \"\n> +\t\telse\n> +\t\t\tcomment_out=\n> +\t\tfi\n> \n> -\tif test t != \"$preserve_merges\"\n> -\tthen\n> -\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n> -\telse\n> \t\tif test -z \"$rebase_root\"\n> \t\tthen\n> \t\t\tpreserve=t\n> @@ -1248,8 +1250,8 @@ do\n> \t\t\ttouch \"$rewritten\"/$sha1\n> \t\t\tprintf '%s\\n' \"${comment_out}pick $sha1 $rest\" >>\"$todo\"\n> \t\tfi\n> -\tfi\n> -done\n> +\tdone\n> +fi\n> \n> # Watch for commits that been dropped by --cherry-pick\n> if test t = \"$preserve_merges\"\n> diff --git a/sequencer.c b/sequencer.c\n> index 130cc868e51..88819a1a2a9 100644\n> --- a/sequencer.c\n> +++ b/sequencer.c\n> @@ -2388,3 +2388,52 @@ void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)\n> \n> \tstrbuf_release(&sob);\n> }\n> +\n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv)\n> +{\n> +\tchar *format = NULL;\n> +\tstruct pretty_print_context pp = {0};\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tstruct rev_info revs;\n> +\tstruct commit *commit;\n> +\n> +\tinit_revisions(&revs, NULL);\n> +\trevs.verbose_header = 1;\n> +\trevs.max_parents = 1;\n> +\trevs.cherry_pick = 1;\n> +\trevs.limited = 1;\n> +\trevs.reverse = 1;\n> +\trevs.right_only = 1;\n> +\trevs.sort_order = REV_SORT_IN_GRAPH_ORDER;\n> +\trevs.topo_order = 1;\n> +\n> +\trevs.pretty_given = 1;\n> +\tgit_config_get_string(\"rebase.instructionFormat\", &format);\n> +\tif (!format || !*format) {\n> +\t\tfree(format);\n> +\t\tformat = xstrdup(\"%s\");\n> +\t}\n> +\tget_commit_format(format, &revs);\n> +\tfree(format);\n> +\tpp.fmt = revs.commit_format;\n> +\tpp.output_encoding = get_log_output_encoding();\n> +\n> +\tif (setup_revisions(argc, argv, &revs, NULL) > 1)\n> +\t\treturn error(_(\"make_script: unhandled options\"));\n> +\n> +\tif (prepare_revision_walk(&revs) < 0)\n> +\t\treturn error(_(\"make_script: error preparing revisions\"));\n> +\n> +\twhile ((commit = get_revision(&revs))) {\n> +\t\tstrbuf_reset(&buf);\n> +\t\tif (!keep_empty && is_original_commit_empty(commit))\n> +\t\t\tstrbuf_addf(&buf, \"%c \", comment_line_char);\n> +\t\tstrbuf_addf(&buf, \"pick %s \", oid_to_hex(&commit->object.oid));\n> +\t\tpretty_print_commit(&pp, commit, &buf);\n> +\t\tstrbuf_addch(&buf, '\\n');\n> +\t\tfputs(buf.buf, out);\n> +\t}\n> +\tstrbuf_release(&buf);\n> +\treturn 0;\n> +}\n> diff --git a/sequencer.h b/sequencer.h\n> index f885b68395f..83f2943b7a9 100644\n> --- a/sequencer.h\n> +++ b/sequencer.h\n> @@ -45,6 +45,9 @@ int sequencer_continue(struct replay_opts *opts);\n> int sequencer_rollback(struct replay_opts *opts);\n> int sequencer_remove_state(struct replay_opts *opts);\n> \n> +int sequencer_make_script(int keep_empty, FILE *out,\n> +\t\tint argc, const char **argv);\n> +\n> extern const char sign_off_header[];\n> \n> void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);\n> -- \n> 2.12.2.windows.2.800.gede8f145e06\n\nIf you want, I could add to what I did in \"rebase -i: add config to\nabbreviate command-names\"[1] and try to resolve these naming inconsistencies\nin a separate patch.\n\nLiam\n\n[1]: https://public-inbox.org/git/20170502040048.9065-1-liambeguin@gmail.com/\n"},{"id":"321076","messageId":"CACBZZX6h1qEC5Drt0LzNLh=_ZXFTHjGCa=DDbhLUMtK9KjPjCg@mail.gmail.com","threadId":"43983","inReplyTo":"alpine.DEB.2.21.1.1705301635440.3610@virtualbox","subject":"Re: [PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-30T20:22:31Z","receivedAt":"2017-05-30T20:23:05Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 30, 2017 at 5:44 PM, Johannes Schindelin\n wrote:\n> Hi Ævar,\n>\n> On Mon, 29 May 2017, Ævar Arnfjörð Bjarmason wrote:\n>\n>> On Mon, May 29, 2017 at 12:51 PM, Johannes Schindelin\n>> wrote:\n>> >\n>> > On Sat, 27 May 2017, René Scharfe wrote:\n>> >> Am 26.05.2017 um 05:15 schrieb Liam Beguin:\n>> >> > I tried to time the execution on an interactive rebase (on Linux)\n>> >> > but I did not notice a significant change in speed. Do we have a\n>> >> > way to measure performance / speed changes between version?\n>> >>\n>> >> Well, there's performance test script p3404-rebase-interactive.sh.\n>> >> You could run it e.g. like this:\n>> >>\n>> >> $ (cd t/perf && ./run origin/master HEAD ./p3404*.sh)\n>> >>\n>> >> This would compare the performance of master with the current branch\n>> >> you're on. The results of p3404 are quite noisy for me on master,\n>> >> though (saw 15% difference between runs without any code changes), so\n>> >> take them with a bag of salt.\n>> >\n>> > Indeed. Our performance tests are simply not very meaningful.\n>> >\n>> > Part of it is the use of shell scripting (which defeats performance\n>> > testing pretty well),\n>>\n>> Don't the performance tests take long enough that the shellscripting\n>> overhead gets lost in the noise?\n>\n> Okay, here you go, my weekly (or so) clarification about the POSIX\n> emulation layer called MSYS2 (which itself kind of a portable Cygwin).\n>\n> Whenever Git for Windows has to execute Unix shell scripts (which are not\n> native to Windows, as the \"Unix\" in \"Unix shell scripts\" so clearly\n> suggests), we resort to calling the Bash from the MSYS2 project, which\n> spins up a POSIX emulation layer. Git for Windows' own .exe files\n> (and in particular, git.exe) is *not* affected by the POSIX emulation\n> layer, as they are real Win32 programs.\n>\n> Whenever execution has to bridge into, or out of, the POSIX emulation\n> layer, a few things need to be done. To emulate signal handling, for\n> example, a completely new process has to be spun up that itself has the\n> non-MSYS2 process as a child. The environment has to be converted, to\n> reflect the fact that some things are Unix-y paths (or path lists) inside\n> the POSIX emulation layer and Windows paths outside.\n>\n> Even when staying within the POSIX emulation layer, some operations are\n> not as cheap as \"Linux folks\" are used to. For example, to spawn a\n> subshell, due to the absence of a spawn syscall fork() is called, followed\n> by exec(). However, fork() itself is not native to Windows and has to be\n> emulated. The POSIX emulation layer spins up a new process, meticulously\n> copies the entire memory, tries to reopen the file descriptors, network\n> connections, etc (to emulate the fork() semantics).\n>\n> Obviously, this is anything but cheap.\n>\n> And this is only a glimpse into the entire problem, as I am not aware of\n> any thorough analysis what is going on in msys-2.0.dll when shell scripts\n> run. All I know is that it slows things down dramatically.\n>\n> As a consequence, even the simple act of creating a repository, or\n> spawning Win32 processes from within a shell, become quite the\n> contributing factors to the noise of the measurements.\n>\n>> E.g. on Windows what do you get when you run this in t/perf:\n>>\n>> $ GIT_PERF_REPEAT_COUNT=3 GIT_PERF_MAKE_OPTS=\"-j6 NO_OPENSSL=Y\n>> BLK_SHA1=Y CFLAGS=-O3\" ./run v2.10.0 v2.12.0 v2.13.0 p3400-rebase.sh\n>\n> In my hands, a repeat count of 3 always resulted in quite a bit of noise\n> previously.\n>\n> Mind you, I am working my machine. It has to run two VMs in the\n> background, has multiple browsers and dozens of tabs open, checks for\n> mails and Tweets and RSS feeds and a couple of Skypes are open, too.\n>\n> So yeah, obviously there is a bit of noise involved.\n>\n>> I get split-index performance improving by 28% in 2.12 and 58% in\n>> 2.13, small error bars even with just 3 runs. This is on Linux, but my\n>> sense of fork overhead on Windows is that it isn't so bad as to matter\n>> here.\n>\n> Test\n> ------------------------------------------------------\n> 3400.2: rebase on top of a lot of unrelated changes\n>\n> v2.10.0 v2.12.0 v2.13.0\n> ------------------------------------------------------------------\n> 60.65(0.01+0.03) 55.75(0.01+0.07) -8.1% 55.97(0.04+0.09) -7.7%\n>\n> (wrapped myself, as the ./run output is a lot wider than the 80 columns\n> allowed in plain text email format)\n>\n> And what does it tell you?\n\nFor all the above: Yes I get that various things are slower on\nWindows, but I think that regardless of that by far the majority of\nthe time in most of the perf tests is spent on the code being tested,\nso it doesn't come into play:\n\n$ parallel -k -j4 '(printf \"%s\\t\" {} && (time GIT_PERF_REPO=~/g/linux\nGIT_PERF_LARGE_REPO=~/g/linux ./run {}) 2>&1 | grep ^real | grep -o\n[0-9].*) | awk \"{print \\$2 \\\" \\\" \\$1}\"' ::: p[0-9]*sh\n0m34.333s p0000-perf-lib-sanity.sh\n4m26.415s p0001-rev-list.sh\n1m41.647s p0002-read-cache.sh\n28m45.001s p0003-delta-base-cache.sh\n0m55.767s p0004-lazy-init-name-hash.sh\n0m15.891s p0005-status.sh\n3m53.143s p0006-read-tree-checkout.sh\n3m28.627s p0071-sort.sh\n0m49.541s p0100-globbing.sh\n14m42.273s p3400-rebase.sh\n0m0.573s p3404-rebase-interactive.sh\n0m54.103s p4000-diff-algorithms.sh\n0m8.102s p4001-diff-no-index.sh\n17m34.819s p4211-line-log.sh\n4m50.497s p4220-log-grep-engines.sh\n3m59.636s p4221-log-grep-engines-fixed.sh\n\nI.e. I'd expct this to come into play with e.g.\np4000-diff-algorithms.sh, but most of the time spent in e.g.\np4220-log-grep-engines.sh on any non-trivially sized repo is hanging\non git-grep to finish.\n\nAlso if you look at test_run_perf_ in perf-lib.sh we only\n/usr/bin/time some tiny shell code, actually I can't see now why we're\nnot just doing *only* /usr/bin/time , right now we\nmeasure some setup code & sourcing the shell library too.\n\n> Not much, right? You have no idea about the trend line of the three tests,\n> not even of the standard deviation (not that it would be meaningful for\n> N=3). It is not immediately obvious whether the first run is always a tad\n> slower (or faster), or whether there is no noticable difference between\n> the first and any subsequent runs.\n\nRight, this could really be improved, but it's purely in the\npresentation layer, we log the raw runtimes:\n\n $ GIT_PERF_REPEAT_COUNT=6 ./p4220-log-grep-engines.sh\n $ cat trash\\ directory.p4220-log-grep-engines/test_time.*\n 0:00.01 0.01 0.00\n 0:00.01 0.01 0.00\n 0:00.02 0.01 0.00\n 0:00.02 0.02 0.00\n 0:00.01 0.01 0.00\n 0:00.02 0.01 0.00\n\n> In other words, there is no measure of confidence in those results. We\n> can't say how reliable those numbers are.\n>\n> And we certainly can't know how much the shell scripting hurts.\n>\n> Although... let's try something. Let's run the same command in a *Linux\n> VM* on the same machine! Yes, that should give us an idea. So here goes:\n>\n> Test\n> ------------------------------------------------------\n> 3400.2: rebase on top of a lot of unrelated changes\n>\n> v2.10.0 v2.12.0 v2.13.0\n> ---------------------------------------------------------------\n> 2.08(1.76+0.15) 2.10(1.76+0.15) +1.0% 2.00(1.65+0.15) -3.8%\n>\n>\n> A ha! Not only does this show a curious *increase* in v2.12.0 (but I'd not\n> put much stock into that, again N=3 is way too low a repetition number),\n> it also shows that the Linux VM runs the same thing roughly 30x faster.\n>\n> I did see a few speed differences between native git.exe on Windows and\n> the git executable on Linux, but it was barely in the two-digit\n> *percentage* region. Nowhere near the four-digit percentage region.\n>\n> So now you know how much shell scripting hurts performance testing.\n>\n> A lot.\n\nMaybe that rebase test is just very fuzzy. Could you try to run this please:\n\n$ cat p0000-perf-lib-sleep.sh\n#!/bin/sh\n\ntest_description=\"Tests sleep performance\"\n\n. ./perf-lib.sh\n\ntest_perf_default_repo\n\nfor s in 1 2 5\ndo\n test_perf \"sleep $s\" \"sleep $s\"\ndone\n\ntest_done\n\nI get:\n\n$ ./run p0000-perf-lib-sleep.sh\n=== Running 1 tests in this tree ===\nperf 1 - sleep 1: 1 2 3 ok\nperf 2 - sleep 2: 1 2 3 ok\nperf 3 - sleep 5: 1 2 3 ok\n# passed all 3 test(s)\n1..3\nTest this tree\n---------------------------------\n0000.1: sleep 1 1.00(0.00+0.00)\n0000.2: sleep 2 2.00(0.00+0.00)\n0000.3: sleep 5 5.00(0.00+0.00)\n\nI'd be very curious to see if you get any different results on any of\nyour systems, particularly Windows.\n\nUnless I'm missing something, if those numbers are 1, 2, 5 or close\nenough and with small enough error bars, then the perf framework's use\nof shellscripting should be fine, if not we've got something...\n\n> It pretty much renders the entire endeavor of testing performance\n> completely and utterly useless.\n>\n>> I'd also be interested to see what sort of results you get for my\n>> \"grep: add support for the PCRE v1 JIT API\" patch which is in pu now,\n>> assuming you have a PCRE newer than 8.32 or so.\n>\n> pu does not build for me:\n>\n> 2017-05-30T11:38:50.0089681Z libgit.a(grep.o): In function `pcre1match':\n> 2017-05-30T11:38:50.0289250Z .../grep.c:411: undefined reference to `__imp_pcre_jit_exec'\n> 2017-05-30T11:38:50.0329160Z collect2.exe: error: ld returned 1 exit\n> status\n\nOuch, looks like I've missed some spot in my pcre1 jit series. What's\nthe PCRE version you have? This is somehow doing the wrong thing with\nthis bit in grep.h:\n\n #include \n #ifdef PCRE_CONFIG_JIT\n #if PCRE_MAJOR >= 8 && PCRE_MINOR >= 32\n #define GIT_PCRE1_USE_JIT\n [...]\n\n>> > Frankly, I have no illusion about this getting fixed, ever.\n>>\n>> I have a project on my TODO that I've been meaning to get to which\n>> would address this. I'd be interested to know what people think about\n>> the design:\n>>\n>> * Run the perf tests in some more where the raw runtimes are saved away\n>\n> You mean a test helper designed to do the timing and the setting up so as\n> to time *just* the operations that should be timed?\n>\n> If so: I am all for it.\n\nNo I just mean save away the raw trash*/test_time.* files, so you can\ndo any sort of analysis on them later, e.g. render stdev, percentiles\netc. But this is assuming the raw numbers are useful, let's find that\nout.\n\n>> * Have some way to dump a static html page from that with graphs over\n>> time (with gnuplot svg?)\n>\n> If you already go HTML, it would make much more sense to go d3.js. I would\n> even prefer to go c3.js (which uses d3.js) right away. Would make\n> everything so much easier.\n>\n> Not to mention more portable.\n>\n>> * Supply some config file to drive this, so you can e.g. run each\n>> tests N times against your repo X for the last 10 versions of git.\n>\n> Sure.\n>\n>> * Since it's static HTML it would be trivial for anyone to share such\n>> results, and e.g. setup running them in cron to regularly publish to\n>> github pages.\n>\n> It does not need to be static. It can use, say, c3.js, for the added\n> benefit of being able to toggle multiple graphs in the same diagram.\n\nRight, I mean \"static\" in the sense that it wouldn't require some\ndynamic backend, it requiring JS is fine, i.e. I'd just like someone\nto be able to publish some raw data, point some JS/html at it, and\nhave it be rendered.\n"},{"id":"321088","messageId":"xmqq1sr6arda.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"alpine.DEB.2.21.1.1705301700440.3610@virtualbox","subject":"Re: revision API design, was Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-30T22:53:37Z","receivedAt":"2017-05-30T22:53:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n>> See 66b2ed09 (\"Fix \"log\" family not to be too agressive about\n>> ...\n>> ask the existing command line parser to set them for you.\n>\n> This is a very eloquent description of a problem with the API.\n\nYes, but ...\n\n> The correct suggestion would be to fix the API, of course. Not to declare\n> an interface intended for command-line parsing the internal API.\n>\n> Maybe the introduction of `pretty_given` was a strong hint at a design\n> flaw to begin with, pointing to the fact that user_format is a singleton\n> (yes, really, you can't have two different user_formats in the same Git\n> process, what were we thinking)?\n\n... this tells me that you do not understand the issue. The\nembarrasing but necessary pretty-given field was not about\nuser_format (let alone singleton-ness of it) at all. It was about\nthe show_notes feature that wants to be on by default most of the\ntime, but needs to be defeated when the end user asked for certain\nformats.\n\nQuite frankly, I am not interested in listening to a proposal to\nupdate API by a person who does not understand the issue in the API,\nbut that is a separate issue. A more important thing is that the\nupdate to \"rebase -i\" is important enough and we do not want to\ndelay it by introducing further dependency. IOW, I do not want to\nspend an extra development cycle or two to update the revision setup\nAPI and have you rebase the series on top after the API update is\ndone.\n\nThe current API to hide such an embarrasing but necessary\nimplementation details from the code that does not need to know\nthem, i.e. the consumer of rev-info structure with various settings,\nis to invoke the command line parser. Bypassing and going directly\ndown to the internal implementation detail of rev_info _is_ being\nsloppy. I would strongly prefer to see that the current series\nwritten for the API to allow us get it over with the \"rebase -i\"\nthing, and think revision setup API separately and later.\n"},{"id":"321152","messageId":"CACBZZX5tyWY0k6m=7HcZJZanDmrzAQX+3mVDQx-jzqV_Wr_tMQ@mail.gmail.com","threadId":"43983","inReplyTo":"CACBZZX6h1qEC5Drt0LzNLh=_ZXFTHjGCa=DDbhLUMtK9KjPjCg@mail.gmail.com","subject":"Re: [PATCH v4 00/10] The final building block for a faster rebase -i","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-31T18:46:38Z","receivedAt":"2017-05-31T18:47:05Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 30, 2017 at 10:22 PM, Ævar Arnfjörð Bjarmason\n wrote:\n> On Tue, May 30, 2017 at 5:44 PM, Johannes Schindelin\n> wrote:\n>> pu does not build for me:\n>>\n>> 2017-05-30T11:38:50.0089681Z libgit.a(grep.o): In function `pcre1match':\n>> 2017-05-30T11:38:50.0289250Z .../grep.c:411: undefined reference to `__imp_pcre_jit_exec'\n>> 2017-05-30T11:38:50.0329160Z collect2.exe: error: ld returned 1 exit\n>> status\n>\n> Ouch, looks like I've missed some spot in my pcre1 jit series. What's\n> the PCRE version you have? This is somehow doing the wrong thing with\n> this bit in grep.h:\n>\n> #include \n> #ifdef PCRE_CONFIG_JIT\n> #if PCRE_MAJOR >= 8 && PCRE_MINOR >= 32\n> #define GIT_PCRE1_USE_JIT\n> [...]\n\nI've found what the problem is myself. I'll submit a new version of\nthe series that fixes this.\n"},{"id":"321251","messageId":"xmqqy3tc5hl3.fsf@gitster.mtv.corp.google.com","threadId":"43983","inReplyTo":"xmqq1sr6arda.fsf@gitster.mtv.corp.google.com","subject":"Re: revision API design, was Re: [PATCH v4 02/10] rebase -i: generate the script via rebase--helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-01T06:48:24Z","receivedAt":"2017-06-01T06:48:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> The current API to hide such an embarrasing but necessary\n> implementation details from the code that does not need to know\n> them, i.e. the consumer of rev-info structure with various settings,\n> is to invoke the command line parser. Bypassing and going directly\n> down to the internal implementation detail of rev_info _is_ being\n> sloppy. I would strongly prefer to see that the current series\n> written for the API to allow us get it over with the \"rebase -i\"\n> thing, and think revision setup API separately and later.\n\nAn updated API that hides the implementation details may look like\na vast enhancement of this patch.\n\nI say \"vast\" because we need to do this for _all_ of the \"else if\"\ncascade you see in revision.c, and probably for fields set by other\nhelper functions in the same file. Otherwise, it doesn't have much\nvalue.\n\nAnybody who is tempted to say \"We need to do these only for the\ncomplex ones, like the one that needs to set revs->pretty_given\nwhile setting something else\" hasn't learned from the example of\n66b2ed09. Interactions between options start happening when new\noptions that are added, and that is when handling of a seemingly\nunrelated old option that used to be very simple needs to do more in\nthe new world order. And that is why this illustration has a\nwrapper for \"--first-parent-only\", which happens to be very simple\ntoday.\n\nWe do not want revision traversal API's customers to write\n\n\trevs.first_parent_only = 1;\n\nbecause that's mucking with the implementation detail. The current\nAPI to hide that detail is:\n\n\tmemset(&revs, 0, sizeof(revs);\n\targv_pushl(&args, \"--first-parent-only\", NULL);\n\t... may be more options ...\n\tsetup_revisions(args.argc, args.argv, &revs, ...);\n\nand\n\n\tmemset(&revs, 0, sizeof(revs);\n\trev_option_first_parent_only(&revs);\n\t... may be more options ...\n\tsetup_revisions(0, NULL, &revs, ...);\n\nwould be a more statically-checked rewrite, if such an API was\navailable.\n\n revision-internal.h | 18 ++++++++++++++++++\n revision.c | 9 +++------\n 2 files changed, 21 insertions(+), 6 deletions(-)\n\ndiff --git a/revision-internal.h b/revision-internal.h\nnew file mode 100644\nindex 0000000000..dea4c48412\n--- /dev/null\n+++ b/revision-internal.h\n@@ -0,0 +1,18 @@\n+#ifndef REVISION_INTERNAL_H\n+#define REVISION_INTERNAL_H 1\n+\n+static inline void rev_option_first_parent_only(struct rev_info *revs)\n+{\n+\trevs->first_parent_only = 1;\n+}\n+\n+static inline void rev_option_simplify_merges(struct rev_info *revs)\n+{\n+\trevs->simplify_merges = 1;\n+\trevs->topo_order = 1;\n+\trevs->rewrite_parents = 1;\n+\trevs->simplify_history = 0;\n+\trevs->limited = 1;\n+}\n+\n+#endif\ndiff --git a/revision.c b/revision.c\nindex 265611e01f..9418676264 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -20,6 +20,7 @@\n #include \"cache-tree.h\"\n #include \"bisect.h\"\n #include \"worktree.h\"\n+#include \"revision-internal.h\"\n \n volatile show_early_output_fn_t show_early_output;\n \n@@ -1807,7 +1808,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n \t\trevs->min_age = approxidate(optarg);\n \t\treturn argcount;\n \t} else if (!strcmp(arg, \"--first-parent\")) {\n-\t\trevs->first_parent_only = 1;\n+\t\trev_option_first_parent_only(&revs);\n \t} else if (!strcmp(arg, \"--ancestry-path\")) {\n \t\trevs->ancestry_path = 1;\n \t\trevs->simplify_history = 0;\n@@ -1825,11 +1826,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n \t\trevs->sort_order = REV_SORT_IN_GRAPH_ORDER;\n \t\trevs->topo_order = 1;\n \t} else if (!strcmp(arg, \"--simplify-merges\")) {\n-\t\trevs->simplify_merges = 1;\n-\t\trevs->topo_order = 1;\n-\t\trevs->rewrite_parents = 1;\n-\t\trevs->simplify_history = 0;\n-\t\trevs->limited = 1;\n+\t\trev_option_simplify_merges(&revs);\n \t} else if (!strcmp(arg, \"--simplify-by-decoration\")) {\n \t\trevs->simplify_merges = 1;\n \t\trevs->topo_order = 1;\n"}]}