{"thread":{"id":"46038","subject":"[PATCHv5 00/17] Diff machine: highlight moved lines.","startedAt":"2017-05-24T21:40:51Z","lastAt":"2017-06-23T21:43:50Z","messageCount":125,"participants":["Stefan Beller","Junio C Hamano","Philip Oakley","Jonathan Tan","Brandon Williams","Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"320665","messageId":"20170524214036.29623-1-sbeller@google.com","threadId":"46038","inReplyTo":"20170523024048.16879-1-sbeller@google.com/","subject":"[PATCHv5 00/17] Diff machine: highlight moved lines.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:19Z","receivedAt":"2017-05-24T21:40:51Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"v5:\n* removed the color passing to the submodule to make the tests pass again.\n* fixed an indentation issue that was introduced from v3 -> v4.\n* I merged it with origin/next and tests pass here.\n\nThanks,\nStefan\n\ndiff to v4:\ndiff --git a/diff.c b/diff.c\nindex 23e70d348e..1292d3c4ad 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -751,7 +751,7 @@ static void mark_color_as_moved(struct diff_options *o,\n }\n \n static void emit_diff_line(struct diff_options *o,\n-\t\t\t\t struct diff_line *e)\n+\t\t\t struct diff_line *e)\n {\n \tconst char *ws;\n \tint has_trailing_newline, has_trailing_carriage_return;\n@@ -804,7 +804,7 @@ static void emit_diff_line(struct diff_options *o,\n }\n \n static void append_diff_line(struct diff_options *o,\n-\t\t\t\t struct diff_line *e)\n+\t\t\t struct diff_line *e)\n {\n \tstruct diff_line *f;\n \tALLOC_GROW(o->line_buffer,\ndiff --git a/submodule.c b/submodule.c\nindex 428c996c97..19c63197fb 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -550,8 +550,6 @@ void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \n \t/* TODO: other options may need to be passed here. */\n \targv_array_push(&cp.args, \"diff\");\n-\tif (o->use_color)\n-\t\targv_array_push(&cp.args, \"--color=always\");\n \targv_array_pushf(&cp.args, \"--line-prefix=%s\", diff_line_prefix(o));\n \tif (DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\targv_array_pushf(&cp.args, \"--src-prefix=%s%s/\",\n\nv4:\n* interdiff to v3 (what is currently origin/sb/diff-color-move) below.\n* renamed the \"buffered_patch_line\" to \"diff_line\". Originally I planned\n to not carry the \"line\" part as it can be a piece of a line as well.\n But for the intended functionality it is best to keep the name.\n If we'd want to add more functionality to say have a move detection\n for words as well, we'd rename the struct to have a better name then.\n For now diff_line is the best. (Thanks Jonathan Nieder!)\n* tests to demonstrate it doesn't mess with --color-words as well as\n submodules. (Thanks Jonathan Tan!)\n* added in the statics (Thanks Ramsay!)\n* smaller scope for the hashmaps (Thanks Jonathan Tan!)\n* some commit messages were updated, prior patch 4-7 is squashed into one\n (Thanks Jonathan Tan!)\n* the tests added revealed an actual fault: now that the submodule process\n is not attached to a dupe of our stdout, it would stop coloring the\n output. We need to pass on use-color explicitly.\n* updated the NEEDSWORK comment in the second last patch.\n\nThanks for bearing,\nStefan\n\nv3:\n* see interdiff below.\n* fixing one invalid computation (Thanks Junio!)\n* I reasoned more about submodule and word diffing, see the commit message\n of the last patch:\n \n A note on the options '--submodule=diff' and '--color-words/--word-diff':\n In the conversion to use emit_line in the prior patches both submodules\n as well as word diff output carefully chose to call emit_line with sign=0.\n All output with sign=0 is ignored for move detection purposes in this\n patch, such that no weird looking output will be generated for these\n cases. This leads to another thought: We could pass on '--color-moved' to\n submodules such that they color up moved lines for themselves. If we'd do\n so only line moves within a repository boundary are marked up.\n\n* better name for emit_line outside of diff.[ch]\n\nv2:\n* emit_line now takes an argument that indicates if we want it\n to emit the line prefix as well. This should allow for a more faithful\n refactoring in the beginning. (Thanks Jonathan!)\n* fixed memleaks (Thanks Brandon!)\n* \"git -c color.moved=true log -p\" works now! (Thanks Jeff)\n* interdiff below, though it is large.\n* less intrusive than v1 (Thanks Jonathan!)\n\nv1:\n\nFor details on *why* see the commit message of the last commit.\n\nThe first five patches are slight refactorings to get into good\nshape, the next patches are funneling all output through emit_line_*.\n\nThe second last patch introduces an option to buffer up all output\nbefore printing, and then the last patch can color up moved lines\nof code.\n\nAny feedback welcome.\n\nThanks,\nStefan\n\nStefan Beller (17):\n diff: readability fix\n diff: move line ending check into emit_hunk_header\n diff.c: factor out diff_flush_patch_all_file_pairs\n diff: introduce more flexible emit function\n diff.c: convert fn_out_consume to use emit_line\n diff.c: convert builtin_diff to use emit_line_*\n diff.c: convert emit_rewrite_diff to use emit_line_*\n diff.c: convert emit_rewrite_lines to use emit_line_*\n submodule.c: convert show_submodule_summary to use emit_line_fmt\n diff.c: convert emit_binary_diff_body to use emit_line_*\n diff.c: convert show_stats to use emit_line_*\n diff.c: convert word diffing to use emit_line_*\n diff.c: convert diff_flush to use emit_line_*\n diff.c: convert diff_summary to use emit_line_*\n diff.c: emit_line includes whitespace highlighting\n diff: buffer all output if asked to\n diff.c: color moved lines differently\n\n Documentation/config.txt | 14 +-\n diff.c | 858 +++++++++++++++++++++++++++++++++------------\n diff.h | 59 +++-\n submodule.c | 87 ++---\n submodule.h | 9 +-\n t/t4015-diff-whitespace.sh | 267 ++++++++++++++\n 6 files changed, 1016 insertions(+), 278 deletions(-)\n\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320666","messageId":"20170524214036.29623-2-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 01/17] diff: readability fix","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:20Z","receivedAt":"2017-05-24T21:41:00Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We already have dereferenced 'p->two' into a local variable 'two'. Use\nthat.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 74283d9001..3f5bf8b5a4 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3283,8 +3283,8 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \tconst char *other;\n \tconst char *attr_path;\n \n-\tname = p->one->path;\n-\tother = (strcmp(name, p->two->path) ? p->two->path : NULL);\n+\tname = one->path;\n+\tother = (strcmp(name, two->path) ? two->path : NULL);\n \tattr_path = name;\n \tif (o->prefix_length)\n \t\tstrip_prefix(o->prefix_length, &name, &other);\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320667","messageId":"20170524214036.29623-4-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 03/17] diff.c: factor out diff_flush_patch_all_file_pairs","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:22Z","receivedAt":"2017-05-24T21:41:01Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we want to do more things before and after all filepairs\nare flushed. So factor flushing out all file pairs into its own function\nthat the new code can be plugged in easily.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 17 ++++++++++++-----\n 1 file changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex c2ed605cd0..2f9722b382 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4737,6 +4737,17 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n \t\twarning(_(rename_limit_advice), varname, needed);\n }\n \n+static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n+{\n+\tint i;\n+\tstruct diff_queue_struct *q = &diff_queued_diff;\n+\tfor (i = 0; i < q->nr; i++) {\n+\t\tstruct diff_filepair *p = q->queue[i];\n+\t\tif (check_pair_status(p))\n+\t\t\tdiff_flush_patch(p, o);\n+\t}\n+}\n+\n void diff_flush(struct diff_options *options)\n {\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n@@ -4831,11 +4842,7 @@ void diff_flush(struct diff_options *options)\n \t\t\t}\n \t\t}\n \n-\t\tfor (i = 0; i < q->nr; i++) {\n-\t\t\tstruct diff_filepair *p = q->queue[i];\n-\t\t\tif (check_pair_status(p))\n-\t\t\t\tdiff_flush_patch(p, options);\n-\t\t}\n+\t\tdiff_flush_patch_all_file_pairs(options);\n \t}\n \n \tif (output_format & DIFF_FORMAT_CALLBACK)\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320668","messageId":"20170524214036.29623-10-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 09/17] submodule.c: convert show_submodule_summary to use emit_line_fmt","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:28Z","receivedAt":"2017-05-24T21:41:04Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nThis prepares the code for submodules to go through the emit_line\nfunction.\n\nAs the submodule process is no longer attached to the same stdout as\nthe superprojects process, one might imagine that we would need to\npass on the usage of colors explicitly as the subprocess can no longer\ndetermine where the output is going to land eventually. But this is not\nthe case. Apparently coloring submodule diffs never worked, so defer\nthe submodule diff coloring to a future patch series.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 14 ++++++----\n diff.h | 3 +++\n submodule.c | 87 ++++++++++++++++++++++++++++++++-----------------------------\n submodule.h | 9 +++----\n 4 files changed, 61 insertions(+), 52 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex ca6b48cf49..3357c0fca3 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -562,6 +562,12 @@ static void emit_line_fmt(struct diff_options *o,\n \tstrbuf_release(&sb);\n }\n \n+void diff_emit_line(struct diff_options *o, const char *set, const char *reset,\n+\t\t const char *line, int len)\n+{\n+\temit_line(o, set, reset, 1, 0, line, len);\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -2384,8 +2390,7 @@ static void builtin_diff(const char *name_a,\n \t (!two->mode || S_ISGITLINK(two->mode))) {\n \t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n \t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_summary(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_summary(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n \t\t\t\ttwo->dirty_submodule,\n \t\t\t\tmeta, del, add, reset);\n@@ -2395,11 +2400,10 @@ static void builtin_diff(const char *name_a,\n \t\t (!two->mode || S_ISGITLINK(two->mode))) {\n \t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n \t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_inline_diff(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_inline_diff(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n \t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset, o);\n+\t\t\t\tmeta, del, add, reset);\n \t\treturn;\n \t}\n \ndiff --git a/diff.h b/diff.h\nindex 5be1ee77a7..9ad546361a 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,9 @@ struct diff_options {\n \tint diff_path_counter;\n };\n \n+void diff_emit_line(struct diff_options *o, const char *set, const char *reset,\n+\t\t const char *line, int len);\n+\n enum color_diff {\n \tDIFF_RESET = 0,\n \tDIFF_CONTEXT = 1,\ndiff --git a/submodule.c b/submodule.c\nindex d3299e29c0..19c63197fb 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -362,8 +362,8 @@ static int prepare_submodule_summary(struct rev_info *rev, const char *path,\n \treturn prepare_revision_walk(rev);\n }\n \n-static void print_submodule_summary(struct rev_info *rev, FILE *f,\n-\t\tconst char *line_prefix,\n+static void print_submodule_summary(struct rev_info *rev,\n+\t\tstruct diff_options *o,\n \t\tconst char *del, const char *add, const char *reset)\n {\n \tstatic const char format[] = \" %m %s\";\n@@ -375,18 +375,12 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \t\tctx.date_mode = rev->date_mode;\n \t\tctx.output_encoding = get_log_output_encoding();\n \t\tstrbuf_setlen(&sb, 0);\n-\t\tstrbuf_addstr(&sb, line_prefix);\n-\t\tif (commit->object.flags & SYMMETRIC_LEFT) {\n-\t\t\tif (del)\n-\t\t\t\tstrbuf_addstr(&sb, del);\n-\t\t}\n-\t\telse if (add)\n-\t\t\tstrbuf_addstr(&sb, add);\n \t\tformat_commit_message(commit, format, &sb, &ctx);\n-\t\tif (reset)\n-\t\t\tstrbuf_addstr(&sb, reset);\n \t\tstrbuf_addch(&sb, '\\n');\n-\t\tfprintf(f, \"%s\", sb.buf);\n+\t\tif (commit->object.flags & SYMMETRIC_LEFT)\n+\t\t\tdiff_emit_line(o, del, reset, sb.buf, sb.len);\n+\t\telse if (add)\n+\t\t\tdiff_emit_line(o, add, reset, sb.buf, sb.len);\n \t}\n \tstrbuf_release(&sb);\n }\n@@ -413,8 +407,7 @@ void prepare_submodule_repo_env(struct argv_array *out)\n * attempt to lookup both the left and right commits and put them into the\n * left and right pointers.\n */\n-static void show_submodule_header(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+static void show_submodule_header(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n \t\tunsigned dirty_submodule, const char *meta,\n \t\tconst char *reset,\n@@ -425,12 +418,17 @@ static void show_submodule_header(FILE *f, const char *path,\n \tstruct strbuf sb = STRBUF_INIT;\n \tint fast_forward = 0, fast_backward = 0;\n \n-\tif (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED)\n-\t\tfprintf(f, \"%sSubmodule %s contains untracked content\\n\",\n-\t\t\tline_prefix, path);\n-\tif (dirty_submodule & DIRTY_SUBMODULE_MODIFIED)\n-\t\tfprintf(f, \"%sSubmodule %s contains modified content\\n\",\n-\t\t\tline_prefix, path);\n+\tif (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED) {\n+\t\tstrbuf_addf(&sb, \"Submodule %s contains untracked content\\n\", path);\n+\t\tdiff_emit_line(o, NULL, NULL, sb.buf, sb.len);\n+\t\tstrbuf_reset(&sb);\n+\t}\n+\n+\tif (dirty_submodule & DIRTY_SUBMODULE_MODIFIED) {\n+\t\tstrbuf_addf(&sb, \"Submodule %s contains modified content\\n\", path);\n+\t\tdiff_emit_line(o, NULL, NULL, sb.buf, sb.len);\n+\t\tstrbuf_reset(&sb);\n+\t}\n \n \tif (is_null_oid(one))\n \t\tmessage = \"(new submodule)\";\n@@ -472,21 +470,20 @@ static void show_submodule_header(FILE *f, const char *path,\n \t}\n \n output_header:\n-\tstrbuf_addf(&sb, \"%s%sSubmodule %s \", line_prefix, meta, path);\n+\tstrbuf_addf(&sb, \"Submodule %s \", path);\n \tstrbuf_add_unique_abbrev(&sb, one->hash, DEFAULT_ABBREV);\n \tstrbuf_addstr(&sb, (fast_backward || fast_forward) ? \"..\" : \"...\");\n \tstrbuf_add_unique_abbrev(&sb, two->hash, DEFAULT_ABBREV);\n \tif (message)\n-\t\tstrbuf_addf(&sb, \" %s%s\\n\", message, reset);\n+\t\tstrbuf_addf(&sb, \" %s\\n\", message);\n \telse\n-\t\tstrbuf_addf(&sb, \"%s:%s\\n\", fast_backward ? \" (rewind)\" : \"\", reset);\n-\tfwrite(sb.buf, sb.len, 1, f);\n+\t\tstrbuf_addf(&sb, \"%s:\\n\", fast_backward ? \" (rewind)\" : \"\");\n+\tdiff_emit_line(o, meta, reset, sb.buf, sb.len);\n \n \tstrbuf_release(&sb);\n }\n \n-void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n \t\tunsigned dirty_submodule, const char *meta,\n \t\tconst char *del, const char *add, const char *reset)\n@@ -495,7 +492,7 @@ void show_submodule_summary(FILE *f, const char *path,\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n \t\t\t meta, reset, &left, &right, &merge_bases);\n \n \t/*\n@@ -508,11 +505,12 @@ void show_submodule_summary(FILE *f, const char *path,\n \n \t/* Treat revision walker failure the same as missing commits */\n \tif (prepare_submodule_summary(&rev, path, left, right, merge_bases)) {\n-\t\tfprintf(f, \"%s(revision walker failed)\\n\", line_prefix);\n+\t\tconst char *error = \"(revision walker failed)\\n\";\n+\t\tdiff_emit_line(o, NULL, NULL, error, strlen(error));\n \t\tgoto out;\n \t}\n \n-\tprint_submodule_summary(&rev, f, line_prefix, del, add, reset);\n+\tprint_submodule_summary(&rev, o, del, add, reset);\n \n out:\n \tif (merge_bases)\n@@ -521,20 +519,18 @@ void show_submodule_summary(FILE *f, const char *path,\n \tclear_commit_marks(right, ~0);\n }\n \n-void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n \t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *o)\n+\t\tconst char *del, const char *add, const char *reset)\n {\n \tconst struct object_id *old = &empty_tree_oid, *new = &empty_tree_oid;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n-\tstruct strbuf submodule_dir = STRBUF_INIT;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n \t\t\t meta, reset, &left, &right, &merge_bases);\n \n \t/* We need a valid left and right commit to display a difference */\n@@ -547,15 +543,14 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \tif (right)\n \t\tnew = two;\n \n-\tfflush(f);\n \tcp.git_cmd = 1;\n \tcp.dir = path;\n-\tcp.out = dup(fileno(f));\n+\tcp.out = -1;\n \tcp.no_stdin = 1;\n \n \t/* TODO: other options may need to be passed here. */\n \targv_array_push(&cp.args, \"diff\");\n-\targv_array_pushf(&cp.args, \"--line-prefix=%s\", line_prefix);\n+\targv_array_pushf(&cp.args, \"--line-prefix=%s\", diff_line_prefix(o));\n \tif (DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\targv_array_pushf(&cp.args, \"--src-prefix=%s%s/\",\n \t\t\t\t o->b_prefix, path);\n@@ -578,11 +573,21 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \t\targv_array_push(&cp.args, oid_to_hex(new));\n \n \tprepare_submodule_repo_env(&cp.env_array);\n-\tif (run_command(&cp))\n-\t\tfprintf(f, \"(diff failed)\\n\");\n+\tif (start_command(&cp)) {\n+\t\tconst char *error = \"(diff failed)\\n\";\n+\t\tdiff_emit_line(o, NULL, NULL, error, strlen(error));\n+\t}\n+\n+\twhile (strbuf_getwholeline_fd(&sb, cp.out, '\\n') != EOF)\n+\t\tdiff_emit_line(o, NULL, NULL, sb.buf, sb.len);\n+\n+\tif (finish_command(&cp)) {\n+\t\tconst char *error = \"(diff failed)\\n\";\n+\t\tdiff_emit_line(o, NULL, NULL, error, strlen(error));\n+\t}\n \n done:\n-\tstrbuf_release(&submodule_dir);\n+\tstrbuf_release(&sb);\n \tif (merge_bases)\n \t\tfree_commit_list(merge_bases);\n \tif (left)\ndiff --git a/submodule.h b/submodule.h\nindex 1277480add..9df0a3aea2 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -53,17 +53,14 @@ extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n extern void handle_ignore_submodules_arg(struct diff_options *, const char *);\n-extern void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+extern void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n \t\tunsigned dirty_submodule, const char *meta,\n \t\tconst char *del, const char *add, const char *reset);\n-extern void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+extern void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n \t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *opt);\n+\t\tconst char *del, const char *add, const char *reset);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n /* Check if we want to update any submodule.*/\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320669","messageId":"20170524214036.29623-5-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 04/17] diff: introduce more flexible emit function","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:23Z","receivedAt":"2017-05-24T21:41:06Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently, diff output is written either through the emit_line_0\nfunction or through the FILE * in struct diff_options directly. To\nmake it easier to teach diff to buffer its output (which will be done\nin a subsequent commit), introduce a more flexible emit_line() function.\nIn this commit, direct usages of emit_line_0() are replaced with\nemit_line(); subsequent commits will also replace usages of the\nFILE * with emit().\n\nInstead of having a 'first' parameter containing the first character\nof a line, have a dedicated 'sign' parameter that is just set when\nthe first character of the line is part of the actual content, i.e.\n' ', '+', '-'.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 76 +++++++++++++++++++++++++++++-------------------------------------\n 1 file changed, 33 insertions(+), 43 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2f9722b382..3569857818 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -516,36 +516,30 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,\n \tecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n }\n \n-static void emit_line_0(struct diff_options *o, const char *set, const char *reset,\n-\t\t\tint first, const char *line, int len)\n+static void emit_line(struct diff_options *o, const char *set, const char *reset,\n+\t\t int add_line_prefix, int sign, const char *line, int len)\n {\n \tint has_trailing_newline, has_trailing_carriage_return;\n-\tint nofirst;\n \tFILE *file = o->file;\n \n-\tfputs(diff_line_prefix(o), file);\n+\tif (add_line_prefix)\n+\t\tfputs(diff_line_prefix(o), file);\n \n-\tif (len == 0) {\n-\t\thas_trailing_newline = (first == '\\n');\n-\t\thas_trailing_carriage_return = (!has_trailing_newline &&\n-\t\t\t\t\t\t(first == '\\r'));\n-\t\tnofirst = has_trailing_newline || has_trailing_carriage_return;\n-\t} else {\n-\t\thas_trailing_newline = (len > 0 && line[len-1] == '\\n');\n-\t\tif (has_trailing_newline)\n-\t\t\tlen--;\n-\t\thas_trailing_carriage_return = (len > 0 && line[len-1] == '\\r');\n-\t\tif (has_trailing_carriage_return)\n-\t\t\tlen--;\n-\t\tnofirst = 0;\n-\t}\n+\thas_trailing_newline = (len > 0 && line[len-1] == '\\n');\n+\tif (has_trailing_newline)\n+\t\tlen--;\n+\thas_trailing_carriage_return = (len > 0 && line[len-1] == '\\r');\n+\tif (has_trailing_carriage_return)\n+\t\tlen--;\n \n-\tif (len || !nofirst) {\n-\t\tfputs(set, file);\n-\t\tif (!nofirst)\n-\t\t\tfputc(first, file);\n+\tif (len || sign) {\n+\t\tif (set)\n+\t\t\tfputs(set, file);\n+\t\tif (sign)\n+\t\t\tfputc(sign, file);\n \t\tfwrite(line, len, 1, file);\n-\t\tfputs(reset, file);\n+\t\tif (reset)\n+\t\t\tfputs(reset, file);\n \t}\n \tif (has_trailing_carriage_return)\n \t\tfputc('\\r', file);\n@@ -553,12 +547,6 @@ static void emit_line_0(struct diff_options *o, const char *set, const char *res\n \t\tfputc('\\n', file);\n }\n \n-static void emit_line(struct diff_options *o, const char *set, const char *reset,\n-\t\t const char *line, int len)\n-{\n-\temit_line_0(o, set, reset, line[0], line+1, len-1);\n-}\n-\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -587,13 +575,13 @@ static void emit_line_checked(const char *reset,\n \t}\n \n \tif (!ws)\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, line, len);\n+\t\temit_line(ecbdata->opt, set, reset, 1, sign, line, len);\n \telse if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))\n \t\t/* Blank line at EOF - paint '+' as well */\n-\t\temit_line_0(ecbdata->opt, ws, reset, sign, line, len);\n+\t\temit_line(ecbdata->opt, ws, reset, 1, sign, line, len);\n \telse {\n \t\t/* Emit just the prefix, then the rest. */\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, \"\", 0);\n+\t\temit_line(ecbdata->opt, set, reset, 1, sign, \"\", 0);\n \t\tws_check_emit(line, len, ecbdata->ws_rule,\n \t\t\t ecbdata->opt->file, set, reset, ws);\n \t}\n@@ -643,7 +631,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tif (len < 10 ||\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n-\t\temit_line(ecbdata->opt, context, reset, line, len);\n+\t\temit_line(ecbdata->opt, context, reset, 1, 0, line, len);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n@@ -679,7 +667,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n \n-\temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n+\temit_line(ecbdata->opt, \"\", \"\", 1, 0, msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n \n@@ -742,8 +730,8 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tconst char *context = diff_get_color(ecb->color_diff,\n \t\t\t\t\t\t DIFF_CONTEXT);\n \t\tputc('\\n', ecb->opt->file);\n-\t\temit_line_0(ecb->opt, context, reset, '\\\\',\n-\t\t\t nneof, strlen(nneof));\n+\t\temit_line(ecb->opt, context, reset, 1, '\\\\',\n+\t\t\t nneof, strlen(nneof));\n \t}\n }\n \n@@ -1341,7 +1329,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t}\n \t\tdiff_words_flush(ecbdata);\n \t\tif (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {\n-\t\t\temit_line(o, context, reset, line, len);\n+\t\t\temit_line(o, context, reset, 1, 0, line, len);\n \t\t\tfputs(\"~\\n\", o->file);\n \t\t} else {\n \t\t\t/*\n@@ -1353,7 +1341,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t\t line++;\n \t\t\t len--;\n \t\t\t}\n-\t\t\temit_line(o, context, reset, line, len);\n+\t\t\temit_line(o, context, reset, 1, 0, line, len);\n \t\t}\n \t\treturn;\n \t}\n@@ -1376,7 +1364,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t/* incomplete line at the end */\n \t\tecbdata->lno_in_preimage++;\n \t\temit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),\n-\t\t\t reset, line, len);\n+\t\t\t reset, 1, 0, line, len);\n \t\tbreak;\n \t}\n }\n@@ -2188,7 +2176,7 @@ static void checkdiff_consume(void *priv, char *line, unsigned long len)\n \t\tfprintf(data->o->file, \"%s%s:%d: %s.\\n\",\n \t\t\tline_prefix, data->filename, data->lineno, err);\n \t\tfree(err);\n-\t\temit_line(data->o, set, reset, line, 1);\n+\t\temit_line(data->o, set, reset, 1, 0, line, 1);\n \t\tws_check_emit(line + 1, len - 1, data->ws_rule,\n \t\t\t data->o->file, set, reset, ws);\n \t} else if (line[0] == ' ') {\n@@ -4833,9 +4821,11 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\tfprintf(options->file, \"%s%c\",\n-\t\t\t\tdiff_line_prefix(options),\n-\t\t\t\toptions->line_termination);\n+\t\t\tchar term[2];\n+\t\t\tterm[0] = options->line_termination;\n+\t\t\tterm[1] = '\\0';\n+\n+\t\t\temit_line(options, NULL, NULL, 1, 0, term, !!term[0]);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320670","messageId":"20170524214036.29623-6-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 05/17] diff.c: convert fn_out_consume to use emit_line","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:24Z","receivedAt":"2017-05-24T21:41:08Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line.\n\nThis covers the parts of fn_out_consume. In the next\npatches we'll convert more functions that want to emit\nformatted output, so we'd want to have a formatted emit\nfunction. Add it here.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 28 ++++++++++++++++++++--------\n 1 file changed, 20 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 3569857818..8186289734 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -547,6 +547,21 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n \t\tfputc('\\n', file);\n }\n \n+static void emit_line_fmt(struct diff_options *o,\n+\t\t\t const char *set, const char *reset,\n+\t\t\t int add_line_prefix,\n+\t\t\t const char *fmt, ...)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tva_list ap;\n+\tva_start(ap, fmt);\n+\tstrbuf_vaddf(&sb, fmt, ap);\n+\tva_end(ap);\n+\n+\temit_line(o, set, reset, add_line_prefix, 0, sb.buf, sb.len);\n+\tstrbuf_release(&sb);\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -1270,7 +1285,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \tconst char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n-\tconst char *line_prefix = diff_line_prefix(o);\n \n \to->found_changes = 1;\n \n@@ -1282,14 +1296,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \n \tif (ecbdata->label_path[0]) {\n \t\tconst char *name_a_tab, *name_b_tab;\n-\n \t\tname_a_tab = strchr(ecbdata->label_path[0], ' ') ? \"\\t\" : \"\";\n \t\tname_b_tab = strchr(ecbdata->label_path[1], ' ') ? \"\\t\" : \"\";\n-\n-\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);\n-\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);\n+\t\temit_line_fmt(o, meta, reset, 1, \"--- %s%s\\n\",\n+\t\t\t ecbdata->label_path[0], name_a_tab);\n+\t\temit_line_fmt(o, meta, reset, 1, \"+++ %s%s\\n\",\n+\t\t\t ecbdata->label_path[1], name_b_tab);\n \t\tecbdata->label_path[0] = ecbdata->label_path[1] = NULL;\n \t}\n \n@@ -1330,7 +1342,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\tdiff_words_flush(ecbdata);\n \t\tif (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {\n \t\t\temit_line(o, context, reset, 1, 0, line, len);\n-\t\t\tfputs(\"~\\n\", o->file);\n+\t\t\temit_line(o, NULL, NULL, 0, 0, \"~\\n\", 2);\n \t\t} else {\n \t\t\t/*\n \t\t\t * Skip the prefix character, if any. With\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320671","messageId":"20170524214036.29623-3-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 02/17] diff: move line ending check into emit_hunk_header","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:21Z","receivedAt":"2017-05-24T21:41:10Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The emit_hunk_header() function is responsible for assembling a\nhunk header and calling emit_line() to send the hunk header\nto the output file. Its only caller fn_out_consume() needs\nto prepare for a case where the function emits an incomplete\nline and add the terminating LF.\n\nInstead make sure emit_hunk_header() to always send a\ncompleted line to emit_line().\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 3f5bf8b5a4..c2ed605cd0 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -677,6 +677,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t}\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n+\tstrbuf_complete_line(&msgbuf);\n+\n \temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n@@ -1315,8 +1317,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\tlen = sane_truncate_line(ecbdata, line, len);\n \t\tfind_lno(line, ecbdata);\n \t\temit_hunk_header(ecbdata, line, len);\n-\t\tif (line[len-1] != '\\n')\n-\t\t\tputc('\\n', o->file);\n \t\treturn;\n \t}\n \n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320672","messageId":"20170524214036.29623-8-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 07/17] diff.c: convert emit_rewrite_diff to use emit_line_*","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:26Z","receivedAt":"2017-05-24T21:41:13Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nThis covers emit_rewrite_diff.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 33 ++++++++++++++++++---------------\n 1 file changed, 18 insertions(+), 15 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 4fa976d43c..3dda9f3c8e 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -704,17 +704,17 @@ static void remove_tempfile(void)\n \t}\n }\n \n-static void print_line_count(FILE *file, int count)\n+static void add_line_count(struct strbuf *out, int count)\n {\n \tswitch (count) {\n \tcase 0:\n-\t\tfprintf(file, \"0,0\");\n+\t\tstrbuf_addstr(out, \"0,0\");\n \t\tbreak;\n \tcase 1:\n-\t\tfprintf(file, \"1\");\n+\t\tstrbuf_addstr(out, \"1\");\n \t\tbreak;\n \tdefault:\n-\t\tfprintf(file, \"1,%d\", count);\n+\t\tstrbuf_addf(out, \"1,%d\", count);\n \t\tbreak;\n \t}\n }\n@@ -768,7 +768,7 @@ static void emit_rewrite_diff(const char *name_a,\n \tchar *data_one, *data_two;\n \tsize_t size_one, size_two;\n \tstruct emit_callback ecbdata;\n-\tconst char *line_prefix = diff_line_prefix(o);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\ta_prefix = o->b_prefix;\n@@ -806,20 +806,23 @@ static void emit_rewrite_diff(const char *name_a,\n \tecbdata.lno_in_preimage = 1;\n \tecbdata.lno_in_postimage = 1;\n \n+\temit_line_fmt(o, metainfo, reset, 1, \"--- %s%s\\n\", a_name.buf, name_a_tab);\n+\temit_line_fmt(o, metainfo, reset, 1, \"+++ %s%s\\n\", b_name.buf, name_b_tab);\n+\n \tlc_a = count_lines(data_one, size_one);\n \tlc_b = count_lines(data_two, size_two);\n-\tfprintf(o->file,\n-\t\t\"%s%s--- %s%s%s\\n%s%s+++ %s%s%s\\n%s%s@@ -\",\n-\t\tline_prefix, metainfo, a_name.buf, name_a_tab, reset,\n-\t\tline_prefix, metainfo, b_name.buf, name_b_tab, reset,\n-\t\tline_prefix, fraginfo);\n+\n+\tstrbuf_addstr(&out, \"@@ -\");\n \tif (!o->irreversible_delete)\n-\t\tprint_line_count(o->file, lc_a);\n+\t\tadd_line_count(&out, lc_a);\n \telse\n-\t\tfprintf(o->file, \"?,?\");\n-\tfprintf(o->file, \" +\");\n-\tprint_line_count(o->file, lc_b);\n-\tfprintf(o->file, \" @@%s\\n\", reset);\n+\t\tstrbuf_addstr(&out, \"?,?\");\n+\tstrbuf_addstr(&out, \" +\");\n+\tadd_line_count(&out, lc_b);\n+\tstrbuf_addstr(&out, \" @@\\n\");\n+\temit_line(o, fraginfo, reset, 1, 0, out.buf, out.len);\n+\tstrbuf_release(&out);\n+\n \tif (lc_a && !o->irreversible_delete)\n \t\temit_rewrite_lines(&ecbdata, '-', data_one, size_one);\n \tif (lc_b)\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320673","messageId":"20170524214036.29623-16-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 15/17] diff.c: emit_line includes whitespace highlighting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:34Z","receivedAt":"2017-05-24T21:41:15Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently any whitespace highlighting happens outside the emit_line\nfunction. Teach the highlighting to emit_line, triggered by a new\nparameter.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 107 ++++++++++++++++++++++++++++++++++++++---------------------------\n diff.h | 2 ++\n 2 files changed, 65 insertions(+), 44 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 76cafde4be..514c5facd7 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -516,15 +516,34 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,\n \tecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n }\n \n-static void emit_line(struct diff_options *o, const char *set, const char *reset,\n-\t\t int add_line_prefix, int sign, const char *line, int len)\n+\n+static void emit_line(struct diff_options *o,\n+\t\t const char *set, const char *reset,\n+\t\t int add_line_prefix, int markup_ws,\n+\t\t int sign, const char *line, int len)\n {\n+\tconst char *ws;\n \tint has_trailing_newline, has_trailing_carriage_return;\n \tFILE *file = o->file;\n \n \tif (add_line_prefix)\n \t\tfputs(diff_line_prefix(o), file);\n \n+\tif (markup_ws) {\n+\t\tws = diff_get_color(o->use_color, DIFF_WHITESPACE);\n+\n+\t\tif (set)\n+\t\t\tfputs(set, file);\n+\t\tif (sign)\n+\t\t\tfputc(sign, file);\n+\t\tif (reset)\n+\t\t\tfputs(reset, file);\n+\t\tws = diff_get_color(o->use_color, DIFF_WHITESPACE);\n+\t\tws_check_emit(line, len, o->ws_rule,\n+\t\t\t file, set, reset, ws);\n+\t\treturn;\n+\t}\n+\n \thas_trailing_newline = (len > 0 && line[len-1] == '\\n');\n \tif (has_trailing_newline)\n \t\tlen--;\n@@ -558,14 +577,14 @@ static void emit_line_fmt(struct diff_options *o,\n \tstrbuf_vaddf(&sb, fmt, ap);\n \tva_end(ap);\n \n-\temit_line(o, set, reset, add_line_prefix, 0, sb.buf, sb.len);\n+\temit_line(o, set, reset, add_line_prefix, 0, 0, sb.buf, sb.len);\n \tstrbuf_release(&sb);\n }\n \n void diff_emit_line(struct diff_options *o, const char *set, const char *reset,\n \t\t const char *line, int len)\n {\n-\temit_line(o, set, reset, 1, 0, line, len);\n+\temit_line(o, set, reset, 1, 0, 0, line, len);\n }\n \n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n@@ -596,16 +615,15 @@ static void emit_line_checked(const char *reset,\n \t}\n \n \tif (!ws)\n-\t\temit_line(ecbdata->opt, set, reset, 1, sign, line, len);\n+\t\temit_line(ecbdata->opt, set, reset, 1, 0, sign, line, len);\n \telse if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))\n \t\t/* Blank line at EOF - paint '+' as well */\n-\t\temit_line(ecbdata->opt, ws, reset, 1, sign, line, len);\n+\t\temit_line(ecbdata->opt, ws, reset, 1, 1, sign, line, len);\n \telse {\n \t\t/* Emit just the prefix, then the rest. */\n-\t\temit_line(ecbdata->opt, set, reset, 1, sign, \"\", 0);\n-\t\tws_check_emit(line, len, ecbdata->ws_rule,\n-\t\t\t ecbdata->opt->file, set, reset, ws);\n+\t\temit_line(ecbdata->opt, set, reset, 1, 1, sign, line, len);\n \t}\n+\n }\n \n static void emit_add_line(const char *reset,\n@@ -652,7 +670,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tif (len < 10 ||\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n-\t\temit_line(ecbdata->opt, context, reset, 1, 0, line, len);\n+\t\temit_line(ecbdata->opt, context, reset, 1, 0, 0, line, len);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n@@ -688,7 +706,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n \n-\temit_line(ecbdata->opt, \"\", \"\", 1, 0, msgbuf.buf, msgbuf.len);\n+\temit_line(ecbdata->opt, \"\", \"\", 1, 0, 0, msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n \n@@ -759,7 +777,7 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tstatic const char *nneof = \"\\\\ No newline at end of file\\n\";\n \t\tconst char *context = diff_get_color(ecb->color_diff,\n \t\t\t\t\t\t DIFF_CONTEXT);\n-\t\temit_line(ecb->opt, context, reset, 1, 0,\n+\t\temit_line(ecb->opt, context, reset, 1, 0, 0,\n \t\t\t nneof, strlen(nneof));\n \t\tstrbuf_release(&sb);\n \t}\n@@ -835,7 +853,7 @@ static void emit_rewrite_diff(const char *name_a,\n \tstrbuf_addstr(&out, \" +\");\n \tadd_line_count(&out, lc_b);\n \tstrbuf_addstr(&out, \" @@\\n\");\n-\temit_line(o, fraginfo, reset, 1, 0, out.buf, out.len);\n+\temit_line(o, fraginfo, reset, 1, 0, 0, out.buf, out.len);\n \tstrbuf_release(&out);\n \n \tif (lc_a && !o->irreversible_delete)\n@@ -908,7 +926,7 @@ static int fn_out_diff_words_write_helper(struct diff_options *o,\n \twhile (count) {\n \t\tchar *p = memchr(buf, '\\n', count);\n \t\tif (print)\n-\t\t\temit_line(o, NULL, NULL, 1, 0, \"\", 0);\n+\t\t\temit_line(o, NULL, NULL, 1, 0, 0, \"\", 0);\n \n \t\tif (p != buf) {\n \t\t\tconst char *reset = st_el->color && *st_el->color ?\n@@ -917,14 +935,14 @@ static int fn_out_diff_words_write_helper(struct diff_options *o,\n \t\t\tstrbuf_add(&sb, buf, p ? p - buf : count);\n \t\t\tstrbuf_addstr(&sb, st_el->suffix);\n \t\t\temit_line(o, st_el->color, reset,\n-\t\t\t\t 0, 0, sb.buf, sb.len);\n+\t\t\t\t 0, 0, 0, sb.buf, sb.len);\n \t\t\tstrbuf_reset(&sb);\n \t\t}\n \t\tif (!p)\n \t\t\tgoto out;\n \n \t\tstrbuf_addstr(&sb, newline);\n-\t\temit_line(o, NULL, NULL, 0, 0, sb.buf, sb.len);\n+\t\temit_line(o, NULL, NULL, 0, 0, 0, sb.buf, sb.len);\n \t\tstrbuf_reset(&sb);\n \t\tcount -= p + 1 - buf;\n \t\tbuf = p + 1;\n@@ -1139,7 +1157,7 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \tif (diff_words->current_plus != diff_words->plus.text.ptr +\n \t\t\tdiff_words->plus.text.size) {\n \t\tif (color_words_output_graph_prefix(diff_words))\n-\t\t\temit_line(diff_words->opt, NULL, NULL, 1, 0, \"\", 0);\n+\t\t\temit_line(diff_words->opt, NULL, NULL, 1, 0, 0, \"\", 0);\n \t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->ctx, style->newline,\n \t\t\tdiff_words->plus.text.ptr + diff_words->plus.text.size\n@@ -1298,7 +1316,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \to->found_changes = 1;\n \n \tif (ecbdata->header) {\n-\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\temit_line(o, NULL, NULL, 0, 0, 0,\n \t\t\t ecbdata->header->buf, ecbdata->header->len);\n \t\tstrbuf_release(ecbdata->header);\n \t\tecbdata->header = NULL;\n@@ -1351,8 +1369,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t}\n \t\tdiff_words_flush(ecbdata);\n \t\tif (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {\n-\t\t\temit_line(o, context, reset, 1, 0, line, len);\n-\t\t\temit_line(o, NULL, NULL, 0, 0, \"~\\n\", 2);\n+\t\t\temit_line(o, context, reset, 1, 0, 0, line, len);\n+\t\t\temit_line(o, NULL, NULL, 0, 0, 0, \"~\\n\", 2);\n \t\t} else {\n \t\t\t/*\n \t\t\t * Skip the prefix character, if any. With\n@@ -1363,7 +1381,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t\t line++;\n \t\t\t len--;\n \t\t\t}\n-\t\t\temit_line(o, context, reset, 1, 0, line, len);\n+\t\t\temit_line(o, context, reset, 1, 0, 0, line, len);\n \t\t}\n \t\treturn;\n \t}\n@@ -1386,7 +1404,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t/* incomplete line at the end */\n \t\tecbdata->lno_in_preimage++;\n \t\temit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),\n-\t\t\t reset, 1, 0, line, len);\n+\t\t\t reset, 1, 0, 0, line, len);\n \t\tbreak;\n \t}\n }\n@@ -1575,7 +1593,7 @@ static void print_stat_summary_0(struct diff_options *options, int files,\n \tif (!files) {\n \t\tassert(insertions == 0 && deletions == 0);\n \t\tstrbuf_addstr(&sb, \" 0 files changed\");\n-\t\temit_line(options, NULL, NULL, 1, 0, sb.buf, sb.len);\n+\t\temit_line(options, NULL, NULL, 1, 0, 0, sb.buf, sb.len);\n \t\treturn;\n \t}\n \n@@ -1603,7 +1621,7 @@ static void print_stat_summary_0(struct diff_options *options, int files,\n \t\t\t deletions);\n \t}\n \tstrbuf_addch(&sb, '\\n');\n-\temit_line(options, NULL, NULL, 1, 0, sb.buf, sb.len);\n+\temit_line(options, NULL, NULL, 1, 0, 0, sb.buf, sb.len);\n \tstrbuf_release(&sb);\n }\n \n@@ -1787,7 +1805,7 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t\tstrbuf_addf(&out, \" %*s\", number_width, \"Bin\");\n \t\t\tif (!added && !deleted) {\n \t\t\t\tstrbuf_addch(&out, '\\n');\n-\t\t\t\temit_line(options, NULL, NULL, 1, 0, out.buf, out.len);\n+\t\t\t\temit_line(options, NULL, NULL, 1, 0, 0, out.buf, out.len);\n \t\t\t\tstrbuf_reset(&out);\n \t\t\t\tcontinue;\n \t\t\t}\n@@ -1797,14 +1815,14 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t\tstrbuf_addf(&out, \"%s%\"PRIuMAX\"%s\",\n \t\t\t\tadd_c, added, reset);\n \t\t\tstrbuf_addstr(&out, \" bytes\\n\");\n-\t\t\temit_line(options, NULL, NULL, 1, 0, out.buf, out.len);\n+\t\t\temit_line(options, NULL, NULL, 1, 0, 0, out.buf, out.len);\n \t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \t\telse if (file->is_unmerged) {\n \t\t\tshow_name(&out, prefix, name, len);\n \t\t\tstrbuf_addstr(&out, \" Unmerged\\n\");\n-\t\t\temit_line(options, NULL, NULL, 1, 0, out.buf, out.len);\n+\t\t\temit_line(options, NULL, NULL, 1, 0, 0, out.buf, out.len);\n \t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n@@ -1835,7 +1853,7 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\tshow_graph(&out, '+', add, add_c, reset);\n \t\tshow_graph(&out, '-', del, del_c, reset);\n \t\tstrbuf_addch(&out, '\\n');\n-\t\temit_line(options, NULL, NULL, 1, 0, out.buf, out.len);\n+\t\temit_line(options, NULL, NULL, 1, 0, 0, out.buf, out.len);\n \t\tstrbuf_reset(&out);\n \t}\n \n@@ -1857,7 +1875,7 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\tif (i < count)\n \t\t\tcontinue;\n \t\tif (!extra_shown)\n-\t\t\temit_line(options, NULL, NULL, 1, 0,\n+\t\t\temit_line(options, NULL, NULL, 1, 0, 0,\n \t\t\t\t \" ...\\n\", strlen(\" ...\\n\"));\n \t\textra_shown = 1;\n \t}\n@@ -2211,7 +2229,7 @@ static void checkdiff_consume(void *priv, char *line, unsigned long len)\n \t\tfprintf(data->o->file, \"%s%s:%d: %s.\\n\",\n \t\t\tline_prefix, data->filename, data->lineno, err);\n \t\tfree(err);\n-\t\temit_line(data->o, set, reset, 1, 0, line, 1);\n+\t\temit_line(data->o, set, reset, 1, 0, 0, line, 1);\n \t\tws_check_emit(line + 1, len - 1, data->ws_rule,\n \t\t\t data->o->file, set, reset, ws);\n \t} else if (line[0] == ' ') {\n@@ -2307,9 +2325,9 @@ static void emit_binary_diff_body(struct diff_options *o,\n \t\tline[len++] = '\\n';\n \t\tline[len] = '\\0';\n \n-\t\temit_line(o, NULL, NULL, 1, 0, line, len);\n+\t\temit_line(o, NULL, NULL, 1, 0, 0, line, len);\n \t}\n-\temit_line(o, NULL, NULL, 1, 0, \"\\n\", 1);\n+\temit_line(o, NULL, NULL, 1, 0, 0, \"\\n\", 1);\n \tfree(data);\n }\n \n@@ -2318,7 +2336,7 @@ static void emit_binary_diff(struct diff_options *o,\n {\n \tconst char *s = \"GIT binary patch\\n\";\n \tconst int len = strlen(s);\n-\temit_line(o, NULL, NULL, 1, 0, s, len);\n+\temit_line(o, NULL, NULL, 1, 0, 0, s, len);\n \temit_binary_diff_body(o, one, two);\n \temit_binary_diff_body(o, two, one);\n }\n@@ -2461,7 +2479,7 @@ static void builtin_diff(const char *name_a,\n \t\tif (complete_rewrite &&\n \t\t (textconv_one || !diff_filespec_is_binary(one)) &&\n \t\t (textconv_two || !diff_filespec_is_binary(two))) {\n-\t\t\temit_line(o, NULL, NULL, 0, 0, header.buf, header.len);\n+\t\t\temit_line(o, NULL, NULL, 0, 0, 0, header.buf, header.len);\n \t\t\tstrbuf_reset(&header);\n \t\t\temit_rewrite_diff(name_a, name_b, one, two,\n \t\t\t\t\t\ttextconv_one, textconv_two, o);\n@@ -2471,7 +2489,7 @@ static void builtin_diff(const char *name_a,\n \t}\n \n \tif (o->irreversible_delete && lbl[1][0] == '/') {\n-\t\temit_line(o, NULL, NULL, 0, 0, header.buf, header.len);\n+\t\temit_line(o, NULL, NULL, 0, 0, 0, header.buf, header.len);\n \t\tstrbuf_reset(&header);\n \t\tgoto free_ab_and_return;\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n@@ -2482,11 +2500,11 @@ static void builtin_diff(const char *name_a,\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n \t\t\tif (!oidcmp(&one->oid, &two->oid)) {\n \t\t\t\tif (must_show_header)\n-\t\t\t\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\t\t\t\temit_line(o, NULL, NULL, 0, 0, 0,\n \t\t\t\t\t\t header.buf, header.len);\n \t\t\t\tgoto free_ab_and_return;\n \t\t\t}\n-\t\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\t\temit_line(o, NULL, NULL, 0, 0, 0,\n \t\t\t\t header.buf, header.len);\n \t\t\temit_line_fmt(o, NULL, NULL, 1,\n \t\t\t\t \"Binary files %s and %s differ\\n\",\n@@ -2499,11 +2517,11 @@ static void builtin_diff(const char *name_a,\n \t\tif (mf1.size == mf2.size &&\n \t\t !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {\n \t\t\tif (must_show_header)\n-\t\t\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\t\t\temit_line(o, NULL, NULL, 0, 0, 0,\n \t\t\t\t\t header.buf, header.len);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n-\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\temit_line(o, NULL, NULL, 0, 0, 0,\n \t\t\t header.buf, header.len);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n@@ -2523,7 +2541,7 @@ static void builtin_diff(const char *name_a,\n \t\tconst struct userdiff_funcname *pe;\n \n \t\tif (must_show_header) {\n-\t\t\temit_line(o, NULL, NULL, 0, 0, header.buf, header.len);\n+\t\t\temit_line(o, NULL, NULL, 0, 0, 0, header.buf, header.len);\n \t\t\tstrbuf_reset(&header);\n \t\t}\n \n@@ -2540,6 +2558,7 @@ static void builtin_diff(const char *name_a,\n \t\tecbdata.label_path = lbl;\n \t\tecbdata.color_diff = want_color(o->use_color);\n \t\tecbdata.ws_rule = whitespace_rule(name_b);\n+\t\to->ws_rule = ecbdata.ws_rule;\n \t\tif (ecbdata.ws_rule & WS_BLANK_AT_EOF)\n \t\t\tcheck_blank_at_eof(&mf1, &mf2, &ecbdata);\n \t\tecbdata.opt = o;\n@@ -4514,7 +4533,7 @@ static void show_file_mode_name(struct diff_options *opt, const char *newdelete,\n \n \tquote_c_style(fs->path, &sb, NULL, 0);\n \tstrbuf_addch(&sb, '\\n');\n-\temit_line(opt, NULL, NULL, 1, 0, sb.buf, sb.len);\n+\temit_line(opt, NULL, NULL, 1, 0, 0, sb.buf, sb.len);\n \tstrbuf_release(&sb);\n }\n \n@@ -4566,7 +4585,7 @@ static void diff_summary(struct diff_options *opt, struct diff_filepair *p)\n \t\t\tstrbuf_addstr(&sb, \" rewrite \");\n \t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n \t\t\tstrbuf_addf(&sb, \" (%d%%)\\n\", similarity_index(p));\n-\t\t\temit_line(opt, NULL, NULL, 1, 0, sb.buf, sb.len);\n+\t\t\temit_line(opt, NULL, NULL, 1, 0, 0, sb.buf, sb.len);\n \t\t}\n \t\tshow_mode_change(opt, p, !p->score);\n \t\tbreak;\n@@ -4873,10 +4892,10 @@ void diff_flush(struct diff_options *options)\n \t\t\tterm[0] = options->line_termination;\n \t\t\tterm[1] = '\\0';\n \n-\t\t\temit_line(options, NULL, NULL, 1, 0, term, !!term[0]);\n+\t\t\temit_line(options, NULL, NULL, 1, 0, 0, term, !!term[0]);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n-\t\t\t\temit_line(options, NULL, NULL, 0, 0,\n+\t\t\t\temit_line(options, NULL, NULL, 0, 0, 0,\n \t\t\t\t\t options->stat_sep,\n \t\t\t\t\t strlen(options->stat_sep));\n \t\t\t}\ndiff --git a/diff.h b/diff.h\nindex 56d8dd036e..85948ed65a 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -186,6 +186,8 @@ struct diff_options {\n \tvoid *output_prefix_data;\n \n \tint diff_path_counter;\n+\n+\tunsigned ws_rule;\n };\n \n void diff_emit_line(struct diff_options *o, const char *set, const char *reset,\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320674","messageId":"20170524214036.29623-12-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 11/17] diff.c: convert show_stats to use emit_line_*","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:30Z","receivedAt":"2017-05-24T21:41:19Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nWe call print_stat_summary from builtin/apply, so we still\nneed the version with a file pointer, so introduce\nprint_stat_summary_0 that uses emit_line_* machinery and\nkeep print_stat_summary with the same arguments around.\n\nThe responsibility to print the line prefix moves from the callers\nof print_stat_summary_0 into the function itself.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 89 ++++++++++++++++++++++++++++++++++++++----------------------------\n diff.h | 4 +--\n 2 files changed, 53 insertions(+), 40 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex b5a5261a4e..a76abf5f69 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -1540,20 +1540,19 @@ static int scale_linear(int it, int width, int max_change)\n \treturn 1 + (it * (width - 1) / max_change);\n }\n \n-static void show_name(FILE *file,\n+static void show_name(struct strbuf *out,\n \t\t const char *prefix, const char *name, int len)\n {\n-\tfprintf(file, \" %s%-*s |\", prefix, len, name);\n+\tstrbuf_addf(out, \" %s%-*s |\", prefix, len, name);\n }\n \n-static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)\n+static void show_graph(struct strbuf *out, char ch, int cnt, const char *set, const char *reset)\n {\n \tif (cnt <= 0)\n \t\treturn;\n-\tfprintf(file, \"%s\", set);\n-\twhile (cnt--)\n-\t\tputc(ch, file);\n-\tfprintf(file, \"%s\", reset);\n+\tstrbuf_addstr(out, set);\n+\tstrbuf_addchars(out, ch, cnt);\n+\tstrbuf_addstr(out, reset);\n }\n \n static void fill_print_name(struct diffstat_file *file)\n@@ -1577,14 +1576,16 @@ static void fill_print_name(struct diffstat_file *file)\n \tfile->print_name = pname;\n }\n \n-int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n+static void print_stat_summary_0(struct diff_options *options, int files,\n+\t\t\t\t int insertions, int deletions)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint ret;\n \n \tif (!files) {\n \t\tassert(insertions == 0 && deletions == 0);\n-\t\treturn fprintf(fp, \"%s\\n\", \" 0 files changed\");\n+\t\tstrbuf_addstr(&sb, \" 0 files changed\");\n+\t\temit_line(options, NULL, NULL, 1, 0, sb.buf, sb.len);\n+\t\treturn;\n \t}\n \n \tstrbuf_addf(&sb,\n@@ -1611,9 +1612,17 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n \t\t\t deletions);\n \t}\n \tstrbuf_addch(&sb, '\\n');\n-\tret = fputs(sb.buf, fp);\n+\temit_line(options, NULL, NULL, 1, 0, sb.buf, sb.len);\n \tstrbuf_release(&sb);\n-\treturn ret;\n+}\n+\n+void print_stat_summary(FILE *fp, int files,\n+\t\t\tint insertions, int deletions)\n+{\n+\tstruct diff_options o;\n+\tmemset(&o, 0, sizeof(o));\n+\to.file = fp;\n+\tprint_stat_summary_0(&o, files, insertions, deletions);\n }\n \n static void show_stats(struct diffstat_t *data, struct diff_options *options)\n@@ -1623,13 +1632,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \tint total_files = data->nr, count;\n \tint width, name_width, graph_width, number_width = 0, bin_width = 0;\n \tconst char *reset, *add_c, *del_c;\n-\tconst char *line_prefix = \"\";\n \tint extra_shown = 0;\n+\tconst char *line_prefix = diff_line_prefix(options);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (data->nr == 0)\n \t\treturn;\n \n-\tline_prefix = diff_line_prefix(options);\n \tcount = options->stat_count ? options->stat_count : data->nr;\n \n \treset = diff_get_color_opt(options, DIFF_RESET);\n@@ -1783,26 +1792,29 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t}\n \n \t\tif (file->is_binary) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" %*s\", number_width, \"Bin\");\n+\t\t\tshow_name(&out, prefix, name, len);\n+\t\t\tstrbuf_addf(&out, \" %*s\", number_width, \"Bin\");\n \t\t\tif (!added && !deleted) {\n-\t\t\t\tputc('\\n', options->file);\n+\t\t\t\tstrbuf_addch(&out, '\\n');\n+\t\t\t\temit_line(options, NULL, NULL, 1, 0, out.buf, out.len);\n+\t\t\t\tstrbuf_reset(&out);\n \t\t\t\tcontinue;\n \t\t\t}\n-\t\t\tfprintf(options->file, \" %s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addf(&out, \" %s%\"PRIuMAX\"%s\",\n \t\t\t\tdel_c, deleted, reset);\n-\t\t\tfprintf(options->file, \" -> \");\n-\t\t\tfprintf(options->file, \"%s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addstr(&out, \" -> \");\n+\t\t\tstrbuf_addf(&out, \"%s%\"PRIuMAX\"%s\",\n \t\t\t\tadd_c, added, reset);\n-\t\t\tfprintf(options->file, \" bytes\");\n-\t\t\tfprintf(options->file, \"\\n\");\n+\t\t\tstrbuf_addstr(&out, \" bytes\\n\");\n+\t\t\temit_line(options, NULL, NULL, 1, 0, out.buf, out.len);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \t\telse if (file->is_unmerged) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" Unmerged\\n\");\n+\t\t\tshow_name(&out, prefix, name, len);\n+\t\t\tstrbuf_addstr(&out, \" Unmerged\\n\");\n+\t\t\temit_line(options, NULL, NULL, 1, 0, out.buf, out.len);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \n@@ -1825,14 +1837,15 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t\t\tadd = total - del;\n \t\t\t}\n \t\t}\n-\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\tshow_name(options->file, prefix, name, len);\n-\t\tfprintf(options->file, \" %*\"PRIuMAX\"%s\",\n+\t\tshow_name(&out, prefix, name, len);\n+\t\tstrbuf_addf(&out, \" %*\"PRIuMAX\"%s\",\n \t\t\tnumber_width, added + deleted,\n \t\t\tadded + deleted ? \" \" : \"\");\n-\t\tshow_graph(options->file, '+', add, add_c, reset);\n-\t\tshow_graph(options->file, '-', del, del_c, reset);\n-\t\tfprintf(options->file, \"\\n\");\n+\t\tshow_graph(&out, '+', add, add_c, reset);\n+\t\tshow_graph(&out, '-', del, del_c, reset);\n+\t\tstrbuf_addch(&out, '\\n');\n+\t\temit_line(options, NULL, NULL, 1, 0, out.buf, out.len);\n+\t\tstrbuf_reset(&out);\n \t}\n \n \tfor (i = 0; i < data->nr; i++) {\n@@ -1853,11 +1866,12 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\tif (i < count)\n \t\t\tcontinue;\n \t\tif (!extra_shown)\n-\t\t\tfprintf(options->file, \"%s ...\\n\", line_prefix);\n+\t\t\temit_line(options, NULL, NULL, 1, 0,\n+\t\t\t\t \" ...\\n\", strlen(\" ...\\n\"));\n \t\textra_shown = 1;\n \t}\n-\tfprintf(options->file, \"%s\", line_prefix);\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\n+\tprint_stat_summary_0(options, total_files, adds, dels);\n }\n \n static void show_shortstats(struct diffstat_t *data, struct diff_options *options)\n@@ -1869,7 +1883,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \n \tfor (i = 0; i < data->nr; i++) {\n \t\tint added = data->files[i]->added;\n-\t\tint deleted= data->files[i]->deleted;\n+\t\tint deleted = data->files[i]->deleted;\n \n \t\tif (data->files[i]->is_unmerged ||\n \t\t (!data->files[i]->is_interesting && (added + deleted == 0))) {\n@@ -1879,8 +1893,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \t\t\tdels += deleted;\n \t\t}\n \t}\n-\tfprintf(options->file, \"%s\", diff_line_prefix(options));\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\tprint_stat_summary_0(options, total_files, adds, dels);\n }\n \n static void show_numstat(struct diffstat_t *data, struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex 9ad546361a..56d8dd036e 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -392,8 +392,8 @@ extern int parse_rename_score(const char **cp_p);\n \n extern long parse_algorithm_value(const char *value);\n \n-extern int print_stat_summary(FILE *fp, int files,\n-\t\t\t int insertions, int deletions);\n+extern void print_stat_summary(FILE *fp, int files,\n+\t\t\t int insertions, int deletions);\n extern void setup_diff_pager(struct diff_options *);\n \n #endif /* DIFF_H */\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320675","messageId":"20170524214036.29623-18-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 17/17] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:36Z","receivedAt":"2017-05-24T21:41:24Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a patch consists mostly of moving blocks of code around, it can\nbe quite tedious to ensure that the blocks are moved verbatim, and not\nundesirably modified in the move. To that end, color blocks that are\nmoved within the same patch differently. For example (OM, del, add,\nand NM are different colors):\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OM] - if (!is_authorized_user())\n [OM] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OM] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NM] + sensitive_stuff(spanning,\n [NM] + multiple,\n [NM] + lines);\n [NM] +}\n\nAdjacent blocks are colored differently. For example, in this\npotentially malicious patch, the swapping of blocks can be spotted:\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OMA] - if (!is_authorized_user())\n [OMA] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OMA] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NMA] + sensitive_stuff(spanning,\n [NMA] + multiple,\n [NMA] + lines);\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NMA] +}\n\nIf the moved code is larger, it is easier to hide some permutation in the\ncode, which is why the alternative coloring is really needed.\n\nAs the reviewers attention should be brought to the places, where the\ndifference is introduced to the moved code, we cannot just have one new\ncolor for all of moved code.\n\nFirst I implemented an alternative design, which would show a moved hunk\nin one color, and its boundaries in another color. This idea was error\nprone as it inspected each line and its neighboring lines to determine\nif the line was (a) moved and (b) if was deep inside a hunk by having\nmatching neighboring lines. This is unreliable as the we can construct\nhunks which have equal neighbors that just exceed the number of lines\ninspected. (Think of 'AXYZBXYZCXYZD..' with each letter as a line, that\nis permutated to AXYZCXYZBXYZD..').\n\nInstead this provides a dynamic programming greedy algorithm that finds\nthe largest moved hunk and then switches color to the alternative color\nfor the next hunk. By doing this any permutation is recognized and\ndisplayed. That implies that there is no dedicated boundary or\ninside-hunk color, but instead we'll have just two colors alternating\nfor hunks.\n\nIt would be a bit more UX friendly if the two corresponding hunks\n(of added and deleted lines) for one move would get the same color id.\n(Both get \"regular moved\" or \"alternative moved\"). This problem is\ndeferred to a later patch for now.\n\nA note on the options '--submodule=diff' and '--color-words/--word-diff':\nIn the conversion to use emit_line in the prior patches both submodules\nas well as word diff output carefully chose to call emit_line with sign=0.\nAll output with sign=0 is ignored for move detection purposes in this\npatch, such that no weird looking output will be generated for these\ncases. This leads to another thought: We could pass on '--color-moved' to\nsubmodules such that they color up moved lines for themselves. If we'd do\nso only line moves within a repository boundary are marked up.\n\nHelped-by: Jonathan Tan \nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n\n# Conflicts:\n#\tdiff.c\nSigned-off-by: Junio C Hamano \nSigned-off-by: Stefan Beller \n---\n Documentation/config.txt | 14 ++-\n diff.c | 275 +++++++++++++++++++++++++++++++++++++++++++--\n diff.h | 9 +-\n t/t4015-diff-whitespace.sh | 267 +++++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 552 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 475e874d51..902d017c3b 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1051,14 +1051,24 @@ This does not affect linkgit:git-format-patch[1] or the\n 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n command line with the `--color[=]` option.\n \n+color.moved::\n+\tA boolean value, whether a diff should color moved lines\n+\tdifferently. The moved lines are searched for in the diff only.\n+\tDuplicated lines from somewhere in the project that are not\n+\tpart of the diff are not colored as moved.\n+\tDefaults to false.\n+\n color.diff.::\n \tUse customized color for diff colorization. `` specifies\n \twhich part of the patch to use the specified color, and is one\n \tof `context` (context text - `plain` is a historical synonym),\n \t`meta` (metainformation), `frag`\n \t(hunk header), 'func' (function in hunk header), `old` (removed lines),\n-\t`new` (added lines), `commit` (commit headers), or `whitespace`\n-\t(highlighting whitespace errors).\n+\t`new` (added lines), `commit` (commit headers), `whitespace`\n+\t(highlighting whitespace errors), `oldMoved` (removed lines that\n+\treappear), `newMoved` (added lines that were removed elsewhere),\n+\t`oldMovedAlternative` and `newMovedAlternative` (as a fallback to\n+\tcover adjacent blocks of moved code)\n \n color.decorate.::\n \tUse customized color for 'git log --decorate' output. `` is one\ndiff --git a/diff.c b/diff.c\nindex 8e06206881..1292d3c4ad 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -31,6 +31,7 @@ static int diff_indent_heuristic; /* experimental */\n static int diff_rename_limit_default = 400;\n static int diff_suppress_blank_empty;\n static int diff_use_color_default = -1;\n+static int diff_color_moved_default;\n static int diff_context_default = 3;\n static int diff_interhunk_context_default;\n static const char *diff_word_regex_cfg;\n@@ -55,6 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n+\tGIT_COLOR_BOLD_RED,\t/* OLD_MOVED_A */\n+\tGIT_COLOR_BG_RED,\t/* OLD_MOVED_B */\n+\tGIT_COLOR_BOLD_GREEN,\t/* NEW_MOVED_A */\n+\tGIT_COLOR_BG_GREEN,\t/* NEW_MOVED_B */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -80,6 +85,14 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_WHITESPACE;\n \tif (!strcasecmp(var, \"func\"))\n \t\treturn DIFF_FUNCINFO;\n+\tif (!strcasecmp(var, \"oldmoved\"))\n+\t\treturn DIFF_FILE_OLD_MOVED;\n+\tif (!strcasecmp(var, \"oldmovedalternative\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoved\"))\n+\t\treturn DIFF_FILE_NEW_MOVED;\n+\tif (!strcasecmp(var, \"newmovedalternative\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT;\n \treturn -1;\n }\n \n@@ -234,6 +247,10 @@ int git_diff_ui_config(const char *var, const char *value, void *cb)\n \t\tdiff_use_color_default = git_config_colorbool(var, value);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"color.moved\")) {\n+\t\tdiff_color_moved_default = git_config_bool(var, value);\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"diff.context\")) {\n \t\tdiff_context_default = git_config_int(var, value);\n \t\tif (diff_context_default < 0)\n@@ -354,6 +371,88 @@ int git_diff_basic_config(const char *var, const char *value, void *cb)\n \treturn git_default_config(var, value, cb);\n }\n \n+struct moved_entry {\n+\tstruct hashmap_entry ent;\n+\tconst struct diff_line *line;\n+\tstruct moved_entry *next_line;\n+};\n+\n+static void get_ws_cleaned_string(const struct diff_line *l,\n+\t\t\t\t struct strbuf *out)\n+{\n+\tint i;\n+\tfor (i = 0; i < l->len; i++) {\n+\t\tif (isspace(l->line[i]))\n+\t\t\tcontinue;\n+\t\tstrbuf_addch(out, l->line[i]);\n+\t}\n+}\n+\n+static int diff_line_cmp_no_ws(const struct diff_line *a,\n+\t\t\t\t\t const struct diff_line *b,\n+\t\t\t\t\t const void *keydata)\n+{\n+\tint ret;\n+\tstruct strbuf sba = STRBUF_INIT;\n+\tstruct strbuf sbb = STRBUF_INIT;\n+\n+\tget_ws_cleaned_string(a, &sba);\n+\tget_ws_cleaned_string(b, &sbb);\n+\tret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);\n+\n+\tstrbuf_release(&sba);\n+\tstrbuf_release(&sbb);\n+\treturn ret;\n+}\n+\n+static int diff_line_cmp(const struct diff_line *a,\n+\t\t\t\t const struct diff_line *b,\n+\t\t\t\t const void *keydata)\n+{\n+\treturn a->len != b->len || strncmp(a->line, b->line, a->len);\n+}\n+\n+static int moved_entry_cmp(const struct moved_entry *a,\n+\t\t\t const struct moved_entry *b,\n+\t\t\t const void *keydata)\n+{\n+\treturn diff_line_cmp(a->line, b->line, keydata);\n+}\n+\n+static int moved_entry_cmp_no_ws(const struct moved_entry *a,\n+\t\t\t\t const struct moved_entry *b,\n+\t\t\t\t const void *keydata)\n+{\n+\treturn diff_line_cmp_no_ws(a->line, b->line, keydata);\n+}\n+\n+static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)\n+{\n+\tstatic struct strbuf sb = STRBUF_INIT;\n+\n+\tif (ignore_ws) {\n+\t\tstrbuf_reset(&sb);\n+\t\tget_ws_cleaned_string(line, &sb);\n+\t\treturn memhash(sb.buf, sb.len);\n+\t} else {\n+\t\treturn memhash(line->line, line->len);\n+\t}\n+}\n+\n+static struct moved_entry *prepare_entry(struct diff_options *o,\n+\t\t\t\t\t int line_no)\n+{\n+\tstruct moved_entry *ret = xmalloc(sizeof(*ret));\n+\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\tstruct diff_line *l = &o->line_buffer[line_no];\n+\n+\tret->ent.hash = get_line_hash(l, ignore_ws);\n+\tret->line = l;\n+\tret->next_line = NULL;\n+\n+\treturn ret;\n+}\n+\n static char *quote_two(const char *one, const char *two)\n {\n \tint need_one = quote_c_style(one, NULL, NULL, 1);\n@@ -516,6 +615,141 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,\n \tecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n }\n \n+static void add_lines_to_move_detection(struct diff_options *o,\n+\t\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry *prev_line = NULL;\n+\n+\tint n;\n+\tfor (n = 0; n < o->line_buffer_nr; n++) {\n+\t\tint sign = 0;\n+\t\tstruct hashmap *hm;\n+\t\tstruct moved_entry *key;\n+\n+\t\tswitch (o->line_buffer[n].sign) {\n+\t\tcase '+':\n+\t\t\tsign = '+';\n+\t\t\thm = add_lines;\n+\t\t\tbreak;\n+\t\tcase '-':\n+\t\t\tsign = '-';\n+\t\t\thm = del_lines;\n+\t\t\tbreak;\n+\t\tcase ' ':\n+\t\tdefault:\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tkey = prepare_entry(o, n);\n+\t\tif (prev_line &&\n+\t\t prev_line->line->sign == sign)\n+\t\t\tprev_line->next_line = key;\n+\n+\t\thashmap_add(hm, key);\n+\t\tprev_line = key;\n+\t}\n+}\n+\n+static void mark_color_as_moved(struct diff_options *o,\n+\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry **pmb = NULL; /* potentially moved blocks */\n+\tint pmb_nr = 0, pmb_alloc = 0;\n+\tint use_alt_color = 0;\n+\tint n;\n+\n+\tfor (n = 0; n < o->line_buffer_nr; n++) {\n+\t\tstruct hashmap *hm = NULL;\n+\t\tstruct moved_entry *key;\n+\t\tstruct moved_entry *match = NULL;\n+\t\tstruct diff_line *l = &o->line_buffer[n];\n+\t\tint i, lp, rp;\n+\n+\t\tswitch (l->sign) {\n+\t\tcase '+':\n+\t\t\thm = del_lines;\n+\t\t\tbreak;\n+\t\tcase '-':\n+\t\t\thm = add_lines;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tuse_alt_color = 0;\n+\t\t\tpmb_nr = 0; /* no running sets */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Check for any match to color it as a move. */\n+\t\tkey = prepare_entry(o, n);\n+\t\tmatch = hashmap_get(hm, key, o);\n+\t\tfree(key);\n+\t\tif (!match)\n+\t\t\tcontinue;\n+\n+\t\t/* Check any potential block runs, advance each or nullify */\n+\t\tfor (i = 0; i < pmb_nr; i++) {\n+\t\t\tstruct moved_entry *p = pmb[i];\n+\t\t\tstruct moved_entry *pnext = (p && p->next_line) ?\n+\t\t\t\t\tp->next_line : NULL;\n+\t\t\tif (pnext &&\n+\t\t\t !diff_line_cmp(pnext->line, l, o)) {\n+\t\t\t\tpmb[i] = p->next_line;\n+\t\t\t} else {\n+\t\t\t\tpmb[i] = NULL;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Shrink the set to the remaining runs */\n+\t\tfor (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n+\t\t\twhile (lp < pmb_nr && pmb[lp])\n+\t\t\t\tlp++;\n+\t\t\t/* lp points at the first NULL now */\n+\n+\t\t\twhile (rp > -1 && !pmb[rp])\n+\t\t\t\trp--;\n+\t\t\t/* rp points at the last non-NULL */\n+\n+\t\t\tif (lp < pmb_nr && rp > -1 && lp < rp) {\n+\t\t\t\tpmb[lp] = pmb[rp];\n+\t\t\t\tpmb[rp] = NULL;\n+\t\t\t\trp--;\n+\t\t\t\tlp++;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (rp > -1) {\n+\t\t\t/* Remember the number of running sets */\n+\t\t\tpmb_nr = rp + 1;\n+\t\t} else {\n+\t\t\t/* Toggle color */\n+\t\t\tuse_alt_color = (use_alt_color + 1) % 2;\n+\n+\t\t\t/* Build up a new set */\n+\t\t\tpmb_nr = 0;\n+\t\t\tfor (; match; match = hashmap_get_next(hm, match)) {\n+\t\t\t\tALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n+\t\t\t\tpmb[pmb_nr++] = match;\n+\t\t\t}\n+\t\t}\n+\n+\t\tswitch (l->sign) {\n+\t\tcase '+':\n+\t\t\tl->set = diff_get_color_opt(o,\n+\t\t\t\tDIFF_FILE_NEW_MOVED + use_alt_color);\n+\t\t\tbreak;\n+\t\tcase '-':\n+\t\t\tl->set = diff_get_color_opt(o,\n+\t\t\t\tDIFF_FILE_OLD_MOVED + use_alt_color);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tdie(\"BUG: we should have continued earlier?\");\n+\t\t}\n+\t}\n+\tfree(pmb);\n+}\n+\n static void emit_diff_line(struct diff_options *o,\n \t\t\t struct diff_line *e)\n {\n@@ -3518,6 +3752,8 @@ void diff_setup(struct diff_options *options)\n \toptions->line_buffer = NULL;\n \toptions->line_buffer_nr = 0;\n \toptions->line_buffer_alloc = 0;\n+\n+\toptions->color_moved = diff_color_moved_default;\n }\n \n void diff_setup_done(struct diff_options *options)\n@@ -3627,6 +3863,9 @@ void diff_setup_done(struct diff_options *options)\n \n \tif (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)\n \t\tdie(_(\"--follow requires exactly one pathspec\"));\n+\n+\tif (!options->use_color || external_diff())\n+\t\toptions->color_moved = 0;\n }\n \n static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)\n@@ -4051,6 +4290,10 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n+\telse if (!strcmp(arg, \"--color-moved\"))\n+\t\toptions->color_moved = 1;\n+\telse if (!strcmp(arg, \"--no-color-moved\"))\n+\t\toptions->color_moved = 0;\n \telse if (!strcmp(arg, \"--color-words\")) {\n \t\toptions->use_color = 1;\n \t\toptions->word_diff = DIFF_WORDS_COLOR;\n@@ -4856,16 +5099,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n-\t/*\n-\t * For testing purposes we want to make sure the diff machinery\n-\t * works completely with the buffer. If there is anything emitted\n-\t * outside the emit_diff_line, then the order is screwed\n-\t * up and the tests will fail.\n-\t *\n-\t * TODO (later in this series):\n-\t * We'll unset this flag in a later patch.\n-\t */\n-\to->use_buffer = 1;\n+\n+\tif (o->color_moved)\n+\t\to->use_buffer = 1;\n \n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n@@ -4874,6 +5110,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \t}\n \n \tif (o->use_buffer) {\n+\t\tif (o->color_moved) {\n+\t\t\tstruct hashmap add_lines, del_lines;\n+\t\t\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\n+\t\t\thashmap_init(&del_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\t\t\thashmap_init(&add_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\n+\t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n+\t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\n+\t\t\thashmap_free(&add_lines, 0);\n+\t\t\thashmap_free(&del_lines, 0);\n+\t\t}\n+\n \t\tfor (i = 0; i < o->line_buffer_nr; i++)\n \t\t\temit_diff_line(o, &o->line_buffer[i]);\n \n@@ -4962,6 +5216,7 @@ void diff_flush(struct diff_options *options)\n \t\tif (!options->file)\n \t\t\tdie_errno(\"Could not open /dev/null\");\n \t\toptions->close_file = 1;\n+\t\toptions->color_moved = 0;\n \t\tfor (i = 0; i < q->nr; i++) {\n \t\t\tstruct diff_filepair *p = q->queue[i];\n \t\t\tif (check_pair_status(p))\ndiff --git a/diff.h b/diff.h\nindex fad1258556..445259ebf7 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -7,6 +7,7 @@\n #include \"tree-walk.h\"\n #include \"pathspec.h\"\n #include \"object.h\"\n+#include \"hashmap.h\"\n \n struct rev_info;\n struct diff_options;\n@@ -228,6 +229,8 @@ struct diff_options {\n \n \tstruct diff_line *line_buffer;\n \tint line_buffer_nr, line_buffer_alloc;\n+\n+\tint color_moved;\n };\n \n /* Emit [line_prefix] [set] line [reset] */\n@@ -243,7 +246,11 @@ enum color_diff {\n \tDIFF_FILE_NEW = 5,\n \tDIFF_COMMIT = 6,\n \tDIFF_WHITESPACE = 7,\n-\tDIFF_FUNCINFO = 8\n+\tDIFF_FUNCINFO = 8,\n+\tDIFF_FILE_OLD_MOVED = 9,\n+\tDIFF_FILE_OLD_MOVED_ALT = 10,\n+\tDIFF_FILE_NEW_MOVED = 11,\n+\tDIFF_FILE_NEW_MOVED_ALT = 12\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 289806d0c7..0e92bf94bf 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -972,4 +972,271 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '\n \n '\n \n+test_expect_success 'detect moved code, complete file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >test.c &&\n+\t#include\n+\tmain()\n+\t{\n+\tprintf(\"Hello World\");\n+\t}\n+\tEOF\n+\tgit add test.c &&\n+\tgit commit -m \"add main function\" &&\n+\tgit mv test.c main.c &&\n+\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tdiff --git a/main.c b/main.c\n+\tnew file mode 100644\n+\tindex 0000000..a986c57\n+\t--- /dev/null\n+\t+++ b/main.c\n+\t@@ -0,0 +1,5 @@\n+\t+#include\n+\t+main()\n+\t+{\n+\t+printf(\"Hello World\");\n+\t+}\n+\tdiff --git a/test.c b/test.c\n+\tdeleted file mode 100644\n+\tindex a986c57..0000000\n+\t--- a/test.c\n+\t+++ /dev/null\n+\t@@ -1,5 +0,0 @@\n+\t-#include\n+\t-main()\n+\t-{\n+\t-printf(\"Hello World\");\n+\t-}\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect moved code, inside file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit add main.c test.c &&\n+\tgit commit -m \"add main and test file\" &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..e34eb69 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+foo(u);\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect permutations inside moved code' '\n+\t# reusing the move example from last test:\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tfoo(u);\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'move detection does not mess up colored words' '\n+\tcat <<-\\EOF >text.txt &&\n+\tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n+\tEOF\n+\tgit add text.txt &&\n+\tgit commit -a -m \"clean state\" &&\n+\tcat <<-\\EOF >text.txt &&\n+\tsimply Lorem Ipsum dummy is text of the typesetting and printing industry.\n+\tEOF\n+\tgit diff --color-moved --word-diff >actual &&\n+\tgit diff --word-diff >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'move detection with submodules' '\n+\ttest_create_repo bananas &&\n+\techo ripe >bananas/recipe &&\n+\tgit -C bananas add recipe &&\n+\ttest_commit fruit &&\n+\ttest_commit -C bananas recipe &&\n+\tgit submodule add ./bananas &&\n+\tgit add bananas &&\n+\tgit commit -a -m \"bananas are like a heavy library?\" &&\n+\techo foul >bananas/recipe &&\n+\techo ripe >fruit.t &&\n+\n+\tgit diff --submodule=diff --color-moved >actual &&\n+\n+\t# no move detection as the moved line is across repository boundaries.\n+\ttest_decode_color decoded_actual &&\n+\t! grep BGREEN decoded_actual &&\n+\t! grep BRED decoded_actual &&\n+\n+\t# nor did we mess with it another way\n+\tgit diff --submodule=diff | test_decode_color >expect &&\n+\ttest_cmp expect decoded_actual\n+'\n+\n test_done\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320676","messageId":"20170524214036.29623-11-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 10/17] diff.c: convert emit_binary_diff_body to use emit_line_*","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:29Z","receivedAt":"2017-05-24T21:41:27Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nThis covers emit_binary_diff_body.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 39 ++++++++++++++++++++++-----------------\n 1 file changed, 22 insertions(+), 17 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 3357c0fca3..b5a5261a4e 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2244,8 +2244,8 @@ static unsigned char *deflate_it(char *data,\n \treturn deflated;\n }\n \n-static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t\t const char *prefix)\n+static void emit_binary_diff_body(struct diff_options *o,\n+\t\t\t\t mmfile_t *one, mmfile_t *two)\n {\n \tvoid *cp;\n \tvoid *delta;\n@@ -2274,13 +2274,12 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t}\n \n \tif (delta && delta_size < deflate_size) {\n-\t\tfprintf(file, \"%sdelta %lu\\n\", prefix, orig_size);\n+\t\temit_line_fmt(o, NULL, NULL, 1, \"delta %lu\\n\", orig_size);\n \t\tfree(deflated);\n \t\tdata = delta;\n \t\tdata_size = delta_size;\n-\t}\n-\telse {\n-\t\tfprintf(file, \"%sliteral %lu\\n\", prefix, two->size);\n+\t} else {\n+\t\temit_line_fmt(o, NULL, NULL, 1, \"literal %lu\\n\", two->size);\n \t\tfree(delta);\n \t\tdata = deflated;\n \t\tdata_size = deflate_size;\n@@ -2289,8 +2288,9 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t/* emit data encoded in base85 */\n \tcp = data;\n \twhile (data_size) {\n+\t\tint len;\n \t\tint bytes = (52 < data_size) ? 52 : data_size;\n-\t\tchar line[70];\n+\t\tchar line[71];\n \t\tdata_size -= bytes;\n \t\tif (bytes <= 26)\n \t\t\tline[0] = bytes + 'A' - 1;\n@@ -2298,20 +2298,25 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t\t\tline[0] = bytes - 26 + 'a' - 1;\n \t\tencode_85(line + 1, cp, bytes);\n \t\tcp = (char *) cp + bytes;\n-\t\tfprintf(file, \"%s\", prefix);\n-\t\tfputs(line, file);\n-\t\tfputc('\\n', file);\n+\n+\t\tlen = strlen(line);\n+\t\tline[len++] = '\\n';\n+\t\tline[len] = '\\0';\n+\n+\t\temit_line(o, NULL, NULL, 1, 0, line, len);\n \t}\n-\tfprintf(file, \"%s\\n\", prefix);\n+\temit_line(o, NULL, NULL, 1, 0, \"\\n\", 1);\n \tfree(data);\n }\n \n-static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t const char *prefix)\n+static void emit_binary_diff(struct diff_options *o,\n+\t\t\t mmfile_t *one, mmfile_t *two)\n {\n-\tfprintf(file, \"%sGIT binary patch\\n\", prefix);\n-\temit_binary_diff_body(file, one, two, prefix);\n-\temit_binary_diff_body(file, two, one, prefix);\n+\tconst char *s = \"GIT binary patch\\n\";\n+\tconst int len = strlen(s);\n+\temit_line(o, NULL, NULL, 1, 0, s, len);\n+\temit_binary_diff_body(o, one, two);\n+\temit_binary_diff_body(o, two, one);\n }\n \n int diff_filespec_is_binary(struct diff_filespec *one)\n@@ -2498,7 +2503,7 @@ static void builtin_diff(const char *name_a,\n \t\t\t header.buf, header.len);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n-\t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n+\t\t\temit_binary_diff(o, &mf1, &mf2);\n \t\telse\n \t\t\temit_line_fmt(o, NULL, NULL, 1,\n \t\t\t\t \"Binary files %s and %s differ\\n\",\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320677","messageId":"20170524214036.29623-9-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 08/17] diff.c: convert emit_rewrite_lines to use emit_line_*","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:27Z","receivedAt":"2017-05-24T21:41:30Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nThis covers emit_rewrite_lines.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 25 +++++++++++++++++--------\n 1 file changed, 17 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 3dda9f3c8e..ca6b48cf49 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -722,15 +722,23 @@ static void add_line_count(struct strbuf *out, int count)\n static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\t\t int prefix, const char *data, int size)\n {\n-\tconst char *endp = NULL;\n-\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);\n+\tstruct strbuf sb = STRBUF_INIT;\n \n \twhile (0 < size) {\n \t\tint len;\n \n-\t\tendp = memchr(data, '\\n', size);\n-\t\tlen = endp ? (endp - data + 1) : size;\n+\t\tconst char *endp = memchr(data, '\\n', size);\n+\t\tif (endp)\n+\t\t\tlen = endp - data + 1;\n+\t\telse {\n+\t\t\tstrbuf_add(&sb, data, size);\n+\t\t\tstrbuf_addch(&sb, '\\n');\n+\t\t\tsize = 0; /* to exit the loop. */\n+\n+\t\t\tdata = sb.buf;\n+\t\t\tlen = sb.len;\n+\t\t}\n \t\tif (prefix != '+') {\n \t\t\tecb->lno_in_preimage++;\n \t\t\temit_del_line(reset, ecb, data, len);\n@@ -741,12 +749,13 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tsize -= len;\n \t\tdata += len;\n \t}\n-\tif (!endp) {\n+\tif (sb.len) {\n+\t\tstatic const char *nneof = \"\\\\ No newline at end of file\\n\";\n \t\tconst char *context = diff_get_color(ecb->color_diff,\n \t\t\t\t\t\t DIFF_CONTEXT);\n-\t\tputc('\\n', ecb->opt->file);\n-\t\temit_line(ecb->opt, context, reset, 1, '\\\\',\n-\t\t\t nneof, strlen(nneof));\n+\t\temit_line(ecb->opt, context, reset, 1, 0,\n+\t\t\t nneof, strlen(nneof));\n+\t\tstrbuf_release(&sb);\n \t}\n }\n \n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320678","messageId":"20170524214036.29623-17-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 16/17] diff: buffer all output if asked to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:35Z","receivedAt":"2017-05-24T21:41:34Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Introduce a new option 'use_buffer' in the struct diff_options which\ncontrols whether all output is buffered up until all output is available.\n\nWe'll have a new struct 'diff_line' in diff.h which will be used to buffer\neach line. The diff_line will duplicate the memory of the line to buffer\nas that is easiest to reason about for now. In a future patch we may want\nto decrease the memory usage by not duplicating all output for buffering\nbut rather we may want to store offsets into the file or in case of hunk\ndescriptions such as the similarity score, we could just store the\nrelevant number and reproduce the text later on.\n\nThis approach was chosen as a first step because it is quite simple\ncompared to the alternative with less memory footprint.\n\nemit_line factors out the emission part into emit_line_emission,\nand depending on the diff_options->use_buffer the emission\nwill be performed directly when calling emit_line or after the\nwhole process is done, i.e. by buffering we have add the possibility\nfor a second pass over the whole output before doing the actual\noutput.\n\nIn 6440d34 (2012-03-14, diff: tweak a _copy_ of diff_options with\nword-diff) we introduced a duplicate diff options struct for word\nemissions as we may have different regex settings in there.\nWhen buffering the output, we need to operate on just one buffer,\nso we have to copy back the emissions of the word buffer into the\nmain buffer.\n\nUnconditionally enable output via buffer in this patch as it yields\na great opportunity for testing, i.e. all the diff tests from the\ntest suite pass without having reordering issues (i.e. only parts\nof the output got buffered, and we forgot to buffer other parts).\nThe test suite passes, which gives confidence that we converted all\nfunctions to use emit_line for output.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++---------------\n diff.h | 41 +++++++++++++++++\n 2 files changed, 161 insertions(+), 35 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 514c5facd7..8e06206881 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -516,54 +516,85 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,\n \tecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n }\n \n-\n-static void emit_line(struct diff_options *o,\n-\t\t const char *set, const char *reset,\n-\t\t int add_line_prefix, int markup_ws,\n-\t\t int sign, const char *line, int len)\n+static void emit_diff_line(struct diff_options *o,\n+\t\t\t struct diff_line *e)\n {\n \tconst char *ws;\n \tint has_trailing_newline, has_trailing_carriage_return;\n+\tint len = e->len;\n \tFILE *file = o->file;\n \n-\tif (add_line_prefix)\n+\tif (e->add_line_prefix)\n \t\tfputs(diff_line_prefix(o), file);\n \n-\tif (markup_ws) {\n+\tswitch (e->state) {\n+\tcase DIFF_LINE_WS:\n \t\tws = diff_get_color(o->use_color, DIFF_WHITESPACE);\n+\t\tif (e->set)\n+\t\t\tfputs(e->set, file);\n+\t\tif (e->sign)\n+\t\t\tfputc(e->sign, file);\n+\t\tif (e->reset)\n+\t\t\tfputs(e->reset, file);\n+\t\tws_check_emit(e->line, e->len, o->ws_rule,\n+\t\t\t file, e->set, e->reset, ws);\n+\t\treturn;\n+\tcase DIFF_LINE_ASIS:\n+\t\thas_trailing_newline = (len > 0 && e->line[len-1] == '\\n');\n+\t\tif (has_trailing_newline)\n+\t\t\tlen--;\n+\t\thas_trailing_carriage_return = (len > 0 && e->line[len-1] == '\\r');\n+\t\tif (has_trailing_carriage_return)\n+\t\t\tlen--;\n \n-\t\tif (set)\n-\t\t\tfputs(set, file);\n-\t\tif (sign)\n-\t\t\tfputc(sign, file);\n-\t\tif (reset)\n-\t\t\tfputs(reset, file);\n-\t\tws = diff_get_color(o->use_color, DIFF_WHITESPACE);\n-\t\tws_check_emit(line, len, o->ws_rule,\n-\t\t\t file, set, reset, ws);\n+\t\tif (len || e->sign) {\n+\t\t\tif (e->set)\n+\t\t\t\tfputs(e->set, file);\n+\t\t\tif (e->sign)\n+\t\t\t\tfputc(e->sign, file);\n+\t\t\tfwrite(e->line, len, 1, file);\n+\t\t\tif (e->reset)\n+\t\t\t\tfputs(e->reset, file);\n+\t\t}\n+\t\tif (has_trailing_carriage_return)\n+\t\t\tfputc('\\r', file);\n+\t\tif (has_trailing_newline)\n+\t\t\tfputc('\\n', file);\n+\t\treturn;\n+\tcase DIFF_LINE_RELOAD_WS_RULE:\n+\t\to->ws_rule = whitespace_rule(e->line);\n \t\treturn;\n+\tdefault:\n+\t\tdie(\"BUG: malformatted buffered patch line: '%d'\", e->state);\n \t}\n+}\n \n-\thas_trailing_newline = (len > 0 && line[len-1] == '\\n');\n-\tif (has_trailing_newline)\n-\t\tlen--;\n-\thas_trailing_carriage_return = (len > 0 && line[len-1] == '\\r');\n-\tif (has_trailing_carriage_return)\n-\t\tlen--;\n+static void append_diff_line(struct diff_options *o,\n+\t\t\t struct diff_line *e)\n+{\n+\tstruct diff_line *f;\n+\tALLOC_GROW(o->line_buffer,\n+\t\t o->line_buffer_nr + 1,\n+\t\t o->line_buffer_alloc);\n+\tf = &o->line_buffer[o->line_buffer_nr++];\n \n-\tif (len || sign) {\n-\t\tif (set)\n-\t\t\tfputs(set, file);\n-\t\tif (sign)\n-\t\t\tfputc(sign, file);\n-\t\tfwrite(line, len, 1, file);\n-\t\tif (reset)\n-\t\t\tfputs(reset, file);\n-\t}\n-\tif (has_trailing_carriage_return)\n-\t\tfputc('\\r', file);\n-\tif (has_trailing_newline)\n-\t\tfputc('\\n', file);\n+\tmemcpy(f, e, sizeof(struct diff_line));\n+\tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n+}\n+\n+static void emit_line(struct diff_options *o,\n+\t\t const char *set, const char *reset,\n+\t\t int add_line_prefix, int markup_ws,\n+\t\t int sign, const char *line, int len)\n+{\n+\tstruct diff_line e = {set, reset, line,\n+\t\tlen, sign, add_line_prefix,\n+\t\tmarkup_ws ? DIFF_LINE_WS : DIFF_LINE_ASIS};\n+\n+\tif (o->use_buffer)\n+\t\tappend_diff_line(o, &e);\n+\telse\n+\t\temit_diff_line(o, &e);\n }\n \n static void emit_line_fmt(struct diff_options *o,\n@@ -1172,6 +1203,18 @@ static void diff_words_flush(struct emit_callback *ecbdata)\n \tif (ecbdata->diff_words->minus.text.size ||\n \t ecbdata->diff_words->plus.text.size)\n \t\tdiff_words_show(ecbdata->diff_words);\n+\n+\tif (ecbdata->diff_words->opt->line_buffer_nr) {\n+\t\tint i;\n+\t\tfor (i = 0; i < ecbdata->diff_words->opt->line_buffer_nr; i++)\n+\t\t\tappend_diff_line(ecbdata->opt,\n+\t\t\t\t&ecbdata->diff_words->opt->line_buffer[i]);\n+\n+\t\tfor (i = 0; i < ecbdata->diff_words->opt->line_buffer_nr; i++)\n+\t\t\tfree((void *)ecbdata->diff_words->opt->line_buffer[i].line);\n+\n+\t\tecbdata->diff_words->opt->line_buffer_nr = 0;\n+\t}\n }\n \n static void diff_filespec_load_driver(struct diff_filespec *one)\n@@ -1207,6 +1250,11 @@ static void init_diff_words_data(struct emit_callback *ecbdata,\n \t\txcalloc(1, sizeof(struct diff_words_data));\n \tecbdata->diff_words->type = o->word_diff;\n \tecbdata->diff_words->opt = o;\n+\n+\to->line_buffer = NULL;\n+\to->line_buffer_nr = 0;\n+\to->line_buffer_alloc = 0;\n+\n \tif (!o->word_regex)\n \t\to->word_regex = userdiff_word_regex(one);\n \tif (!o->word_regex)\n@@ -1241,6 +1289,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)\n {\n \tif (ecbdata->diff_words) {\n \t\tdiff_words_flush(ecbdata);\n+\t\tfree (ecbdata->diff_words->opt->line_buffer);\n \t\tfree (ecbdata->diff_words->opt);\n \t\tfree (ecbdata->diff_words->minus.text.ptr);\n \t\tfree (ecbdata->diff_words->minus.orig);\n@@ -2579,6 +2628,13 @@ static void builtin_diff(const char *name_a,\n \t\t\txecfg.ctxlen = strtoul(v, NULL, 10);\n \t\tif (o->word_diff)\n \t\t\tinit_diff_words_data(&ecbdata, o, one, two);\n+\t\tif (o->use_buffer) {\n+\t\t\tstruct diff_line e = diff_line_INIT;\n+\t\t\te.state = DIFF_LINE_RELOAD_WS_RULE;\n+\t\t\te.line = name_b;\n+\t\t\te.len = strlen(name_b);\n+\t\t\tappend_diff_line(o, &e);\n+\t\t}\n \t\tif (xdi_diff_outf(&mf1, &mf2, fn_out_consume, &ecbdata,\n \t\t\t\t &xpp, &xecfg))\n \t\t\tdie(\"unable to generate diff for %s\", one->path);\n@@ -3458,6 +3514,10 @@ void diff_setup(struct diff_options *options)\n \t\toptions->a_prefix = \"a/\";\n \t\toptions->b_prefix = \"b/\";\n \t}\n+\n+\toptions->line_buffer = NULL;\n+\toptions->line_buffer_nr = 0;\n+\toptions->line_buffer_alloc = 0;\n }\n \n void diff_setup_done(struct diff_options *options)\n@@ -4796,11 +4856,36 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n+\t/*\n+\t * For testing purposes we want to make sure the diff machinery\n+\t * works completely with the buffer. If there is anything emitted\n+\t * outside the emit_diff_line, then the order is screwed\n+\t * up and the tests will fail.\n+\t *\n+\t * TODO (later in this series):\n+\t * We'll unset this flag in a later patch.\n+\t */\n+\to->use_buffer = 1;\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tif (check_pair_status(p))\n \t\t\tdiff_flush_patch(p, o);\n \t}\n+\n+\tif (o->use_buffer) {\n+\t\tfor (i = 0; i < o->line_buffer_nr; i++)\n+\t\t\temit_diff_line(o, &o->line_buffer[i]);\n+\n+\t\tfor (i = 0; i < o->line_buffer_nr; i++)\n+\t\t\tfree((void *)o->line_buffer[i].line);\n+\n+\t\tfree(o->line_buffer);\n+\n+\t\to->line_buffer = NULL;\n+\t\to->line_buffer_nr = 0;\n+\t\to->line_buffer_alloc = 0;\n+\t}\n }\n \n void diff_flush(struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex 85948ed65a..fad1258556 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -115,6 +115,42 @@ enum diff_submodule_format {\n \tDIFF_SUBMODULE_INLINE_DIFF\n };\n \n+/*\n+ * This struct is used when we need to buffer the output of the diff output.\n+ *\n+ * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer\n+ * into the pre/post image file. This pointer could be a union with the\n+ * line pointer. By storing an offset into the file instead of the literal line,\n+ * we can decrease the memory footprint for the buffered output. At first we\n+ * may want to only have indirection for the content lines, but we could also\n+ * enhance the state for emitting prefabricated lines, e.g. the similarity\n+ * score line or hunk/file headers would only need to store a number or path\n+ * and then the output can be constructed later on depending on state.\n+ */\n+struct diff_line {\n+\tconst char *set;\n+\tconst char *reset;\n+\tconst char *line;\n+\tint len;\n+\tint sign;\n+\tint add_line_prefix;\n+\tenum {\n+\t\t/*\n+\t\t * Emits [lineprefix][set][sign][reset] and then calls\n+\t\t * ws_check_emit which will output \"line\", marked up\n+\t\t * according to ws_rule.\n+\t\t */\n+\t\tDIFF_LINE_WS,\n+\n+\t\t/* Emits [lineprefix][set][sign] line [reset] */\n+\t\tDIFF_LINE_ASIS,\n+\n+\t\t/* Reloads the ws_rule; line contains the file name */\n+\t\tDIFF_LINE_RELOAD_WS_RULE\n+\t} state;\n+};\n+#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}\n+\n struct diff_options {\n \tconst char *orderfile;\n \tconst char *pickaxe;\n@@ -188,8 +224,13 @@ struct diff_options {\n \tint diff_path_counter;\n \n \tunsigned ws_rule;\n+\tint use_buffer;\n+\n+\tstruct diff_line *line_buffer;\n+\tint line_buffer_nr, line_buffer_alloc;\n };\n \n+/* Emit [line_prefix] [set] line [reset] */\n void diff_emit_line(struct diff_options *o, const char *set, const char *reset,\n \t\t const char *line, int len);\n \n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320679","messageId":"20170524214036.29623-13-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 12/17] diff.c: convert word diffing to use emit_line_*","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:31Z","receivedAt":"2017-05-24T21:41:38Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nThis covers all code related to diffing words.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 73 +++++++++++++++++++++++++++++-------------------------------------\n 1 file changed, 32 insertions(+), 41 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex a76abf5f69..8317824963 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -897,37 +897,42 @@ struct diff_words_data {\n \tstruct diff_words_style *style;\n };\n \n-static int fn_out_diff_words_write_helper(FILE *fp,\n+static int fn_out_diff_words_write_helper(struct diff_options *o,\n \t\t\t\t\t struct diff_words_style_elem *st_el,\n \t\t\t\t\t const char *newline,\n-\t\t\t\t\t size_t count, const char *buf,\n-\t\t\t\t\t const char *line_prefix)\n+\t\t\t\t\t size_t count, const char *buf)\n {\n \tint print = 0;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n \twhile (count) {\n \t\tchar *p = memchr(buf, '\\n', count);\n \t\tif (print)\n-\t\t\tfputs(line_prefix, fp);\n+\t\t\temit_line(o, NULL, NULL, 1, 0, \"\", 0);\n+\n \t\tif (p != buf) {\n-\t\t\tif (st_el->color && fputs(st_el->color, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (fputs(st_el->prefix, fp) < 0 ||\n-\t\t\t fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||\n-\t\t\t fputs(st_el->suffix, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (st_el->color && *st_el->color\n-\t\t\t && fputs(GIT_COLOR_RESET, fp) < 0)\n-\t\t\t\treturn -1;\n+\t\t\tconst char *reset = st_el->color && *st_el->color ?\n+\t\t\t\t\t GIT_COLOR_RESET : NULL;\n+\t\t\tstrbuf_addstr(&sb, st_el->prefix);\n+\t\t\tstrbuf_add(&sb, buf, p ? p - buf : count);\n+\t\t\tstrbuf_addstr(&sb, st_el->suffix);\n+\t\t\temit_line(o, st_el->color, reset,\n+\t\t\t\t 0, 0, sb.buf, sb.len);\n+\t\t\tstrbuf_reset(&sb);\n \t\t}\n \t\tif (!p)\n-\t\t\treturn 0;\n-\t\tif (fputs(newline, fp) < 0)\n-\t\t\treturn -1;\n+\t\t\tgoto out;\n+\n+\t\tstrbuf_addstr(&sb, newline);\n+\t\temit_line(o, NULL, NULL, 0, 0, sb.buf, sb.len);\n+\t\tstrbuf_reset(&sb);\n \t\tcount -= p + 1 - buf;\n \t\tbuf = p + 1;\n \t\tprint = 1;\n \t}\n+\n+out:\n+\tstrbuf_release(&sb);\n \treturn 0;\n }\n \n@@ -981,14 +986,12 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)\n \tint minus_first, minus_len, plus_first, plus_len;\n \tconst char *minus_begin, *minus_end, *plus_begin, *plus_end;\n \tstruct diff_options *opt = diff_words->opt;\n-\tconst char *line_prefix;\n \n \tif (line[0] != '@' || parse_hunk_header(line, len,\n \t\t\t&minus_first, &minus_len, &plus_first, &plus_len))\n \t\treturn;\n \n \tassert(opt);\n-\tline_prefix = diff_line_prefix(opt);\n \n \t/* POSIX requires that first be decremented by one if len == 0... */\n \tif (minus_len) {\n@@ -1005,28 +1008,21 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)\n \t} else\n \t\tplus_begin = plus_end = diff_words->plus.orig[plus_first].end;\n \n-\tif (color_words_output_graph_prefix(diff_words)) {\n-\t\tfputs(line_prefix, diff_words->opt->file);\n-\t}\n \tif (diff_words->current_plus != plus_begin) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->ctx, style->newline,\n \t\t\t\tplus_begin - diff_words->current_plus,\n-\t\t\t\tdiff_words->current_plus, line_prefix);\n-\t\tif (*(plus_begin - 1) == '\\n')\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n+\t\t\t\tdiff_words->current_plus);\n \t}\n \tif (minus_begin != minus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->old, style->newline,\n-\t\t\t\tminus_end - minus_begin, minus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tminus_end - minus_begin, minus_begin);\n \t}\n \tif (plus_begin != plus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->new, style->newline,\n-\t\t\t\tplus_end - plus_begin, plus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tplus_end - plus_begin, plus_begin);\n \t}\n \n \tdiff_words->current_plus = plus_end;\n@@ -1113,18 +1109,14 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \tstruct diff_words_style *style = diff_words->style;\n \n \tstruct diff_options *opt = diff_words->opt;\n-\tconst char *line_prefix;\n-\n \tassert(opt);\n-\tline_prefix = diff_line_prefix(opt);\n \n \t/* special case: only removal */\n \tif (!diff_words->plus.text.size) {\n-\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->old, style->newline,\n \t\t\tdiff_words->minus.text.size,\n-\t\t\tdiff_words->minus.text.ptr, line_prefix);\n+\t\t\tdiff_words->minus.text.ptr);\n \t\tdiff_words->minus.text.size = 0;\n \t\treturn;\n \t}\n@@ -1147,12 +1139,11 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \tif (diff_words->current_plus != diff_words->plus.text.ptr +\n \t\t\tdiff_words->plus.text.size) {\n \t\tif (color_words_output_graph_prefix(diff_words))\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\t\temit_line(diff_words->opt, NULL, NULL, 1, 0, \"\", 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->ctx, style->newline,\n \t\t\tdiff_words->plus.text.ptr + diff_words->plus.text.size\n-\t\t\t- diff_words->current_plus, diff_words->current_plus,\n-\t\t\tline_prefix);\n+\t\t\t- diff_words->current_plus, diff_words->current_plus);\n \t}\n \tdiff_words->minus.text.size = diff_words->plus.text.size = 0;\n }\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320680","messageId":"20170524214036.29623-15-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 14/17] diff.c: convert diff_summary to use emit_line_*","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:33Z","receivedAt":"2017-05-24T21:41:44Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nThis covers diff_summary.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 64 ++++++++++++++++++++++++++++++++++------------------------------\n 1 file changed, 34 insertions(+), 30 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 8ebe673331..76cafde4be 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4504,67 +4504,71 @@ static void flush_one_pair(struct diff_filepair *p, struct diff_options *opt)\n \t}\n }\n \n-static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)\n+static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tif (fs->mode)\n-\t\tfprintf(file, \" %s mode %06o \", newdelete, fs->mode);\n+\t\tstrbuf_addf(&sb, \" %s mode %06o \", newdelete, fs->mode);\n \telse\n-\t\tfprintf(file, \" %s \", newdelete);\n-\twrite_name_quoted(fs->path, file, '\\n');\n-}\n+\t\tstrbuf_addf(&sb, \" %s \", newdelete);\n \n+\tquote_c_style(fs->path, &sb, NULL, 0);\n+\tstrbuf_addch(&sb, '\\n');\n+\temit_line(opt, NULL, NULL, 1, 0, sb.buf, sb.len);\n+\tstrbuf_release(&sb);\n+}\n \n-static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,\n-\t\tconst char *line_prefix)\n+static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,\n+\t\tint show_name)\n {\n \tif (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {\n-\t\tfprintf(file, \"%s mode change %06o => %06o%c\", line_prefix, p->one->mode,\n-\t\t\tp->two->mode, show_name ? ' ' : '\\n');\n+\t\tstruct strbuf sb = STRBUF_INIT;\n \t\tif (show_name) {\n-\t\t\twrite_name_quoted(p->two->path, file, '\\n');\n+\t\t\tstrbuf_addch(&sb, ' ');\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n \t\t}\n+\t\temit_line_fmt(opt, NULL, NULL, 1,\n+\t\t\t \" mode change %06o => %06o%s\\n\",\n+\t\t\t p->one->mode, p->two->mode,\n+\t\t\t show_name ? sb.buf : \"\");\n+\t\tstrbuf_release(&sb);\n \t}\n }\n \n-static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,\n-\t\t\tconst char *line_prefix)\n+static void show_rename_copy(struct diff_options *opt, const char *renamecopy,\n+\t\tstruct diff_filepair *p)\n {\n \tchar *names = pprint_rename(p->one->path, p->two->path);\n-\n-\tfprintf(file, \" %s %s (%d%%)\\n\", renamecopy, names, similarity_index(p));\n+\temit_line_fmt(opt, NULL, NULL, 1, \" %s %s (%d%%)\\n\",\n+\t\t renamecopy, names, similarity_index(p));\n \tfree(names);\n-\tshow_mode_change(file, p, 0, line_prefix);\n+\tshow_mode_change(opt, p, 0);\n }\n \n static void diff_summary(struct diff_options *opt, struct diff_filepair *p)\n {\n-\tFILE *file = opt->file;\n-\tconst char *line_prefix = diff_line_prefix(opt);\n-\n \tswitch(p->status) {\n \tcase DIFF_STATUS_DELETED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"delete\", p->one);\n+\t\tshow_file_mode_name(opt, \"delete\", p->one);\n \t\tbreak;\n \tcase DIFF_STATUS_ADDED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"create\", p->two);\n+\t\tshow_file_mode_name(opt, \"create\", p->two);\n \t\tbreak;\n \tcase DIFF_STATUS_COPIED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"copy\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"copy\", p);\n \t\tbreak;\n \tcase DIFF_STATUS_RENAMED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"rename\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"rename\", p);\n \t\tbreak;\n \tdefault:\n \t\tif (p->score) {\n-\t\t\tfprintf(file, \"%s rewrite \", line_prefix);\n-\t\t\twrite_name_quoted(p->two->path, file, ' ');\n-\t\t\tfprintf(file, \"(%d%%)\\n\", similarity_index(p));\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addstr(&sb, \" rewrite \");\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n+\t\t\tstrbuf_addf(&sb, \" (%d%%)\\n\", similarity_index(p));\n+\t\t\temit_line(opt, NULL, NULL, 1, 0, sb.buf, sb.len);\n \t\t}\n-\t\tshow_mode_change(file, p, !p->score, line_prefix);\n+\t\tshow_mode_change(opt, p, !p->score);\n \t\tbreak;\n \t}\n }\n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320681","messageId":"20170524214036.29623-14-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 13/17] diff.c: convert diff_flush to use emit_line_*","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:32Z","receivedAt":"2017-05-24T21:41:47Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nThis covers diff_flush.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 +++-\n 1 file changed, 3 insertions(+), 1 deletion(-)\n\ndiff --git a/diff.c b/diff.c\nindex 8317824963..8ebe673331 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4872,7 +4872,9 @@ void diff_flush(struct diff_options *options)\n \t\t\temit_line(options, NULL, NULL, 1, 0, term, !!term[0]);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n-\t\t\t\tfputs(options->stat_sep, options->file);\n+\t\t\t\temit_line(options, NULL, NULL, 0, 0,\n+\t\t\t\t\t options->stat_sep,\n+\t\t\t\t\t strlen(options->stat_sep));\n \t\t\t}\n \t\t}\n \n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320682","messageId":"20170524214036.29623-7-sbeller@google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"[PATCHv5 06/17] diff.c: convert builtin_diff to use emit_line_*","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-24T21:40:25Z","receivedAt":"2017-05-24T21:41:49Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch, I want to propose an option to detect&color\nmoved lines in a diff, which cannot be done in a one-pass over\nthe diff. Instead we need to go over the whole diff twice,\nbecause we cannot detect the first line of the two corresponding\nlines (+ and -) that got moved.\n\nSo to prepare the diff machinery for two pass algorithms\n(i.e. buffer it all up and then operate on the result),\nmove all emissions to places, such that the only emitting\nfunction is emit_line_0.\n\nThis covers builtin_diff.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 33 ++++++++++++++++++++-------------\n 1 file changed, 20 insertions(+), 13 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 8186289734..4fa976d43c 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -1289,8 +1289,9 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \to->found_changes = 1;\n \n \tif (ecbdata->header) {\n-\t\tfprintf(o->file, \"%s\", ecbdata->header->buf);\n-\t\tstrbuf_reset(ecbdata->header);\n+\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\t\t ecbdata->header->buf, ecbdata->header->len);\n+\t\tstrbuf_release(ecbdata->header);\n \t\tecbdata->header = NULL;\n \t}\n \n@@ -2435,7 +2436,7 @@ static void builtin_diff(const char *name_a,\n \t\tif (complete_rewrite &&\n \t\t (textconv_one || !diff_filespec_is_binary(one)) &&\n \t\t (textconv_two || !diff_filespec_is_binary(two))) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_line(o, NULL, NULL, 0, 0, header.buf, header.len);\n \t\t\tstrbuf_reset(&header);\n \t\t\temit_rewrite_diff(name_a, name_b, one, two,\n \t\t\t\t\t\ttextconv_one, textconv_two, o);\n@@ -2445,7 +2446,7 @@ static void builtin_diff(const char *name_a,\n \t}\n \n \tif (o->irreversible_delete && lbl[1][0] == '/') {\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_line(o, NULL, NULL, 0, 0, header.buf, header.len);\n \t\tstrbuf_reset(&header);\n \t\tgoto free_ab_and_return;\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n@@ -2456,12 +2457,15 @@ static void builtin_diff(const char *name_a,\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n \t\t\tif (!oidcmp(&one->oid, &two->oid)) {\n \t\t\t\tif (must_show_header)\n-\t\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\t\t\t\t\t header.buf, header.len);\n \t\t\t\tgoto free_ab_and_return;\n \t\t\t}\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\t\t\t header.buf, header.len);\n+\t\t\temit_line_fmt(o, NULL, NULL, 1,\n+\t\t\t\t \"Binary files %s and %s differ\\n\",\n+\t\t\t\t lbl[0], lbl[1]);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n \t\tif (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)\n@@ -2470,16 +2474,19 @@ static void builtin_diff(const char *name_a,\n \t\tif (mf1.size == mf2.size &&\n \t\t !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {\n \t\t\tif (must_show_header)\n-\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\t\t\t\t header.buf, header.len);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_line(o, NULL, NULL, 0, 0,\n+\t\t\t header.buf, header.len);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n \t\telse\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\t\temit_line_fmt(o, NULL, NULL, 1,\n+\t\t\t\t \"Binary files %s and %s differ\\n\",\n+\t\t\t\t lbl[0], lbl[1]);\n \t\to->found_changes = 1;\n \t} else {\n \t\t/* Crazy xdl interfaces.. */\n@@ -2491,7 +2498,7 @@ static void builtin_diff(const char *name_a,\n \t\tconst struct userdiff_funcname *pe;\n \n \t\tif (must_show_header) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_line(o, NULL, NULL, 0, 0, header.buf, header.len);\n \t\t\tstrbuf_reset(&header);\n \t\t}\n \n-- \n2.13.0.18.g7d86cc8ba0\n\n"},{"id":"320686","messageId":"xmqqo9uhfz8g.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170524214036.29623-17-sbeller@google.com","subject":"Re: [PATCHv5 16/17] diff: buffer all output if asked to","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-25T02:26:39Z","receivedAt":"2017-05-25T02:26:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Introduce a new option 'use_buffer' in the struct diff_options which\n> controls whether all output is buffered up until all output is available.\n> ...\n> Unconditionally enable output via buffer in this patch as it yields\n> a great opportunity for testing, i.e. all the diff tests from the\n> test suite pass without having reordering issues (i.e. only parts\n> of the output got buffered, and we forgot to buffer other parts).\n> The test suite passes, which gives confidence that we converted all\n> functions to use emit_line for output.\n>\n> Signed-off-by: Stefan Beller \n> Signed-off-by: Junio C Hamano \n\nOh, did I?\n\n> ---\n> diff.c | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++---------------\n> diff.h | 41 +++++++++++++++++\n> 2 files changed, 161 insertions(+), 35 deletions(-)\n>\n> diff --git a/diff.c b/diff.c\n> index 514c5facd7..8e06206881 100644\n> --- a/diff.c\n> +++ b/diff.c\n> ...\n> @@ -2579,6 +2628,13 @@ static void builtin_diff(const char *name_a,\n> \t\t\txecfg.ctxlen = strtoul(v, NULL, 10);\n> \t\tif (o->word_diff)\n> \t\t\tinit_diff_words_data(&ecbdata, o, one, two);\n> +\t\tif (o->use_buffer) {\n> +\t\t\tstruct diff_line e = diff_line_INIT;\n\nThis ...\n\n> +\t\t\te.state = DIFF_LINE_RELOAD_WS_RULE;\n> ...\n> +#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}\n\n... and this should be in all caps. We do not say\n\n\tstruct strbuf buf = strbuf_INIT;\n\nand we should do the same for this new thing.\n\n"},{"id":"320687","messageId":"xmqqk255fz7b.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170524214036.29623-18-sbeller@google.com","subject":"Re: [PATCHv5 17/17] diff.c: color moved lines differently","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-25T02:27:20Z","receivedAt":"2017-05-25T02:27:28Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> When a patch consists mostly of moving blocks of code around, it can\n> be quite tedious to ensure that the blocks are moved verbatim, and not\n> ...\n> cases. This leads to another thought: We could pass on '--color-moved' to\n> submodules such that they color up moved lines for themselves. If we'd do\n> so only line moves within a repository boundary are marked up.\n>\n> Helped-by: Jonathan Tan \n> Signed-off-by: Stefan Beller \n> Signed-off-by: Junio C Hamano \n>\n> # Conflicts:\n> #\tdiff.c\n> Signed-off-by: Junio C Hamano \n> Signed-off-by: Stefan Beller \n> ---\n\nHmph, what are these final lines about?\n"},{"id":"320696","messageId":"CAGZ79kZM5KPP7iDo7rT5gxsooRrftJrA=+3-WBXW3nqfXeTy4Q@mail.gmail.com","threadId":"46038","inReplyTo":"xmqqo9uhfz8g.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCHv5 16/17] diff: buffer all output if asked to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-25T05:34:33Z","receivedAt":"2017-05-25T05:34:39Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, May 24, 2017 at 7:26 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> Introduce a new option 'use_buffer' in the struct diff_options which\n>> controls whether all output is buffered up until all output is available.\n>> ...\n>> Unconditionally enable output via buffer in this patch as it yields\n>> a great opportunity for testing, i.e. all the diff tests from the\n>> test suite pass without having reordering issues (i.e. only parts\n>> of the output got buffered, and we forgot to buffer other parts).\n>> The test suite passes, which gives confidence that we converted all\n>> functions to use emit_line for output.\n>>\n>> Signed-off-by: Stefan Beller \n>> Signed-off-by: Junio C Hamano \n>\n> Oh, did I?\n\nYes, this is essentially the v4 with small indentation fixes, so I\nassumed your signoff is still valid.\n\nWhich leads to explaining my workflow (which I think we\ndiscussed that half a year ago with Dscho in a longer thread).\n\nAs soon as you apply a series I take that series and work off\nthat series, because you explained you would do smaller\nfixups occasionally.\n\nPatches that change drastically, I strip off your signoff and\npretend like I created them from scratch, but for those which\nI barely touch (if at all), I do not remove your signoff, as\nI'd be just passing them along again.\n\nMaybe I have to rethink that strategy.\n\n>\n>> ---\n>> diff.c | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++---------------\n>> diff.h | 41 +++++++++++++++++\n>> 2 files changed, 161 insertions(+), 35 deletions(-)\n>>\n>> diff --git a/diff.c b/diff.c\n>> index 514c5facd7..8e06206881 100644\n>> --- a/diff.c\n>> +++ b/diff.c\n>> ...\n>> @@ -2579,6 +2628,13 @@ static void builtin_diff(const char *name_a,\n>> xecfg.ctxlen = strtoul(v, NULL, 10);\n>> if (o->word_diff)\n>> init_diff_words_data(&ecbdata, o, one, two);\n>> + if (o->use_buffer) {\n>> + struct diff_line e = diff_line_INIT;\n>\n> This ...\n>\n>> + e.state = DIFF_LINE_RELOAD_WS_RULE;\n>> ...\n>> +#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}\n>\n> ... and this should be in all caps. We do not say\n>\n> struct strbuf buf = strbuf_INIT;\n>\n> and we should do the same for this new thing.\n\nYes. Totally agree. That is fallout from a mechanical\nreplace all s/buffered_patch_line/diff_line/ and the case\nsensitivity got lost.\n\nWill fix again.\n\nStefan\n"},{"id":"320697","messageId":"CAGZ79kZBOoK2vd5EqottnUr1AYMHa1=TzQB8ga00TehQxgtuhA@mail.gmail.com","threadId":"46038","inReplyTo":"xmqqk255fz7b.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCHv5 17/17] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-25T05:39:40Z","receivedAt":"2017-05-25T05:40:02Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, May 24, 2017 at 7:27 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> When a patch consists mostly of moving blocks of code around, it can\n>> be quite tedious to ensure that the blocks are moved verbatim, and not\n>> ...\n>> cases. This leads to another thought: We could pass on '--color-moved' to\n>> submodules such that they color up moved lines for themselves. If we'd do\n>> so only line moves within a repository boundary are marked up.\n>>\n>> Helped-by: Jonathan Tan \n>> Signed-off-by: Stefan Beller \n>> Signed-off-by: Junio C Hamano \n>>\n>> # Conflicts:\n>> # diff.c\n>> Signed-off-by: Junio C Hamano \n>> Signed-off-by: Stefan Beller \n>> ---\n>\n> Hmph, what are these final lines about?\n\nSee the explanation in the patch 16/17.\nMy guess is that one of us (me) was careless again.\n\nThe commented lines are easily produced by git-gui, that\nI use. I think it just takes the commit message from the underlying\ngit-core. But unlike git-core, it doesn't strip off commented lines\nas there is no extra information in comments presented.\nI wonder how you are the first signoff after the conflict markers\nthough, as that would hint that you signed off a commit message\nwith the commented conflict lines first, before I had them.\n\nPuzzeled, will fix in a reroll.\n\nThanks,\nStefan\n"},{"id":"320701","messageId":"xmqq7f15e8pu.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170524214036.29623-1-sbeller@google.com","subject":"Re: [PATCHv5 00/17] Diff machine: highlight moved lines.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-25T06:44:45Z","receivedAt":"2017-05-25T06:44:53Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"I was trying to see how this is useful in code moving change, with\nthis command line:\n\n$ git -c color.moved diff js/blame-lib~6 js/blame-lib blame.c blame.h builtin/blame.c\n\nSome random observations:\n\n * You do not seem to have any command line option yet. I guess\n that is OK while the series is still in RFC state, but when we\n are ready to seriously start considering this for 'next', we'd\n need something like --color-moved that can be used across \"diff\",\n \"log -p\" and \"show\".\n\n * As configuration variable names go, \"color.moved\" is probably in\n a wrong section. Perhaps \"diff.colorMoved\" or something?\n\n * The fact that I am using\n\n [diff \"color\"] \n\told = red reverse\n\twhitespace = blue reverse\n\n on a \"black ink on white paper\" terminal might have an effect on\n this, but lines printed in either bold green and on green\n background (i.e. not new ones but are the ones moved from\n elsewhere) stood out a lot more prominently than lines printed in\n green (i.e. truly new additions), and I felt that this is totally\n backwards from what I needed for this exercise. That is, while\n reviewing a code moving change, I want to be able to concentrate\n primarily of three things:\n\n - What are the new lines that are *not* moved from elsewhere?\n Did the submitter try to sneak in unrelated changes?\n\n - What are the changes that are truly lost, not moved to\n elsewhere?\n\n - Do the lines moved from elsewhere form a coherent whole? Where\n is the boundary between blocks of text that are copied? Do\n adjacent two blocks of moved text come from the same old place\n next to each other?\n\n Using colors that stand out more prominently than for the regular\n new/old lines is counter-productive for all of these, especially\n for the first two purposes. I may suggest using cyan (or any\n color that is very close to the background) so that the presense\n of moved lines are merely felt without being distracting. IOW,\n while reviewing code moving patch, moved parts want to be dimmed,\n not highlighted.\n\n"},{"id":"320716","messageId":"CAGZ79kYO8DoTDCXCk1uSFSAJA5JusD-t4BZoQfGy5n9DF34EvA@mail.gmail.com","threadId":"46038","inReplyTo":"xmqq7f15e8pu.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCHv5 00/17] Diff machine: highlight moved lines.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-25T16:31:27Z","receivedAt":"2017-05-25T16:31:34Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, May 24, 2017 at 11:44 PM, Junio C Hamano wrote:\n> I was trying to see how this is useful in code moving change, with\n> this command line:\n>\n> $ git -c color.moved diff js/blame-lib~6 js/blame-lib blame.c blame.h builtin/blame.c\n>\n> Some random observations:\n>\n> * You do not seem to have any command line option yet. I guess\n> that is OK while the series is still in RFC state, but when we\n> are ready to seriously start considering this for 'next', we'd\n> need something like --color-moved that can be used across \"diff\",\n> \"log -p\" and \"show\".\n\nThere is \"--color-moved\" as a command line option. (See diff.c:4290)\nOh, it is not documented! That will be fixed.\n\n> * As configuration variable names go, \"color.moved\" is probably in\n> a wrong section. Perhaps \"diff.colorMoved\" or something?\n\nAs you turn on/off normal coloring via \"color.diff\" and this only extends\nthe coloring scheme, I have the impression \"color\" is the right section.\nMaybe color.diffStyle=[normal/enhanced] and \"enhanced\" means this\nfeature for now?\n\nThe only option in the \"diff\" section related to color is diff.wsErrorHighlight\nwhich has a very similar purpose, so \"diff.colorMoved\" would fit in that\nscheme.\n\n>\n> * The fact that I am using\n>\n> [diff \"color\"]\n> old = red reverse\n> whitespace = blue reverse\n>\n> on a \"black ink on white paper\" terminal might have an effect on\n> this,\n\nYes very much!\n\n> but lines printed in either bold green and on green\n> background (i.e. not new ones but are the ones moved from\n> elsewhere) stood out a lot more prominently than lines printed in\n> green (i.e. truly new additions), and I felt that this is totally\n> backwards from what I needed for this exercise. That is, while\n> reviewing a code moving change, I want to be able to concentrate\n> primarily of three things:\n>\n> - What are the new lines that are *not* moved from elsewhere?\n> Did the submitter try to sneak in unrelated changes?\n>\n> - What are the changes that are truly lost, not moved to\n> elsewhere?\n>\n> - Do the lines moved from elsewhere form a coherent whole? Where\n> is the boundary between blocks of text that are copied? Do\n> adjacent two blocks of moved text come from the same old place\n> next to each other?\n\nSo with these questions, I wonder if we want to color moved lines\nas \"color.diff.context\" (i.e. plain white text in the normal coloring scheme)\nThis would serve the intended purpose of\ndimming the attention to moved lines.\n\nRegarding the last point of marking up adjacent blocks (which would\nindicate that there is a coherency issue or just moving from different\nplaces), we could highlight the last line of the previous block\nand first line of the next block in their \"normal\" colors (i.e.\ncolor.diff.old and color.diff.new).\n\nThe very first version had some boundary coloring, but then\nI switched to alternating block coloring based on an idea\nby Jonathan Tan.\n\nMaybe it is time to go back to boundary coloring, but optional\nand apply the boundary color only if two blocks are adjacent?\n\nExample of how it could look:\n---8<---\n diff --git a/poetry.txt b/poetry.txt\n index 9d32b3b..cc50ca1 100644\n --- a/poetry.txt\n +++ b/poetry.txt\n @@ -1,12 +1,4 @@\n[W] -A simple text is\n[W] -written in paragraphs and\n[W] -many more lines.\n[R] -\n[W] A diff is smaller\n[W] than the pre or post image text form\n[W] used for review\n[W]\n[W] -In between focus\n[W] -is hard to keep consistently\n[W] -errors may occur\n[W] -\n diff --git a/engineer.txt b/engineer.txt\n new file mode 100644\n index 0000000..8fbc0ce\n --- /dev/null\n +++ b/engineer.txt\n @@ -0,0 +1,7 @@\n[W] +A simple text is\n[W] +written in paragraphs and\n[B] +many more lines.\n[B] +In between focus\n[W] +is hard to keep consistently\n[W] +errors may occur\n[W] +\n---8<---\n\nW -> simple White text (diff.color.context)\nR -> Red text (diff.color.old)\nB -> Boundary marker (Maybe just diff.color.{old/new}\n or a new color to configure)\n\n>\n> Using colors that stand out more prominently than for the regular\n> new/old lines is counter-productive for all of these, especially\n> for the first two purposes. I may suggest using cyan (or any\n> color that is very close to the background) so that the presense\n> of moved lines are merely felt without being distracting. IOW,\n> while reviewing code moving patch, moved parts want to be dimmed,\n> not highlighted.\n\nI agree. So I could resend the algorithm used with other defaults\nor try out the \"boundary only iff adjacent blocks, else context color\".\n\nThanks,\nStefan\n"},{"id":"320780","messageId":"xmqqd1awctkg.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"CAGZ79kZM5KPP7iDo7rT5gxsooRrftJrA=+3-WBXW3nqfXeTy4Q@mail.gmail.com","subject":"Re: [PATCHv5 16/17] diff: buffer all output if asked to","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-26T01:09:35Z","receivedAt":"2017-05-26T01:09:53Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Yes, this is essentially the v4 with small indentation fixes, so I\n> assumed your signoff is still valid.\n\nOK. I just didn't expect to see one without \"no changes since v4\"\nunder the three-dash line.\n\nThanks.\n"},{"id":"320781","messageId":"xmqq8tlkct27.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"CAGZ79kYO8DoTDCXCk1uSFSAJA5JusD-t4BZoQfGy5n9DF34EvA@mail.gmail.com","subject":"Re: [PATCHv5 00/17] Diff machine: highlight moved lines.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-26T01:20:32Z","receivedAt":"2017-05-26T01:20:40Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> As you turn on/off normal coloring via \"color.diff\" and this only extends\n> the coloring scheme, I have the impression \"color\" is the right section.\n> Maybe color.diffStyle=[normal/enhanced] and \"enhanced\" means this\n> feature for now?\n\nHmph, I thought the intent of color.diff is \"is the diff command\nitself is colored?\" In other words, color.diff=false should give\nyou monochrome if you say \"diff --word-diff\", etc.\n\n> The only option in the \"diff\" section related to color is diff.wsErrorHighlight\n> which has a very similar purpose, so \"diff.colorMoved\" would fit in that\n> scheme.\n\nI didn't have \"should diff output highlight whitespace errors?\" in\nmind when I wrote the message you are responding to, but yes, that\nis quite similar to \"should diff output show lines moved and lines\ndeleted/added differently?\".\n\n> So with these questions, I wonder if we want to color moved lines\n> as \"color.diff.context\" (i.e. plain white text in the normal coloring scheme)\n> This would serve the intended purpose of\n> dimming the attention to moved lines.\n\nYes, but two points.\n\n (1) We want to do so while making it obvious where the boundary\n between two moved blocks of text whose destination (for\n moved-deleted lines) or source (for moved-added lines) is.\n\n (2) My message was an impression from using the code to review a\n patch that is meant to be \"move without changing other things\".\n For other purposes, there may be cases where moved ones may\n want to be highlighted, not dimmed.\n\n> Regarding the last point of marking up adjacent blocks (which would\n> indicate that there is a coherency issue or just moving from different\n> places), we could highlight the last line of the previous block\n> and first line of the next block in their \"normal\" colors (i.e.\n> color.diff.old and color.diff.new).\n\nHmm. That is an interesting thought.\n"},{"id":"320848","messageId":"CAGZ79kbD6QggYH9+M8FM0Khg7Cs0egJR6Jm+kMJkscaANV6UXA@mail.gmail.com","threadId":"46038","inReplyTo":"xmqq8tlkct27.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCHv5 00/17] Diff machine: highlight moved lines.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-26T19:30:05Z","receivedAt":"2017-05-26T19:30:12Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, May 25, 2017 at 6:20 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> As you turn on/off normal coloring via \"color.diff\" and this only extends\n>> the coloring scheme, I have the impression \"color\" is the right section.\n>> Maybe color.diffStyle=[normal/enhanced] and \"enhanced\" means this\n>> feature for now?\n>\n> Hmph, I thought the intent of color.diff is \"is the diff command\n> itself is colored?\" In other words, color.diff=false should give\n> you monochrome if you say \"diff --word-diff\", etc.\n\nYes, but in my understanding the \"diff\" doesn't apply to\nthe command, but the part of the output. I arrived at that\nunderstanding as other commands (show/log/..) also respect\nthat setting, so the \"diff\" in color.diff is not the command, but\nreferring to something else, the output being the closest match. :)\n\nAnd by that understanding color.diffStyle is just natural?\n\nBut I think that setting would be a bad idea as we'd rather have\nmultiple uncorrelated settings for coloring, which a style argument\nwould not.\n\n>> The only option in the \"diff\" section related to color is diff.wsErrorHighlight\n>> which has a very similar purpose, so \"diff.colorMoved\" would fit in that\n>> scheme.\n\nWith the above reasoning, this may be missnamed and should rather be\ncolor.wsErrorHighlight as it applies to more than just the diff command.\n\nNote: The average user may not aware that diff/show/log are tiny wrappers\naround the same backend for the heavy lifting.\n\n> I didn't have \"should diff output highlight whitespace errors?\" in\n> mind when I wrote the message you are responding to, but yes, that\n> is quite similar to \"should diff output show lines moved and lines\n> deleted/added differently?\".\n>\n>> So with these questions, I wonder if we want to color moved lines\n>> as \"color.diff.context\" (i.e. plain white text in the normal coloring scheme)\n>> This would serve the intended purpose of\n>> dimming the attention to moved lines.\n>\n> Yes, but two points.\n>\n> (1) We want to do so while making it obvious where the boundary\n> between two moved blocks of text whose destination (for\n> moved-deleted lines) or source (for moved-added lines) is.\n\nYes, that is what the boundary color would accomplish.\nAny two adjacent blocks with the same sign would have\ntheir boundary line colored this way.\n\n> (2) My message was an impression from using the code to review a\n> patch that is meant to be \"move without changing other things\".\n\nOk, glad you found it somewhat useful already.\n\n> For other purposes, there may be cases where moved ones may\n> want to be highlighted, not dimmed.\n\nI wonder what these use cases would be?\n(barring a --find-copies harder extension that would not just search the\ncurrent diff, but the whole tree)\n\nThat hints at just having an extra slot for the moved block, but the default\ncolor could be the same as color.diff.context for dimming.\n\nBy now I also think we may not need different colors for additions\nor removals, but keeping two colors is easy enough.\n\n>> Regarding the last point of marking up adjacent blocks (which would\n>> indicate that there is a coherency issue or just moving from different\n>> places), we could highlight the last line of the previous block\n>> and first line of the next block in their \"normal\" colors (i.e.\n>> color.diff.old and color.diff.new).\n>\n> Hmm. That is an interesting thought.\n\nI'll try the implementation for that and see if it looks good.\n\nThanks,\nStefan\n"},{"id":"320851","messageId":"20170527001820.25214-1-sbeller@google.com","threadId":"46038","inReplyTo":"CAGZ79kbD6QggYH9+M8FM0Khg7Cs0egJR6Jm+kMJkscaANV6UXA@mail.gmail.com","subject":"[PATCH 0/1] Diff machine: highlight moved lines.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-27T00:18:19Z","receivedAt":"2017-05-27T01:12:10Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This is a new version of the latest patch of sb/diff-color-move.\n\nIt seems as if different people prefer different settings for the\nboundary/alternate coloring, so let's have all of them. (As the\nblock detection is rather simple we do not need a lot of overhead\nto support multiple modes).\n\nWith more tests:\n * for each mode;\n * check that diff.colorMoved is respected\n \nAnd it is documented as well.\n\nThanks,\nStefan\n\nStefan Beller (1):\n diff.c: color moved lines differently\n\n Documentation/config.txt | 10 +-\n Documentation/diff-options.txt | 32 ++++\n color.h | 2 +\n diff.c | 342 +++++++++++++++++++++++++++++++++++--\n diff.h | 15 +-\n t/t4015-diff-whitespace.sh | 373 +++++++++++++++++++++++++++++++++++++++++\n 6 files changed, 760 insertions(+), 14 deletions(-)\n\nInterdiff to what Junio has queued:\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 902d017c3b..73511a4603 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1051,12 +1051,9 @@ This does not affect linkgit:git-format-patch[1] or the\n 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n command line with the `--color[=]` option.\n \n-color.moved::\n-\tA boolean value, whether a diff should color moved lines\n-\tdifferently. The moved lines are searched for in the diff only.\n-\tDuplicated lines from somewhere in the project that are not\n-\tpart of the diff are not colored as moved.\n-\tDefaults to false.\n+diff.colorMoved::\n+\tIf set moved lines in a diff are colored differently,\n+\tfor details see '--color-moved' in linkgit:git-diff[1].\n \n color.diff.::\n \tUse customized color for diff colorization. `` specifies\n@@ -1065,10 +1062,9 @@ color.diff.::\n \t`meta` (metainformation), `frag`\n \t(hunk header), 'func' (function in hunk header), `old` (removed lines),\n \t`new` (added lines), `commit` (commit headers), `whitespace`\n-\t(highlighting whitespace errors), `oldMoved` (removed lines that\n-\treappear), `newMoved` (added lines that were removed elsewhere),\n-\t`oldMovedAlternative` and `newMovedAlternative` (as a fallback to\n-\tcover adjacent blocks of moved code)\n+\t(highlighting whitespace errors), `oldMoved`, `newMoved`,\n+\t`oldMovedAlternative` and `newMovedAlternative` (See the ''\n+\tsetting of '--color-moved' in linkgit:git-diff[1] for details).\n \n color.decorate.::\n \tUse customized color for 'git log --decorate' output. `` is one\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex 89cc0f48de..25259dbbc3 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -231,6 +231,38 @@ ifdef::git-diff[]\n endif::git-diff[]\n \tIt is the same as `--color=never`.\n \n+--color-moved[=]::\n+\tMoved lines of code are colored differently.\n+ifdef::git-diff[]\n+\tIt can be changed by the `diff.colorMoved` configuration setting.\n+endif::git-diff[]\n+\tThe defaults to 'no' if the option is not given\n+\tand to 'adjacentbounds' if the option with no mode is given.\n+\tThe mode must be one of:\n++\n+--\n+no::\n+\tMoved lines are not highlighted.\n+nobounds::\n+\tAny line that is added in on location and was removed\n+\tin another location will be colored with 'color.diff.newmoved'.\n+\tAny line that is removed in on location and was added\n+\tin another location will be colored with 'color.diff.oldmoved'.\n+allbounds::\n+\tBased on 'nobounds'. Additionally blocks of moved code are\n+\tdetected and the first and last line of a block will be highlighted\n+\tusing 'color.diff.newMovedAlternate' or\n+\t'color.diff.oldMovedAlternate'.\n+adjacentbounds::\n+\tThe same as 'allbounds' except that highlighting is only performed\n+\tat adjacent block boundaries of blocks that have the same sign.\n+alternate::\n+\tBased on 'nobounds'. Additionally blocks of moved code are\n+\tdetected. If moved blocks are adjacent mark one of them with the\n+\talternative move color using 'color.diff.newMovedAlternate' or\n+\t'color.diff.oldMovedAlternate'.\n+--\n+\n --word-diff[=]::\n \tShow a word diff, using the to delimit changed words.\n \tBy default, words are delimited by whitespace; see\ndiff --git a/color.h b/color.h\nindex 90627650fc..04b3b87929 100644\n--- a/color.h\n+++ b/color.h\n@@ -42,6 +42,8 @@ struct strbuf;\n #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n+#define GIT_COLOR_DI_IT_CYAN\t\"\\033[2;3;36m\"\n+#define GIT_COLOR_DI_IT_MAGENTA\t\"\\033[2;3;35m\"\n \n /* A special value meaning \"no color selected\" */\n #define GIT_COLOR_NIL \"NIL\"\ndiff --git a/diff.c b/diff.c\nindex 40cccafb67..efd2530a89 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -56,10 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n-\tGIT_COLOR_BOLD_RED,\t/* OLD_MOVED_A */\n-\tGIT_COLOR_BG_RED,\t/* OLD_MOVED_B */\n-\tGIT_COLOR_BOLD_GREEN,\t/* NEW_MOVED_A */\n-\tGIT_COLOR_BG_GREEN,\t/* NEW_MOVED_B */\n+\tGIT_COLOR_DI_IT_MAGENTA,/* OLD_MOVED */\n+\tGIT_COLOR_BG_RED,\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_DI_IT_CYAN,\t/* NEW_MOVED */\n+\tGIT_COLOR_BG_GREEN,\t/* NEW_MOVED ALTERNATIVE */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -241,14 +241,33 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)\n \treturn 0;\n }\n \n+static int parse_color_moved(const char *arg)\n+{\n+\tif (!strcmp(arg, \"no\"))\n+\t\treturn MOVED_LINES_NO;\n+\telse if (!strcmp(arg, \"nobounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_NO;\n+\telse if (!strcmp(arg, \"allbounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_ALL;\n+\telse if (!strcmp(arg, \"adjacentbounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_ADJACENT;\n+\telse if (!strcmp(arg, \"alternate\"))\n+\t\treturn MOVED_LINES_ALTERNATE;\n+\telse\n+\t\treturn -1;\n+}\n+\n int git_diff_ui_config(const char *var, const char *value, void *cb)\n {\n \tif (!strcmp(var, \"diff.color\") || !strcmp(var, \"color.diff\")) {\n \t\tdiff_use_color_default = git_config_colorbool(var, value);\n \t\treturn 0;\n \t}\n-\tif (!strcmp(var, \"color.moved\")) {\n-\t\tdiff_color_moved_default = git_config_bool(var, value);\n+\tif (!strcmp(var, \"diff.colormoved\")) {\n+\t\tint cm = parse_color_moved(value);\n+\t\tif (cm < 0)\n+\t\t\treturn -1;\n+\t\tdiff_color_moved_default = cm;\n \t\treturn 0;\n \t}\n \tif (!strcmp(var, \"diff.context\")) {\n@@ -652,41 +671,69 @@ static void add_lines_to_move_detection(struct diff_options *o,\n \t}\n }\n \n+static void mark_color_as_moved_single_line(struct diff_options *o,\n+\t\t\t\t\t struct diff_line *l, int alt_color)\n+{\n+\tswitch (l->sign) {\n+\tcase '+':\n+\t\tl->set = diff_get_color_opt(o,\n+\t\t\tDIFF_FILE_NEW_MOVED + alt_color);\n+\t\tbreak;\n+\tcase '-':\n+\t\tl->set = diff_get_color_opt(o,\n+\t\t\tDIFF_FILE_OLD_MOVED + alt_color);\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: we should have continued earlier?\");\n+\t}\n+}\n+\n static void mark_color_as_moved(struct diff_options *o,\n \t\t\t\tstruct hashmap *add_lines,\n \t\t\t\tstruct hashmap *del_lines)\n {\n \tstruct moved_entry **pmb = NULL; /* potentially moved blocks */\n+\tstruct diff_line *prev_line = NULL;\n \tint pmb_nr = 0, pmb_alloc = 0;\n-\tint use_alt_color = 0;\n-\tint n;\n+\tint n, flipped_block = 0;\n \n \tfor (n = 0; n < o->line_buffer_nr; n++) {\n \t\tstruct hashmap *hm = NULL;\n \t\tstruct moved_entry *key;\n \t\tstruct moved_entry *match = NULL;\n \t\tstruct diff_line *l = &o->line_buffer[n];\n-\t\tint i, lp, rp;\n+\t\tint i, lp, rp, adjacent_blocks = 0;\n \n+\t\t/* Check for any match to color it as a move. */\n \t\tswitch (l->sign) {\n \t\tcase '+':\n \t\t\thm = del_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n \t\t\tbreak;\n \t\tcase '-':\n \t\t\thm = add_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n \t\t\tbreak;\n-\t\tdefault:\n-\t\t\tuse_alt_color = 0;\n-\t\t\tpmb_nr = 0; /* no running sets */\n+\t\tdefault: ;\n+\t\t}\n+\n+\t\tif (!match) {\n+\t\t\tpmb_nr = 0;\n+\t\t\tif (prev_line &&\n+\t\t\t o->color_moved == MOVED_LINES_BOUNDARY_ALL)\n+\t\t\t\tmark_color_as_moved_single_line(o, prev_line, 1);\n+\t\t\tprev_line = NULL;\n \t\t\tcontinue;\n \t\t}\n \n-\t\t/* Check for any match to color it as a move. */\n-\t\tkey = prepare_entry(o, n);\n-\t\tmatch = hashmap_get(hm, key, o);\n-\t\tfree(key);\n-\t\tif (!match)\n+\t\tif (o->color_moved == MOVED_LINES_BOUNDARY_NO) {\n+\t\t\tmark_color_as_moved_single_line(o, l, 0);\n \t\t\tcontinue;\n+\t\t}\n \n \t\t/* Check any potential block runs, advance each or nullify */\n \t\tfor (i = 0; i < pmb_nr; i++) {\n@@ -701,7 +748,7 @@ static void mark_color_as_moved(struct diff_options *o,\n \t\t\t}\n \t\t}\n \n-\t\t/* Shrink the set to the remaining runs */\n+\t\t/* Shrink the set of potential block to the remaining running */\n \t\tfor (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n \t\t\twhile (lp < pmb_nr && pmb[lp])\n \t\t\t\tlp++;\n@@ -719,34 +766,44 @@ static void mark_color_as_moved(struct diff_options *o,\n \t\t\t}\n \t\t}\n \n-\t\tif (rp > -1) {\n-\t\t\t/* Remember the number of running sets */\n-\t\t\tpmb_nr = rp + 1;\n-\t\t} else {\n-\t\t\t/* Toggle color */\n-\t\t\tuse_alt_color = (use_alt_color + 1) % 2;\n+\t\t/* Remember the number of running sets */\n+\t\tpmb_nr = rp + 1;\n \n-\t\t\t/* Build up a new set */\n-\t\t\tpmb_nr = 0;\n+\t\tif (pmb_nr == 0) {\n+\t\t\t/*\n+\t\t\t * This line is the start of a new block.\n+\t\t\t * Setup the set of potential blocks.\n+\t\t\t */\n \t\t\tfor (; match; match = hashmap_get_next(hm, match)) {\n \t\t\t\tALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n \t\t\t\tpmb[pmb_nr++] = match;\n \t\t\t}\n+\n+\t\t\tif (o->color_moved == MOVED_LINES_BOUNDARY_ALL) {\n+\t\t\t\tadjacent_blocks = 1;\n+\t\t\t} else {\n+\t\t\t\t/* Check if two blocks are adjacent */\n+\t\t\t\tadjacent_blocks = prev_line &&\n+\t\t\t\t\t\t prev_line->sign == l->sign;\n+\t\t\t}\n \t\t}\n \n-\t\tswitch (l->sign) {\n-\t\tcase '+':\n-\t\t\tl->set = diff_get_color_opt(o,\n-\t\t\t\tDIFF_FILE_NEW_MOVED + use_alt_color);\n-\t\t\tbreak;\n-\t\tcase '-':\n-\t\t\tl->set = diff_get_color_opt(o,\n-\t\t\t\tDIFF_FILE_OLD_MOVED + use_alt_color);\n-\t\t\tbreak;\n-\t\tdefault:\n-\t\t\tdie(\"BUG: we should have continued earlier?\");\n+\t\tif (o->color_moved == MOVED_LINES_ALTERNATE) {\n+\t\t\tif (adjacent_blocks)\n+\t\t\t\tflipped_block = (flipped_block + 1) % 2;\n+\t\t\tmark_color_as_moved_single_line(o, l, flipped_block);\n+\t\t} else {\n+\t\t\t/* MOVED_LINES_BOUNDARY_{ADJACENT, ALL} */\n+\t\t\tmark_color_as_moved_single_line(o, l, adjacent_blocks);\n+\t\t\tif (adjacent_blocks && prev_line)\n+\t\t\t\tprev_line->set = l->set;\n \t\t}\n+\n+\t\tprev_line = l;\n \t}\n+\tif (prev_line && o->color_moved == MOVED_LINES_BOUNDARY_ALL)\n+\t\tmark_color_as_moved_single_line(o, prev_line, 1);\n+\n \tfree(pmb);\n }\n \n@@ -4291,10 +4348,18 @@ int diff_opt_parse(struct diff_options *options,\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n \telse if (!strcmp(arg, \"--color-moved\"))\n-\t\toptions->color_moved = 1;\n+\t\tif (diff_color_moved_default)\n+\t\t\toptions->color_moved = diff_color_moved_default;\n+\t\telse\n+\t\t\toptions->color_moved = MOVED_LINES_BOUNDARY_ADJACENT;\n \telse if (!strcmp(arg, \"--no-color-moved\"))\n-\t\toptions->color_moved = 0;\n-\telse if (!strcmp(arg, \"--color-words\")) {\n+\t\toptions->color_moved = MOVED_LINES_NO;\n+\telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n+\t\tint cm = parse_color_moved(arg);\n+\t\tif (cm < 0)\n+\t\t\tdie(\"bad --color-moved argument: %s\", arg);\n+\t\toptions->color_moved = cm;\n+\t} else if (!strcmp(arg, \"--color-words\")) {\n \t\toptions->use_color = 1;\n \t\toptions->word_diff = DIFF_WORDS_COLOR;\n \t}\ndiff --git a/diff.h b/diff.h\nindex c6d806c6b5..d9fbafd383 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -230,7 +230,13 @@ struct diff_options {\n \tstruct diff_line *line_buffer;\n \tint line_buffer_nr, line_buffer_alloc;\n \n-\tint color_moved;\n+\tenum {\n+\t\tMOVED_LINES_NO = 0,\n+\t\tMOVED_LINES_BOUNDARY_NO = 1,\n+\t\tMOVED_LINES_BOUNDARY_ALL = 2,\n+\t\tMOVED_LINES_BOUNDARY_ADJACENT = 3,\n+\t\tMOVED_LINES_ALTERNATE = 4,\n+\t} color_moved;\n };\n \n /* Emit [line_prefix] [set] line [reset] */\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 0e92bf94bf..d4bd082af7 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -984,6 +984,8 @@ test_expect_success 'detect moved code, complete file' '\n \tgit add test.c &&\n \tgit commit -m \"add main function\" &&\n \tgit mv test.c main.c &&\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n \tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n \tcat >expected <<-\\EOF &&\n \tdiff --git a/main.c b/main.c\n@@ -1081,6 +1083,10 @@ test_expect_success 'detect moved code, inside file' '\n \t\t\tbar();\n \t\t}\n \tEOF\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"bold red\" &&\n+\ttest_config color.diff.newMovedAlternative \"bold green\" &&\n \tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n \tcat <<-\\EOF >expected &&\n \tdiff --git a/main.c b/main.c\n@@ -1125,79 +1131,179 @@ test_expect_success 'detect moved code, inside file' '\n '\n \n test_expect_success 'detect permutations inside moved code' '\n-\t# reusing the move example from last test:\n-\tcat <<-\\EOF >main.c &&\n-\t\t#include\n-\t\tint stuff()\n-\t\t{\n-\t\t\tprintf(\"Hello \");\n-\t\t\tprintf(\"World\\n\");\n-\t\t}\n-\n-\t\tint main()\n-\t\t{\n-\t\t\tfoo();\n-\t\t}\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n \tEOF\n-\tcat <<-\\EOF >test.c &&\n-\t\t#include\n-\t\tint bar()\n-\t\t{\n-\t\t\tprintf(\"Hello World, but different\\n\");\n-\t\t}\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\tEOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n \n-\t\tint secure_foo(struct user *u)\n-\t\t{\n-\t\t\tfoo(u);\n-\t\t\tif (!u->is_allowed_foo)\n-\t\t\t\treturn;\n-\t\t}\n \n-\t\tint another_function()\n-\t\t{\n-\t\t\tbar();\n-\t\t}\n+\tgit diff HEAD --no-renames --color-moved=nobounds| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\t\tdiff --git a/lines.txt b/lines.txt\n+\t\tindex 47ea9c3..ba96a38 100644\n+\t\t--- a/lines.txt\n+\t\t+++ b/lines.txt\n+\t\t@@ -1,16 +1,16 @@\n+\t\t-line 1\n+\t\t-line 2\n+\t\t-line 3\n+\t\t line 4\n+\t\t line 5\n+\t\t line 6\n+\t\t line 7\n+\t\t line 8\n+\t\t line 9\n+\t\t+line 1\n+\t\t+line 2\n+\t\t+line 3\n+\t\t+line 14\n+\t\t+line 15\n+\t\t+line 16\n+\t\t line 10\n+\t\t line 11\n+\t\t line 12\n+\t\t line 13\n+\t\t-line 14\n+\t\t-line 15\n+\t\t-line 16\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\tgit diff HEAD --no-renames --color-moved=adjacentbounds| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n \tEOF\n+\ttest_cmp expected actual &&\n+\n+\ttest_config diff.colorMoved alternate &&\n \tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n \tcat <<-\\EOF >expected &&\n-\tdiff --git a/main.c b/main.c\n-\tindex 27a619c..7cf9336 100644\n-\t--- a/main.c\n-\t+++ b/main.c\n-\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n-\t printf(\"World\\n\");\n-\t }\n-\t \n-\t-int secure_foo(struct user *u)\n-\t-{\n-\t-if (!u->is_allowed_foo)\n-\t-return;\n-\t-foo(u);\n-\t-}\n-\t-\n-\t int main()\n-\t {\n-\t foo();\n-\tdiff --git a/test.c b/test.c\n-\tindex 1dc1d85..2bedec9 100644\n-\t--- a/test.c\n-\t+++ b/test.c\n-\t@@ -4,6 +4,13 @@ int bar()\n-\t printf(\"Hello World, but different\\n\");\n-\t }\n-\t \n-\t+int secure_foo(struct user *u)\n-\t+{\n-\t+foo(u);\n-\t+if (!u->is_allowed_foo)\n-\t+return;\n-\t+}\n-\t+\n-\t int another_function()\n-\t {\n-\t bar();\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n \tEOF\n+\ttest_cmp expected actual &&\n \n+\ttest_config diff.colorMoved allbounds &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n \ttest_cmp expected actual\n '\n \n\n\n\n-- \n2.13.0.17.gab62347cd9\n\n"},{"id":"320854","messageId":"20170527001820.25214-2-sbeller@google.com","threadId":"46038","inReplyTo":"20170527001820.25214-1-sbeller@google.com","subject":"[PATCH 1/1] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-27T00:18:20Z","receivedAt":"2017-05-27T01:48:34Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a patch consists mostly of moving blocks of code around, it can\nbe quite tedious to ensure that the blocks are moved verbatim, and not\nundesirably modified in the move. To that end, color blocks that are\nmoved within the same patch differently. For example (OM, del, add,\nand NM are different colors):\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OM] - if (!is_authorized_user())\n [OM] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OM] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NM] + sensitive_stuff(spanning,\n [NM] + multiple,\n [NM] + lines);\n [NM] +}\n\nHowever adjacent blocks may be problematic. For example, in this\npotentially malicious patch, the swapping of blocks can be spotted:\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OMA] - if (!is_authorized_user())\n [OMA] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OMA] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NMA] + sensitive_stuff(spanning,\n [NMA] + multiple,\n [NMA] + lines);\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NMA] +}\n\nIf the moved code is larger, it is easier to hide some permutation in the\ncode, which is why some alternative coloring is needed.\n\nAs the reviewers attention should be brought to the places, where the\ndifference is introduced to the moved code, we cannot just have one new\ncolor for all of moved code.\n\nFirst I implemented an alternative design, which would try to fingerprint\na line by its neighbors to detect if we are in a block or at the boundary.\nThis idea iss error prone as it inspected each line and its neighboring\nlines to determine if the line was (a) moved and (b) if was deep inside\na hunk by having matching neighboring lines. This is unreliable as the\nwe can construct hunks which have equal neighbors that just exceed the\nnumber of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter\nas a line, that is permutated to AXYZCXYZBXYZD..').\n\nInstead this provides a dynamic programming greedy algorithm that finds\nthe largest moved hunk and then has several modes on highlighting bounds.\n\nA note on the options '--submodule=diff' and '--color-words/--word-diff':\nIn the conversion to use emit_line in the prior patches both submodules\nas well as word diff output carefully chose to call emit_line with sign=0.\nAll output with sign=0 is ignored for move detection purposes in this\npatch, such that no weird looking output will be generated for these\ncases. This leads to another thought: We could pass on '--color-moved' to\nsubmodules such that they color up moved lines for themselves. If we'd do\nso only line moves within a repository boundary are marked up.\n\nHelped-by: Jonathan Tan \nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n Documentation/config.txt | 10 +-\n Documentation/diff-options.txt | 32 ++++\n color.h | 2 +\n diff.c | 342 +++++++++++++++++++++++++++++++++++--\n diff.h | 15 +-\n t/t4015-diff-whitespace.sh | 373 +++++++++++++++++++++++++++++++++++++++++\n 6 files changed, 760 insertions(+), 14 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 475e874d51..73511a4603 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1051,14 +1051,20 @@ This does not affect linkgit:git-format-patch[1] or the\n 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n command line with the `--color[=]` option.\n \n+diff.colorMoved::\n+\tIf set moved lines in a diff are colored differently,\n+\tfor details see '--color-moved' in linkgit:git-diff[1].\n+\n color.diff.::\n \tUse customized color for diff colorization. `` specifies\n \twhich part of the patch to use the specified color, and is one\n \tof `context` (context text - `plain` is a historical synonym),\n \t`meta` (metainformation), `frag`\n \t(hunk header), 'func' (function in hunk header), `old` (removed lines),\n-\t`new` (added lines), `commit` (commit headers), or `whitespace`\n-\t(highlighting whitespace errors).\n+\t`new` (added lines), `commit` (commit headers), `whitespace`\n+\t(highlighting whitespace errors), `oldMoved`, `newMoved`,\n+\t`oldMovedAlternative` and `newMovedAlternative` (See the ''\n+\tsetting of '--color-moved' in linkgit:git-diff[1] for details).\n \n color.decorate.::\n \tUse customized color for 'git log --decorate' output. `` is one\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex 89cc0f48de..25259dbbc3 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -231,6 +231,38 @@ ifdef::git-diff[]\n endif::git-diff[]\n \tIt is the same as `--color=never`.\n \n+--color-moved[=]::\n+\tMoved lines of code are colored differently.\n+ifdef::git-diff[]\n+\tIt can be changed by the `diff.colorMoved` configuration setting.\n+endif::git-diff[]\n+\tThe defaults to 'no' if the option is not given\n+\tand to 'adjacentbounds' if the option with no mode is given.\n+\tThe mode must be one of:\n++\n+--\n+no::\n+\tMoved lines are not highlighted.\n+nobounds::\n+\tAny line that is added in on location and was removed\n+\tin another location will be colored with 'color.diff.newmoved'.\n+\tAny line that is removed in on location and was added\n+\tin another location will be colored with 'color.diff.oldmoved'.\n+allbounds::\n+\tBased on 'nobounds'. Additionally blocks of moved code are\n+\tdetected and the first and last line of a block will be highlighted\n+\tusing 'color.diff.newMovedAlternate' or\n+\t'color.diff.oldMovedAlternate'.\n+adjacentbounds::\n+\tThe same as 'allbounds' except that highlighting is only performed\n+\tat adjacent block boundaries of blocks that have the same sign.\n+alternate::\n+\tBased on 'nobounds'. Additionally blocks of moved code are\n+\tdetected. If moved blocks are adjacent mark one of them with the\n+\talternative move color using 'color.diff.newMovedAlternate' or\n+\t'color.diff.oldMovedAlternate'.\n+--\n+\n --word-diff[=]::\n \tShow a word diff, using the to delimit changed words.\n \tBy default, words are delimited by whitespace; see\ndiff --git a/color.h b/color.h\nindex 90627650fc..04b3b87929 100644\n--- a/color.h\n+++ b/color.h\n@@ -42,6 +42,8 @@ struct strbuf;\n #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n+#define GIT_COLOR_DI_IT_CYAN\t\"\\033[2;3;36m\"\n+#define GIT_COLOR_DI_IT_MAGENTA\t\"\\033[2;3;35m\"\n \n /* A special value meaning \"no color selected\" */\n #define GIT_COLOR_NIL \"NIL\"\ndiff --git a/diff.c b/diff.c\nindex a3c16ef827..efd2530a89 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -31,6 +31,7 @@ static int diff_indent_heuristic; /* experimental */\n static int diff_rename_limit_default = 400;\n static int diff_suppress_blank_empty;\n static int diff_use_color_default = -1;\n+static int diff_color_moved_default;\n static int diff_context_default = 3;\n static int diff_interhunk_context_default;\n static const char *diff_word_regex_cfg;\n@@ -55,6 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n+\tGIT_COLOR_DI_IT_MAGENTA,/* OLD_MOVED */\n+\tGIT_COLOR_BG_RED,\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_DI_IT_CYAN,\t/* NEW_MOVED */\n+\tGIT_COLOR_BG_GREEN,\t/* NEW_MOVED ALTERNATIVE */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -80,6 +85,14 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_WHITESPACE;\n \tif (!strcasecmp(var, \"func\"))\n \t\treturn DIFF_FUNCINFO;\n+\tif (!strcasecmp(var, \"oldmoved\"))\n+\t\treturn DIFF_FILE_OLD_MOVED;\n+\tif (!strcasecmp(var, \"oldmovedalternative\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoved\"))\n+\t\treturn DIFF_FILE_NEW_MOVED;\n+\tif (!strcasecmp(var, \"newmovedalternative\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT;\n \treturn -1;\n }\n \n@@ -228,12 +241,35 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)\n \treturn 0;\n }\n \n+static int parse_color_moved(const char *arg)\n+{\n+\tif (!strcmp(arg, \"no\"))\n+\t\treturn MOVED_LINES_NO;\n+\telse if (!strcmp(arg, \"nobounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_NO;\n+\telse if (!strcmp(arg, \"allbounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_ALL;\n+\telse if (!strcmp(arg, \"adjacentbounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_ADJACENT;\n+\telse if (!strcmp(arg, \"alternate\"))\n+\t\treturn MOVED_LINES_ALTERNATE;\n+\telse\n+\t\treturn -1;\n+}\n+\n int git_diff_ui_config(const char *var, const char *value, void *cb)\n {\n \tif (!strcmp(var, \"diff.color\") || !strcmp(var, \"color.diff\")) {\n \t\tdiff_use_color_default = git_config_colorbool(var, value);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"diff.colormoved\")) {\n+\t\tint cm = parse_color_moved(value);\n+\t\tif (cm < 0)\n+\t\t\treturn -1;\n+\t\tdiff_color_moved_default = cm;\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"diff.context\")) {\n \t\tdiff_context_default = git_config_int(var, value);\n \t\tif (diff_context_default < 0)\n@@ -354,6 +390,88 @@ int git_diff_basic_config(const char *var, const char *value, void *cb)\n \treturn git_default_config(var, value, cb);\n }\n \n+struct moved_entry {\n+\tstruct hashmap_entry ent;\n+\tconst struct diff_line *line;\n+\tstruct moved_entry *next_line;\n+};\n+\n+static void get_ws_cleaned_string(const struct diff_line *l,\n+\t\t\t\t struct strbuf *out)\n+{\n+\tint i;\n+\tfor (i = 0; i < l->len; i++) {\n+\t\tif (isspace(l->line[i]))\n+\t\t\tcontinue;\n+\t\tstrbuf_addch(out, l->line[i]);\n+\t}\n+}\n+\n+static int diff_line_cmp_no_ws(const struct diff_line *a,\n+\t\t\t\t\t const struct diff_line *b,\n+\t\t\t\t\t const void *keydata)\n+{\n+\tint ret;\n+\tstruct strbuf sba = STRBUF_INIT;\n+\tstruct strbuf sbb = STRBUF_INIT;\n+\n+\tget_ws_cleaned_string(a, &sba);\n+\tget_ws_cleaned_string(b, &sbb);\n+\tret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);\n+\n+\tstrbuf_release(&sba);\n+\tstrbuf_release(&sbb);\n+\treturn ret;\n+}\n+\n+static int diff_line_cmp(const struct diff_line *a,\n+\t\t\t\t const struct diff_line *b,\n+\t\t\t\t const void *keydata)\n+{\n+\treturn a->len != b->len || strncmp(a->line, b->line, a->len);\n+}\n+\n+static int moved_entry_cmp(const struct moved_entry *a,\n+\t\t\t const struct moved_entry *b,\n+\t\t\t const void *keydata)\n+{\n+\treturn diff_line_cmp(a->line, b->line, keydata);\n+}\n+\n+static int moved_entry_cmp_no_ws(const struct moved_entry *a,\n+\t\t\t\t const struct moved_entry *b,\n+\t\t\t\t const void *keydata)\n+{\n+\treturn diff_line_cmp_no_ws(a->line, b->line, keydata);\n+}\n+\n+static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)\n+{\n+\tstatic struct strbuf sb = STRBUF_INIT;\n+\n+\tif (ignore_ws) {\n+\t\tstrbuf_reset(&sb);\n+\t\tget_ws_cleaned_string(line, &sb);\n+\t\treturn memhash(sb.buf, sb.len);\n+\t} else {\n+\t\treturn memhash(line->line, line->len);\n+\t}\n+}\n+\n+static struct moved_entry *prepare_entry(struct diff_options *o,\n+\t\t\t\t\t int line_no)\n+{\n+\tstruct moved_entry *ret = xmalloc(sizeof(*ret));\n+\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\tstruct diff_line *l = &o->line_buffer[line_no];\n+\n+\tret->ent.hash = get_line_hash(l, ignore_ws);\n+\tret->line = l;\n+\tret->next_line = NULL;\n+\n+\treturn ret;\n+}\n+\n static char *quote_two(const char *one, const char *two)\n {\n \tint need_one = quote_c_style(one, NULL, NULL, 1);\n@@ -516,6 +634,179 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,\n \tecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n }\n \n+static void add_lines_to_move_detection(struct diff_options *o,\n+\t\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry *prev_line = NULL;\n+\n+\tint n;\n+\tfor (n = 0; n < o->line_buffer_nr; n++) {\n+\t\tint sign = 0;\n+\t\tstruct hashmap *hm;\n+\t\tstruct moved_entry *key;\n+\n+\t\tswitch (o->line_buffer[n].sign) {\n+\t\tcase '+':\n+\t\t\tsign = '+';\n+\t\t\thm = add_lines;\n+\t\t\tbreak;\n+\t\tcase '-':\n+\t\t\tsign = '-';\n+\t\t\thm = del_lines;\n+\t\t\tbreak;\n+\t\tcase ' ':\n+\t\tdefault:\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tkey = prepare_entry(o, n);\n+\t\tif (prev_line &&\n+\t\t prev_line->line->sign == sign)\n+\t\t\tprev_line->next_line = key;\n+\n+\t\thashmap_add(hm, key);\n+\t\tprev_line = key;\n+\t}\n+}\n+\n+static void mark_color_as_moved_single_line(struct diff_options *o,\n+\t\t\t\t\t struct diff_line *l, int alt_color)\n+{\n+\tswitch (l->sign) {\n+\tcase '+':\n+\t\tl->set = diff_get_color_opt(o,\n+\t\t\tDIFF_FILE_NEW_MOVED + alt_color);\n+\t\tbreak;\n+\tcase '-':\n+\t\tl->set = diff_get_color_opt(o,\n+\t\t\tDIFF_FILE_OLD_MOVED + alt_color);\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: we should have continued earlier?\");\n+\t}\n+}\n+\n+static void mark_color_as_moved(struct diff_options *o,\n+\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry **pmb = NULL; /* potentially moved blocks */\n+\tstruct diff_line *prev_line = NULL;\n+\tint pmb_nr = 0, pmb_alloc = 0;\n+\tint n, flipped_block = 0;\n+\n+\tfor (n = 0; n < o->line_buffer_nr; n++) {\n+\t\tstruct hashmap *hm = NULL;\n+\t\tstruct moved_entry *key;\n+\t\tstruct moved_entry *match = NULL;\n+\t\tstruct diff_line *l = &o->line_buffer[n];\n+\t\tint i, lp, rp, adjacent_blocks = 0;\n+\n+\t\t/* Check for any match to color it as a move. */\n+\t\tswitch (l->sign) {\n+\t\tcase '+':\n+\t\t\thm = del_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tcase '-':\n+\t\t\thm = add_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tdefault: ;\n+\t\t}\n+\n+\t\tif (!match) {\n+\t\t\tpmb_nr = 0;\n+\t\t\tif (prev_line &&\n+\t\t\t o->color_moved == MOVED_LINES_BOUNDARY_ALL)\n+\t\t\t\tmark_color_as_moved_single_line(o, prev_line, 1);\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (o->color_moved == MOVED_LINES_BOUNDARY_NO) {\n+\t\t\tmark_color_as_moved_single_line(o, l, 0);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Check any potential block runs, advance each or nullify */\n+\t\tfor (i = 0; i < pmb_nr; i++) {\n+\t\t\tstruct moved_entry *p = pmb[i];\n+\t\t\tstruct moved_entry *pnext = (p && p->next_line) ?\n+\t\t\t\t\tp->next_line : NULL;\n+\t\t\tif (pnext &&\n+\t\t\t !diff_line_cmp(pnext->line, l, o)) {\n+\t\t\t\tpmb[i] = p->next_line;\n+\t\t\t} else {\n+\t\t\t\tpmb[i] = NULL;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Shrink the set of potential block to the remaining running */\n+\t\tfor (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n+\t\t\twhile (lp < pmb_nr && pmb[lp])\n+\t\t\t\tlp++;\n+\t\t\t/* lp points at the first NULL now */\n+\n+\t\t\twhile (rp > -1 && !pmb[rp])\n+\t\t\t\trp--;\n+\t\t\t/* rp points at the last non-NULL */\n+\n+\t\t\tif (lp < pmb_nr && rp > -1 && lp < rp) {\n+\t\t\t\tpmb[lp] = pmb[rp];\n+\t\t\t\tpmb[rp] = NULL;\n+\t\t\t\trp--;\n+\t\t\t\tlp++;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Remember the number of running sets */\n+\t\tpmb_nr = rp + 1;\n+\n+\t\tif (pmb_nr == 0) {\n+\t\t\t/*\n+\t\t\t * This line is the start of a new block.\n+\t\t\t * Setup the set of potential blocks.\n+\t\t\t */\n+\t\t\tfor (; match; match = hashmap_get_next(hm, match)) {\n+\t\t\t\tALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n+\t\t\t\tpmb[pmb_nr++] = match;\n+\t\t\t}\n+\n+\t\t\tif (o->color_moved == MOVED_LINES_BOUNDARY_ALL) {\n+\t\t\t\tadjacent_blocks = 1;\n+\t\t\t} else {\n+\t\t\t\t/* Check if two blocks are adjacent */\n+\t\t\t\tadjacent_blocks = prev_line &&\n+\t\t\t\t\t\t prev_line->sign == l->sign;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (o->color_moved == MOVED_LINES_ALTERNATE) {\n+\t\t\tif (adjacent_blocks)\n+\t\t\t\tflipped_block = (flipped_block + 1) % 2;\n+\t\t\tmark_color_as_moved_single_line(o, l, flipped_block);\n+\t\t} else {\n+\t\t\t/* MOVED_LINES_BOUNDARY_{ADJACENT, ALL} */\n+\t\t\tmark_color_as_moved_single_line(o, l, adjacent_blocks);\n+\t\t\tif (adjacent_blocks && prev_line)\n+\t\t\t\tprev_line->set = l->set;\n+\t\t}\n+\n+\t\tprev_line = l;\n+\t}\n+\tif (prev_line && o->color_moved == MOVED_LINES_BOUNDARY_ALL)\n+\t\tmark_color_as_moved_single_line(o, prev_line, 1);\n+\n+\tfree(pmb);\n+}\n+\n static void emit_diff_line(struct diff_options *o,\n \t\t\t struct diff_line *e)\n {\n@@ -3518,6 +3809,8 @@ void diff_setup(struct diff_options *options)\n \toptions->line_buffer = NULL;\n \toptions->line_buffer_nr = 0;\n \toptions->line_buffer_alloc = 0;\n+\n+\toptions->color_moved = diff_color_moved_default;\n }\n \n void diff_setup_done(struct diff_options *options)\n@@ -3627,6 +3920,9 @@ void diff_setup_done(struct diff_options *options)\n \n \tif (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)\n \t\tdie(_(\"--follow requires exactly one pathspec\"));\n+\n+\tif (!options->use_color || external_diff())\n+\t\toptions->color_moved = 0;\n }\n \n static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)\n@@ -4051,7 +4347,19 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n-\telse if (!strcmp(arg, \"--color-words\")) {\n+\telse if (!strcmp(arg, \"--color-moved\"))\n+\t\tif (diff_color_moved_default)\n+\t\t\toptions->color_moved = diff_color_moved_default;\n+\t\telse\n+\t\t\toptions->color_moved = MOVED_LINES_BOUNDARY_ADJACENT;\n+\telse if (!strcmp(arg, \"--no-color-moved\"))\n+\t\toptions->color_moved = MOVED_LINES_NO;\n+\telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n+\t\tint cm = parse_color_moved(arg);\n+\t\tif (cm < 0)\n+\t\t\tdie(\"bad --color-moved argument: %s\", arg);\n+\t\toptions->color_moved = cm;\n+\t} else if (!strcmp(arg, \"--color-words\")) {\n \t\toptions->use_color = 1;\n \t\toptions->word_diff = DIFF_WORDS_COLOR;\n \t}\n@@ -4856,16 +5164,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n-\t/*\n-\t * For testing purposes we want to make sure the diff machinery\n-\t * works completely with the buffer. If there is anything emitted\n-\t * outside the emit_diff_line, then the order is screwed\n-\t * up and the tests will fail.\n-\t *\n-\t * TODO (later in this series):\n-\t * We'll unset this flag in a later patch.\n-\t */\n-\to->use_buffer = 1;\n+\n+\tif (o->color_moved)\n+\t\to->use_buffer = 1;\n \n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n@@ -4874,6 +5175,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \t}\n \n \tif (o->use_buffer) {\n+\t\tif (o->color_moved) {\n+\t\t\tstruct hashmap add_lines, del_lines;\n+\t\t\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\n+\t\t\thashmap_init(&del_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\t\t\thashmap_init(&add_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\n+\t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n+\t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\n+\t\t\thashmap_free(&add_lines, 0);\n+\t\t\thashmap_free(&del_lines, 0);\n+\t\t}\n+\n \t\tfor (i = 0; i < o->line_buffer_nr; i++)\n \t\t\temit_diff_line(o, &o->line_buffer[i]);\n \n@@ -4962,6 +5281,7 @@ void diff_flush(struct diff_options *options)\n \t\tif (!options->file)\n \t\t\tdie_errno(\"Could not open /dev/null\");\n \t\toptions->close_file = 1;\n+\t\toptions->color_moved = 0;\n \t\tfor (i = 0; i < q->nr; i++) {\n \t\t\tstruct diff_filepair *p = q->queue[i];\n \t\t\tif (check_pair_status(p))\ndiff --git a/diff.h b/diff.h\nindex be51e8f867..d9fbafd383 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -7,6 +7,7 @@\n #include \"tree-walk.h\"\n #include \"pathspec.h\"\n #include \"object.h\"\n+#include \"hashmap.h\"\n \n struct rev_info;\n struct diff_options;\n@@ -228,6 +229,14 @@ struct diff_options {\n \n \tstruct diff_line *line_buffer;\n \tint line_buffer_nr, line_buffer_alloc;\n+\n+\tenum {\n+\t\tMOVED_LINES_NO = 0,\n+\t\tMOVED_LINES_BOUNDARY_NO = 1,\n+\t\tMOVED_LINES_BOUNDARY_ALL = 2,\n+\t\tMOVED_LINES_BOUNDARY_ADJACENT = 3,\n+\t\tMOVED_LINES_ALTERNATE = 4,\n+\t} color_moved;\n };\n \n /* Emit [line_prefix] [set] line [reset] */\n@@ -243,7 +252,11 @@ enum color_diff {\n \tDIFF_FILE_NEW = 5,\n \tDIFF_COMMIT = 6,\n \tDIFF_WHITESPACE = 7,\n-\tDIFF_FUNCINFO = 8\n+\tDIFF_FUNCINFO = 8,\n+\tDIFF_FILE_OLD_MOVED = 9,\n+\tDIFF_FILE_OLD_MOVED_ALT = 10,\n+\tDIFF_FILE_NEW_MOVED = 11,\n+\tDIFF_FILE_NEW_MOVED_ALT = 12\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 289806d0c7..d4bd082af7 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -972,4 +972,377 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '\n \n '\n \n+test_expect_success 'detect moved code, complete file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >test.c &&\n+\t#include\n+\tmain()\n+\t{\n+\tprintf(\"Hello World\");\n+\t}\n+\tEOF\n+\tgit add test.c &&\n+\tgit commit -m \"add main function\" &&\n+\tgit mv test.c main.c &&\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tdiff --git a/main.c b/main.c\n+\tnew file mode 100644\n+\tindex 0000000..a986c57\n+\t--- /dev/null\n+\t+++ b/main.c\n+\t@@ -0,0 +1,5 @@\n+\t+#include\n+\t+main()\n+\t+{\n+\t+printf(\"Hello World\");\n+\t+}\n+\tdiff --git a/test.c b/test.c\n+\tdeleted file mode 100644\n+\tindex a986c57..0000000\n+\t--- a/test.c\n+\t+++ /dev/null\n+\t@@ -1,5 +0,0 @@\n+\t-#include\n+\t-main()\n+\t-{\n+\t-printf(\"Hello World\");\n+\t-}\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect moved code, inside file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit add main.c test.c &&\n+\tgit commit -m \"add main and test file\" &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"bold red\" &&\n+\ttest_config color.diff.newMovedAlternative \"bold green\" &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..e34eb69 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+foo(u);\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect permutations inside moved code' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\tEOF\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\tEOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\n+\n+\tgit diff HEAD --no-renames --color-moved=nobounds| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\t\tdiff --git a/lines.txt b/lines.txt\n+\t\tindex 47ea9c3..ba96a38 100644\n+\t\t--- a/lines.txt\n+\t\t+++ b/lines.txt\n+\t\t@@ -1,16 +1,16 @@\n+\t\t-line 1\n+\t\t-line 2\n+\t\t-line 3\n+\t\t line 4\n+\t\t line 5\n+\t\t line 6\n+\t\t line 7\n+\t\t line 8\n+\t\t line 9\n+\t\t+line 1\n+\t\t+line 2\n+\t\t+line 3\n+\t\t+line 14\n+\t\t+line 15\n+\t\t+line 16\n+\t\t line 10\n+\t\t line 11\n+\t\t line 12\n+\t\t line 13\n+\t\t-line 14\n+\t\t-line 15\n+\t\t-line 16\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\tgit diff HEAD --no-renames --color-moved=adjacentbounds| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\ttest_config diff.colorMoved alternate &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\ttest_config diff.colorMoved allbounds &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'move detection does not mess up colored words' '\n+\tcat <<-\\EOF >text.txt &&\n+\tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n+\tEOF\n+\tgit add text.txt &&\n+\tgit commit -a -m \"clean state\" &&\n+\tcat <<-\\EOF >text.txt &&\n+\tsimply Lorem Ipsum dummy is text of the typesetting and printing industry.\n+\tEOF\n+\tgit diff --color-moved --word-diff >actual &&\n+\tgit diff --word-diff >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'move detection with submodules' '\n+\ttest_create_repo bananas &&\n+\techo ripe >bananas/recipe &&\n+\tgit -C bananas add recipe &&\n+\ttest_commit fruit &&\n+\ttest_commit -C bananas recipe &&\n+\tgit submodule add ./bananas &&\n+\tgit add bananas &&\n+\tgit commit -a -m \"bananas are like a heavy library?\" &&\n+\techo foul >bananas/recipe &&\n+\techo ripe >fruit.t &&\n+\n+\tgit diff --submodule=diff --color-moved >actual &&\n+\n+\t# no move detection as the moved line is across repository boundaries.\n+\ttest_decode_color decoded_actual &&\n+\t! grep BGREEN decoded_actual &&\n+\t! grep BRED decoded_actual &&\n+\n+\t# nor did we mess with it another way\n+\tgit diff --submodule=diff | test_decode_color >expect &&\n+\ttest_cmp expect decoded_actual\n+'\n+\n test_done\n-- \n2.13.0.17.gab62347cd9\n\n"},{"id":"320862","messageId":"39D67E58AC02490FB6D6124BDF3EE2A9@PhilipOakley","threadId":"46038","inReplyTo":"20170527001820.25214-2-sbeller@google.com","subject":"Re: [PATCH 1/1] diff.c: color moved lines differently","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":"2017-05-27T07:05:48Z","receivedAt":"2017-05-27T07:06:01Z","isPatch":true,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"a couple of mispellings in the doc parts:\n s/on location/one location/\n[code not checked]\n----- Original Message ----- \nFrom: \"Stefan Beller\" \nSubject: [PATCH 1/1] diff.c: color moved lines differently\n\n\n> When a patch consists mostly of moving blocks of code around, it can\n> be quite tedious to ensure that the blocks are moved verbatim, and not\n> undesirably modified in the move. To that end, color blocks that are\n> moved within the same patch differently. For example (OM, del, add,\n> and NM are different colors):\n>\n> [OM] -void sensitive_stuff(void)\n> [OM] -{\n> [OM] - if (!is_authorized_user())\n> [OM] - die(\"unauthorized\");\n> [OM] - sensitive_stuff(spanning,\n> [OM] - multiple,\n> [OM] - lines);\n> [OM] -}\n>\n> void another_function()\n> {\n> [del] - printf(\"foo\");\n> [add] + printf(\"bar\");\n> }\n>\n> [NM] +void sensitive_stuff(void)\n> [NM] +{\n> [NM] + if (!is_authorized_user())\n> [NM] + die(\"unauthorized\");\n> [NM] + sensitive_stuff(spanning,\n> [NM] + multiple,\n> [NM] + lines);\n> [NM] +}\n>\n> However adjacent blocks may be problematic. For example, in this\n> potentially malicious patch, the swapping of blocks can be spotted:\n>\n> [OM] -void sensitive_stuff(void)\n> [OM] -{\n> [OMA] - if (!is_authorized_user())\n> [OMA] - die(\"unauthorized\");\n> [OM] - sensitive_stuff(spanning,\n> [OM] - multiple,\n> [OM] - lines);\n> [OMA] -}\n>\n> void another_function()\n> {\n> [del] - printf(\"foo\");\n> [add] + printf(\"bar\");\n> }\n>\n> [NM] +void sensitive_stuff(void)\n> [NM] +{\n> [NMA] + sensitive_stuff(spanning,\n> [NMA] + multiple,\n> [NMA] + lines);\n> [NM] + if (!is_authorized_user())\n> [NM] + die(\"unauthorized\");\n> [NMA] +}\n>\n> If the moved code is larger, it is easier to hide some permutation in the\n> code, which is why some alternative coloring is needed.\n>\n> As the reviewers attention should be brought to the places, where the\n> difference is introduced to the moved code, we cannot just have one new\n> color for all of moved code.\n>\n> First I implemented an alternative design, which would try to fingerprint\n> a line by its neighbors to detect if we are in a block or at the boundary.\n> This idea iss error prone as it inspected each line and its neighboring\n> lines to determine if the line was (a) moved and (b) if was deep inside\n> a hunk by having matching neighboring lines. This is unreliable as the\n> we can construct hunks which have equal neighbors that just exceed the\n> number of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter\n> as a line, that is permutated to AXYZCXYZBXYZD..').\n>\n> Instead this provides a dynamic programming greedy algorithm that finds\n> the largest moved hunk and then has several modes on highlighting bounds.\n>\n> A note on the options '--submodule=diff' and '--color-words/--word-diff':\n> In the conversion to use emit_line in the prior patches both submodules\n> as well as word diff output carefully chose to call emit_line with sign=0.\n> All output with sign=0 is ignored for move detection purposes in this\n> patch, such that no weird looking output will be generated for these\n> cases. This leads to another thought: We could pass on '--color-moved' to\n> submodules such that they color up moved lines for themselves. If we'd do\n> so only line moves within a repository boundary are marked up.\n>\n> Helped-by: Jonathan Tan \n> Signed-off-by: Stefan Beller \n> Signed-off-by: Junio C Hamano \n> ---\n> Documentation/config.txt | 10 +-\n> Documentation/diff-options.txt | 32 ++++\n> color.h | 2 +\n> diff.c | 342 +++++++++++++++++++++++++++++++++++--\n> diff.h | 15 +-\n> t/t4015-diff-whitespace.sh | 373 \n> +++++++++++++++++++++++++++++++++++++++++\n> 6 files changed, 760 insertions(+), 14 deletions(-)\n>\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 475e874d51..73511a4603 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -1051,14 +1051,20 @@ This does not affect linkgit:git-format-patch[1] \n> or the\n> 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n> command line with the `--color[=]` option.\n>\n> +diff.colorMoved::\n> + If set moved lines in a diff are colored differently,\n> + for details see '--color-moved' in linkgit:git-diff[1].\n> +\n> color.diff.::\n> Use customized color for diff colorization. `` specifies\n> which part of the patch to use the specified color, and is one\n> of `context` (context text - `plain` is a historical synonym),\n> `meta` (metainformation), `frag`\n> (hunk header), 'func' (function in hunk header), `old` (removed lines),\n> - `new` (added lines), `commit` (commit headers), or `whitespace`\n> - (highlighting whitespace errors).\n> + `new` (added lines), `commit` (commit headers), `whitespace`\n> + (highlighting whitespace errors), `oldMoved`, `newMoved`,\n> + `oldMovedAlternative` and `newMovedAlternative` (See the ''\n> + setting of '--color-moved' in linkgit:git-diff[1] for details).\n>\n> color.decorate.::\n> Use customized color for 'git log --decorate' output. `` is one\n> diff --git a/Documentation/diff-options.txt \n> b/Documentation/diff-options.txt\n> index 89cc0f48de..25259dbbc3 100644\n> --- a/Documentation/diff-options.txt\n> +++ b/Documentation/diff-options.txt\n> @@ -231,6 +231,38 @@ ifdef::git-diff[]\n> endif::git-diff[]\n> It is the same as `--color=never`.\n>\n> +--color-moved[=]::\n> + Moved lines of code are colored differently.\n> +ifdef::git-diff[]\n> + It can be changed by the `diff.colorMoved` configuration setting.\n> +endif::git-diff[]\n> + The defaults to 'no' if the option is not given\n> + and to 'adjacentbounds' if the option with no mode is given.\n> + The mode must be one of:\n> ++\n> +--\n> +no::\n> + Moved lines are not highlighted.\n> +nobounds::\n> + Any line that is added in on location and was removed\ns/on location/one location/\n\n> + in another location will be colored with 'color.diff.newmoved'.\n> + Any line that is removed in on location and was added\ns/on location/one location/\n\n> + in another location will be colored with 'color.diff.oldmoved'.\n> +allbounds::\n> + Based on 'nobounds'. Additionally blocks of moved code are\n> + detected and the first and last line of a block will be highlighted\n> + using 'color.diff.newMovedAlternate' or\n> + 'color.diff.oldMovedAlternate'.\n> +adjacentbounds::\n> + The same as 'allbounds' except that highlighting is only performed\n> + at adjacent block boundaries of blocks that have the same sign.\n> +alternate::\n> + Based on 'nobounds'. Additionally blocks of moved code are\n> + detected. If moved blocks are adjacent mark one of them with the\n> + alternative move color using 'color.diff.newMovedAlternate' or\n> + 'color.diff.oldMovedAlternate'.\n> +--\n> +\n> --word-diff[=]::\n> Show a word diff, using the to delimit changed words.\n> By default, words are delimited by whitespace; see\n\n--\nPhilip\n\n> diff --git a/color.h b/color.h\n> index 90627650fc..04b3b87929 100644\n> --- a/color.h\n> +++ b/color.h\n> @@ -42,6 +42,8 @@ struct strbuf;\n> #define GIT_COLOR_BG_BLUE \"\\033[44m\"\n> #define GIT_COLOR_BG_MAGENTA \"\\033[45m\"\n> #define GIT_COLOR_BG_CYAN \"\\033[46m\"\n> +#define GIT_COLOR_DI_IT_CYAN \"\\033[2;3;36m\"\n> +#define GIT_COLOR_DI_IT_MAGENTA \"\\033[2;3;35m\"\n>\n> /* A special value meaning \"no color selected\" */\n> #define GIT_COLOR_NIL \"NIL\"\n> diff --git a/diff.c b/diff.c\n> index a3c16ef827..efd2530a89 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -31,6 +31,7 @@ static int diff_indent_heuristic; /* experimental */\n> static int diff_rename_limit_default = 400;\n> static int diff_suppress_blank_empty;\n> static int diff_use_color_default = -1;\n> +static int diff_color_moved_default;\n> static int diff_context_default = 3;\n> static int diff_interhunk_context_default;\n> static const char *diff_word_regex_cfg;\n> @@ -55,6 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n> GIT_COLOR_YELLOW, /* COMMIT */\n> GIT_COLOR_BG_RED, /* WHITESPACE */\n> GIT_COLOR_NORMAL, /* FUNCINFO */\n> + GIT_COLOR_DI_IT_MAGENTA,/* OLD_MOVED */\n> + GIT_COLOR_BG_RED, /* OLD_MOVED ALTERNATIVE */\n> + GIT_COLOR_DI_IT_CYAN, /* NEW_MOVED */\n> + GIT_COLOR_BG_GREEN, /* NEW_MOVED ALTERNATIVE */\n> };\n>\n> static NORETURN void die_want_option(const char *option_name)\n> @@ -80,6 +85,14 @@ static int parse_diff_color_slot(const char *var)\n> return DIFF_WHITESPACE;\n> if (!strcasecmp(var, \"func\"))\n> return DIFF_FUNCINFO;\n> + if (!strcasecmp(var, \"oldmoved\"))\n> + return DIFF_FILE_OLD_MOVED;\n> + if (!strcasecmp(var, \"oldmovedalternative\"))\n> + return DIFF_FILE_OLD_MOVED_ALT;\n> + if (!strcasecmp(var, \"newmoved\"))\n> + return DIFF_FILE_NEW_MOVED;\n> + if (!strcasecmp(var, \"newmovedalternative\"))\n> + return DIFF_FILE_NEW_MOVED_ALT;\n> return -1;\n> }\n>\n> @@ -228,12 +241,35 @@ int git_diff_heuristic_config(const char *var, const \n> char *value, void *cb)\n> return 0;\n> }\n>\n> +static int parse_color_moved(const char *arg)\n> +{\n> + if (!strcmp(arg, \"no\"))\n> + return MOVED_LINES_NO;\n> + else if (!strcmp(arg, \"nobounds\"))\n> + return MOVED_LINES_BOUNDARY_NO;\n> + else if (!strcmp(arg, \"allbounds\"))\n> + return MOVED_LINES_BOUNDARY_ALL;\n> + else if (!strcmp(arg, \"adjacentbounds\"))\n> + return MOVED_LINES_BOUNDARY_ADJACENT;\n> + else if (!strcmp(arg, \"alternate\"))\n> + return MOVED_LINES_ALTERNATE;\n> + else\n> + return -1;\n> +}\n> +\n> int git_diff_ui_config(const char *var, const char *value, void *cb)\n> {\n> if (!strcmp(var, \"diff.color\") || !strcmp(var, \"color.diff\")) {\n> diff_use_color_default = git_config_colorbool(var, value);\n> return 0;\n> }\n> + if (!strcmp(var, \"diff.colormoved\")) {\n> + int cm = parse_color_moved(value);\n> + if (cm < 0)\n> + return -1;\n> + diff_color_moved_default = cm;\n> + return 0;\n> + }\n> if (!strcmp(var, \"diff.context\")) {\n> diff_context_default = git_config_int(var, value);\n> if (diff_context_default < 0)\n> @@ -354,6 +390,88 @@ int git_diff_basic_config(const char *var, const char \n> *value, void *cb)\n> return git_default_config(var, value, cb);\n> }\n>\n> +struct moved_entry {\n> + struct hashmap_entry ent;\n> + const struct diff_line *line;\n> + struct moved_entry *next_line;\n> +};\n> +\n> +static void get_ws_cleaned_string(const struct diff_line *l,\n> + struct strbuf *out)\n> +{\n> + int i;\n> + for (i = 0; i < l->len; i++) {\n> + if (isspace(l->line[i]))\n> + continue;\n> + strbuf_addch(out, l->line[i]);\n> + }\n> +}\n> +\n> +static int diff_line_cmp_no_ws(const struct diff_line *a,\n> + const struct diff_line *b,\n> + const void *keydata)\n> +{\n> + int ret;\n> + struct strbuf sba = STRBUF_INIT;\n> + struct strbuf sbb = STRBUF_INIT;\n> +\n> + get_ws_cleaned_string(a, &sba);\n> + get_ws_cleaned_string(b, &sbb);\n> + ret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);\n> +\n> + strbuf_release(&sba);\n> + strbuf_release(&sbb);\n> + return ret;\n> +}\n> +\n> +static int diff_line_cmp(const struct diff_line *a,\n> + const struct diff_line *b,\n> + const void *keydata)\n> +{\n> + return a->len != b->len || strncmp(a->line, b->line, a->len);\n> +}\n> +\n> +static int moved_entry_cmp(const struct moved_entry *a,\n> + const struct moved_entry *b,\n> + const void *keydata)\n> +{\n> + return diff_line_cmp(a->line, b->line, keydata);\n> +}\n> +\n> +static int moved_entry_cmp_no_ws(const struct moved_entry *a,\n> + const struct moved_entry *b,\n> + const void *keydata)\n> +{\n> + return diff_line_cmp_no_ws(a->line, b->line, keydata);\n> +}\n> +\n> +static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)\n> +{\n> + static struct strbuf sb = STRBUF_INIT;\n> +\n> + if (ignore_ws) {\n> + strbuf_reset(&sb);\n> + get_ws_cleaned_string(line, &sb);\n> + return memhash(sb.buf, sb.len);\n> + } else {\n> + return memhash(line->line, line->len);\n> + }\n> +}\n> +\n> +static struct moved_entry *prepare_entry(struct diff_options *o,\n> + int line_no)\n> +{\n> + struct moved_entry *ret = xmalloc(sizeof(*ret));\n> + unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n> + struct diff_line *l = &o->line_buffer[line_no];\n> +\n> + ret->ent.hash = get_line_hash(l, ignore_ws);\n> + ret->line = l;\n> + ret->next_line = NULL;\n> +\n> + return ret;\n> +}\n> +\n> static char *quote_two(const char *one, const char *two)\n> {\n> int need_one = quote_c_style(one, NULL, NULL, 1);\n> @@ -516,6 +634,179 @@ static void check_blank_at_eof(mmfile_t *mf1, \n> mmfile_t *mf2,\n> ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n> }\n>\n> +static void add_lines_to_move_detection(struct diff_options *o,\n> + struct hashmap *add_lines,\n> + struct hashmap *del_lines)\n> +{\n> + struct moved_entry *prev_line = NULL;\n> +\n> + int n;\n> + for (n = 0; n < o->line_buffer_nr; n++) {\n> + int sign = 0;\n> + struct hashmap *hm;\n> + struct moved_entry *key;\n> +\n> + switch (o->line_buffer[n].sign) {\n> + case '+':\n> + sign = '+';\n> + hm = add_lines;\n> + break;\n> + case '-':\n> + sign = '-';\n> + hm = del_lines;\n> + break;\n> + case ' ':\n> + default:\n> + prev_line = NULL;\n> + continue;\n> + }\n> +\n> + key = prepare_entry(o, n);\n> + if (prev_line &&\n> + prev_line->line->sign == sign)\n> + prev_line->next_line = key;\n> +\n> + hashmap_add(hm, key);\n> + prev_line = key;\n> + }\n> +}\n> +\n> +static void mark_color_as_moved_single_line(struct diff_options *o,\n> + struct diff_line *l, int alt_color)\n> +{\n> + switch (l->sign) {\n> + case '+':\n> + l->set = diff_get_color_opt(o,\n> + DIFF_FILE_NEW_MOVED + alt_color);\n> + break;\n> + case '-':\n> + l->set = diff_get_color_opt(o,\n> + DIFF_FILE_OLD_MOVED + alt_color);\n> + break;\n> + default:\n> + die(\"BUG: we should have continued earlier?\");\n> + }\n> +}\n> +\n> +static void mark_color_as_moved(struct diff_options *o,\n> + struct hashmap *add_lines,\n> + struct hashmap *del_lines)\n> +{\n> + struct moved_entry **pmb = NULL; /* potentially moved blocks */\n> + struct diff_line *prev_line = NULL;\n> + int pmb_nr = 0, pmb_alloc = 0;\n> + int n, flipped_block = 0;\n> +\n> + for (n = 0; n < o->line_buffer_nr; n++) {\n> + struct hashmap *hm = NULL;\n> + struct moved_entry *key;\n> + struct moved_entry *match = NULL;\n> + struct diff_line *l = &o->line_buffer[n];\n> + int i, lp, rp, adjacent_blocks = 0;\n> +\n> + /* Check for any match to color it as a move. */\n> + switch (l->sign) {\n> + case '+':\n> + hm = del_lines;\n> + key = prepare_entry(o, n);\n> + match = hashmap_get(hm, key, o);\n> + free(key);\n> + break;\n> + case '-':\n> + hm = add_lines;\n> + key = prepare_entry(o, n);\n> + match = hashmap_get(hm, key, o);\n> + free(key);\n> + break;\n> + default: ;\n> + }\n> +\n> + if (!match) {\n> + pmb_nr = 0;\n> + if (prev_line &&\n> + o->color_moved == MOVED_LINES_BOUNDARY_ALL)\n> + mark_color_as_moved_single_line(o, prev_line, 1);\n> + prev_line = NULL;\n> + continue;\n> + }\n> +\n> + if (o->color_moved == MOVED_LINES_BOUNDARY_NO) {\n> + mark_color_as_moved_single_line(o, l, 0);\n> + continue;\n> + }\n> +\n> + /* Check any potential block runs, advance each or nullify */\n> + for (i = 0; i < pmb_nr; i++) {\n> + struct moved_entry *p = pmb[i];\n> + struct moved_entry *pnext = (p && p->next_line) ?\n> + p->next_line : NULL;\n> + if (pnext &&\n> + !diff_line_cmp(pnext->line, l, o)) {\n> + pmb[i] = p->next_line;\n> + } else {\n> + pmb[i] = NULL;\n> + }\n> + }\n> +\n> + /* Shrink the set of potential block to the remaining running */\n> + for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n> + while (lp < pmb_nr && pmb[lp])\n> + lp++;\n> + /* lp points at the first NULL now */\n> +\n> + while (rp > -1 && !pmb[rp])\n> + rp--;\n> + /* rp points at the last non-NULL */\n> +\n> + if (lp < pmb_nr && rp > -1 && lp < rp) {\n> + pmb[lp] = pmb[rp];\n> + pmb[rp] = NULL;\n> + rp--;\n> + lp++;\n> + }\n> + }\n> +\n> + /* Remember the number of running sets */\n> + pmb_nr = rp + 1;\n> +\n> + if (pmb_nr == 0) {\n> + /*\n> + * This line is the start of a new block.\n> + * Setup the set of potential blocks.\n> + */\n> + for (; match; match = hashmap_get_next(hm, match)) {\n> + ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n> + pmb[pmb_nr++] = match;\n> + }\n> +\n> + if (o->color_moved == MOVED_LINES_BOUNDARY_ALL) {\n> + adjacent_blocks = 1;\n> + } else {\n> + /* Check if two blocks are adjacent */\n> + adjacent_blocks = prev_line &&\n> + prev_line->sign == l->sign;\n> + }\n> + }\n> +\n> + if (o->color_moved == MOVED_LINES_ALTERNATE) {\n> + if (adjacent_blocks)\n> + flipped_block = (flipped_block + 1) % 2;\n> + mark_color_as_moved_single_line(o, l, flipped_block);\n> + } else {\n> + /* MOVED_LINES_BOUNDARY_{ADJACENT, ALL} */\n> + mark_color_as_moved_single_line(o, l, adjacent_blocks);\n> + if (adjacent_blocks && prev_line)\n> + prev_line->set = l->set;\n> + }\n> +\n> + prev_line = l;\n> + }\n> + if (prev_line && o->color_moved == MOVED_LINES_BOUNDARY_ALL)\n> + mark_color_as_moved_single_line(o, prev_line, 1);\n> +\n> + free(pmb);\n> +}\n> +\n> static void emit_diff_line(struct diff_options *o,\n> struct diff_line *e)\n> {\n> @@ -3518,6 +3809,8 @@ void diff_setup(struct diff_options *options)\n> options->line_buffer = NULL;\n> options->line_buffer_nr = 0;\n> options->line_buffer_alloc = 0;\n> +\n> + options->color_moved = diff_color_moved_default;\n> }\n>\n> void diff_setup_done(struct diff_options *options)\n> @@ -3627,6 +3920,9 @@ void diff_setup_done(struct diff_options *options)\n>\n> if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)\n> die(_(\"--follow requires exactly one pathspec\"));\n> +\n> + if (!options->use_color || external_diff())\n> + options->color_moved = 0;\n> }\n>\n> static int opt_arg(const char *arg, int arg_short, const char *arg_long, \n> int *val)\n> @@ -4051,7 +4347,19 @@ int diff_opt_parse(struct diff_options *options,\n> }\n> else if (!strcmp(arg, \"--no-color\"))\n> options->use_color = 0;\n> - else if (!strcmp(arg, \"--color-words\")) {\n> + else if (!strcmp(arg, \"--color-moved\"))\n> + if (diff_color_moved_default)\n> + options->color_moved = diff_color_moved_default;\n> + else\n> + options->color_moved = MOVED_LINES_BOUNDARY_ADJACENT;\n> + else if (!strcmp(arg, \"--no-color-moved\"))\n> + options->color_moved = MOVED_LINES_NO;\n> + else if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n> + int cm = parse_color_moved(arg);\n> + if (cm < 0)\n> + die(\"bad --color-moved argument: %s\", arg);\n> + options->color_moved = cm;\n> + } else if (!strcmp(arg, \"--color-words\")) {\n> options->use_color = 1;\n> options->word_diff = DIFF_WORDS_COLOR;\n> }\n> @@ -4856,16 +5164,9 @@ static void diff_flush_patch_all_file_pairs(struct \n> diff_options *o)\n> {\n> int i;\n> struct diff_queue_struct *q = &diff_queued_diff;\n> - /*\n> - * For testing purposes we want to make sure the diff machinery\n> - * works completely with the buffer. If there is anything emitted\n> - * outside the emit_diff_line, then the order is screwed\n> - * up and the tests will fail.\n> - *\n> - * TODO (later in this series):\n> - * We'll unset this flag in a later patch.\n> - */\n> - o->use_buffer = 1;\n> +\n> + if (o->color_moved)\n> + o->use_buffer = 1;\n>\n> for (i = 0; i < q->nr; i++) {\n> struct diff_filepair *p = q->queue[i];\n> @@ -4874,6 +5175,24 @@ static void diff_flush_patch_all_file_pairs(struct \n> diff_options *o)\n> }\n>\n> if (o->use_buffer) {\n> + if (o->color_moved) {\n> + struct hashmap add_lines, del_lines;\n> + unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n> +\n> + hashmap_init(&del_lines, ignore_ws ?\n> + (hashmap_cmp_fn)moved_entry_cmp_no_ws :\n> + (hashmap_cmp_fn)moved_entry_cmp, 0);\n> + hashmap_init(&add_lines, ignore_ws ?\n> + (hashmap_cmp_fn)moved_entry_cmp_no_ws :\n> + (hashmap_cmp_fn)moved_entry_cmp, 0);\n> +\n> + add_lines_to_move_detection(o, &add_lines, &del_lines);\n> + mark_color_as_moved(o, &add_lines, &del_lines);\n> +\n> + hashmap_free(&add_lines, 0);\n> + hashmap_free(&del_lines, 0);\n> + }\n> +\n> for (i = 0; i < o->line_buffer_nr; i++)\n> emit_diff_line(o, &o->line_buffer[i]);\n>\n> @@ -4962,6 +5281,7 @@ void diff_flush(struct diff_options *options)\n> if (!options->file)\n> die_errno(\"Could not open /dev/null\");\n> options->close_file = 1;\n> + options->color_moved = 0;\n> for (i = 0; i < q->nr; i++) {\n> struct diff_filepair *p = q->queue[i];\n> if (check_pair_status(p))\n> diff --git a/diff.h b/diff.h\n> index be51e8f867..d9fbafd383 100644\n> --- a/diff.h\n> +++ b/diff.h\n> @@ -7,6 +7,7 @@\n> #include \"tree-walk.h\"\n> #include \"pathspec.h\"\n> #include \"object.h\"\n> +#include \"hashmap.h\"\n>\n> struct rev_info;\n> struct diff_options;\n> @@ -228,6 +229,14 @@ struct diff_options {\n>\n> struct diff_line *line_buffer;\n> int line_buffer_nr, line_buffer_alloc;\n> +\n> + enum {\n> + MOVED_LINES_NO = 0,\n> + MOVED_LINES_BOUNDARY_NO = 1,\n> + MOVED_LINES_BOUNDARY_ALL = 2,\n> + MOVED_LINES_BOUNDARY_ADJACENT = 3,\n> + MOVED_LINES_ALTERNATE = 4,\n> + } color_moved;\n> };\n>\n> /* Emit [line_prefix] [set] line [reset] */\n> @@ -243,7 +252,11 @@ enum color_diff {\n> DIFF_FILE_NEW = 5,\n> DIFF_COMMIT = 6,\n> DIFF_WHITESPACE = 7,\n> - DIFF_FUNCINFO = 8\n> + DIFF_FUNCINFO = 8,\n> + DIFF_FILE_OLD_MOVED = 9,\n> + DIFF_FILE_OLD_MOVED_ALT = 10,\n> + DIFF_FILE_NEW_MOVED = 11,\n> + DIFF_FILE_NEW_MOVED_ALT = 12\n> };\n> const char *diff_get_color(int diff_use_color, enum color_diff ix);\n> #define diff_get_color_opt(o, ix) \\\n> diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\n> index 289806d0c7..d4bd082af7 100755\n> --- a/t/t4015-diff-whitespace.sh\n> +++ b/t/t4015-diff-whitespace.sh\n> @@ -972,4 +972,377 @@ test_expect_success 'option overrides \n> diff.wsErrorHighlight' '\n>\n> '\n>\n> +test_expect_success 'detect moved code, complete file' '\n> + git reset --hard &&\n> + cat <<-\\EOF >test.c &&\n> + #include\n> + main()\n> + {\n> + printf(\"Hello World\");\n> + }\n> + EOF\n> + git add test.c &&\n> + git commit -m \"add main function\" &&\n> + git mv test.c main.c &&\n> + test_config color.diff.oldMoved \"normal red\" &&\n> + test_config color.diff.newMoved \"normal green\" &&\n> + git diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n> + cat >expected <<-\\EOF &&\n> + diff --git a/main.c b/main.c\n> + new file mode 100644\n> + index 0000000..a986c57\n> + --- /dev/null\n> + +++ b/main.c\n> + @@ -0,0 +1,5 @@\n> + +#include\n> + +main()\n> + +{\n> + +printf(\"Hello World\");\n> + +}\n> + diff --git a/test.c b/test.c\n> + deleted file mode 100644\n> + index a986c57..0000000\n> + --- a/test.c\n> + +++ /dev/null\n> + @@ -1,5 +0,0 @@\n> + -#include\n> + -main()\n> + -{\n> + -printf(\"Hello World\");\n> + -}\n> + EOF\n> +\n> + test_cmp expected actual\n> +'\n> +\n> +test_expect_success 'detect moved code, inside file' '\n> + git reset --hard &&\n> + cat <<-\\EOF >main.c &&\n> + #include\n> + int stuff()\n> + {\n> + printf(\"Hello \");\n> + printf(\"World\\n\");\n> + }\n> +\n> + int secure_foo(struct user *u)\n> + {\n> + if (!u->is_allowed_foo)\n> + return;\n> + foo(u);\n> + }\n> +\n> + int main()\n> + {\n> + foo();\n> + }\n> + EOF\n> + cat <<-\\EOF >test.c &&\n> + #include\n> + int bar()\n> + {\n> + printf(\"Hello World, but different\\n\");\n> + }\n> +\n> + int another_function()\n> + {\n> + bar();\n> + }\n> + EOF\n> + git add main.c test.c &&\n> + git commit -m \"add main and test file\" &&\n> + cat <<-\\EOF >main.c &&\n> + #include\n> + int stuff()\n> + {\n> + printf(\"Hello \");\n> + printf(\"World\\n\");\n> + }\n> +\n> + int main()\n> + {\n> + foo();\n> + }\n> + EOF\n> + cat <<-\\EOF >test.c &&\n> + #include\n> + int bar()\n> + {\n> + printf(\"Hello World, but different\\n\");\n> + }\n> +\n> + int secure_foo(struct user *u)\n> + {\n> + if (!u->is_allowed_foo)\n> + return;\n> + foo(u);\n> + }\n> +\n> + int another_function()\n> + {\n> + bar();\n> + }\n> + EOF\n> + test_config color.diff.oldMoved \"normal red\" &&\n> + test_config color.diff.newMoved \"normal green\" &&\n> + test_config color.diff.oldMovedAlternative \"bold red\" &&\n> + test_config color.diff.newMovedAlternative \"bold green\" &&\n> + git diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n> + cat <<-\\EOF >expected &&\n> + diff --git a/main.c b/main.c\n> + index 27a619c..7cf9336 100644\n> + --- a/main.c\n> + +++ b/main.c\n> + @@ -5,13 +5,6 @@ printf(\"Hello \");\n> + printf(\"World\\n\");\n> + }\n> + \n> + -int secure_foo(struct user *u)\n> + -{\n> + -if (!u->is_allowed_foo)\n> + -return;\n> + -foo(u);\n> + -}\n> + -\n> + int main()\n> + {\n> + foo();\n> + diff --git a/test.c b/test.c\n> + index 1dc1d85..e34eb69 100644\n> + --- a/test.c\n> + +++ b/test.c\n> + @@ -4,6 +4,13 @@ int bar()\n> + printf(\"Hello World, but different\\n\");\n> + }\n> + \n> + +int secure_foo(struct user *u)\n> + +{\n> + +if (!u->is_allowed_foo)\n> + +return;\n> + +foo(u);\n> + +}\n> + +\n> + int another_function()\n> + {\n> + bar();\n> + EOF\n> +\n> + test_cmp expected actual\n> +'\n> +\n> +test_expect_success 'detect permutations inside moved code' '\n> + git reset --hard &&\n> + cat <<-\\EOF >lines.txt &&\n> + line 1\n> + line 2\n> + line 3\n> + line 4\n> + line 5\n> + line 6\n> + line 7\n> + line 8\n> + line 9\n> + line 10\n> + line 11\n> + line 12\n> + line 13\n> + line 14\n> + line 15\n> + line 16\n> + EOF\n> + git add lines.txt &&\n> + git commit -m \"add poetry\" &&\n> + cat <<-\\EOF >lines.txt &&\n> + line 4\n> + line 5\n> + line 6\n> + line 7\n> + line 8\n> + line 9\n> + line 1\n> + line 2\n> + line 3\n> + line 14\n> + line 15\n> + line 16\n> + line 10\n> + line 11\n> + line 12\n> + line 13\n> + EOF\n> + test_config color.diff.oldMoved \"magenta\" &&\n> + test_config color.diff.newMoved \"cyan\" &&\n> + test_config color.diff.oldMovedAlternative \"blue\" &&\n> + test_config color.diff.newMovedAlternative \"yellow\" &&\n> +\n> +\n> + git diff HEAD --no-renames --color-moved=nobounds| test_decode_color \n> >actual &&\n> + cat <<-\\EOF >expected &&\n> + diff --git a/lines.txt b/lines.txt\n> + index 47ea9c3..ba96a38 100644\n> + --- a/lines.txt\n> + +++ b/lines.txt\n> + @@ -1,16 +1,16 @@\n> + -line 1\n> + -line 2\n> + -line 3\n> + line 4\n> + line 5\n> + line 6\n> + line 7\n> + line 8\n> + line 9\n> + +line 1\n> + +line 2\n> + +line 3\n> + +line 14\n> + +line 15\n> + +line 16\n> + line 10\n> + line 11\n> + line 12\n> + line 13\n> + -line 14\n> + -line 15\n> + -line 16\n> + EOF\n> + test_cmp expected actual &&\n> +\n> + git diff HEAD --no-renames --color-moved=adjacentbounds| \n> test_decode_color >actual &&\n> + cat <<-\\EOF >expected &&\n> + diff --git a/lines.txt b/lines.txt\n> + index 47ea9c3..ba96a38 100644\n> + --- a/lines.txt\n> + +++ b/lines.txt\n> + @@ -1,16 +1,16 @@\n> + -line 1\n> + -line 2\n> + -line 3\n> + line 4\n> + line 5\n> + line 6\n> + line 7\n> + line 8\n> + line 9\n> + +line 1\n> + +line 2\n> + +line 3\n> + +line 14\n> + +line 15\n> + +line 16\n> + line 10\n> + line 11\n> + line 12\n> + line 13\n> + -line 14\n> + -line 15\n> + -line 16\n> + EOF\n> + test_cmp expected actual &&\n> +\n> + test_config diff.colorMoved alternate &&\n> + git diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n> + cat <<-\\EOF >expected &&\n> + diff --git a/lines.txt b/lines.txt\n> + index 47ea9c3..ba96a38 100644\n> + --- a/lines.txt\n> + +++ b/lines.txt\n> + @@ -1,16 +1,16 @@\n> + -line 1\n> + -line 2\n> + -line 3\n> + line 4\n> + line 5\n> + line 6\n> + line 7\n> + line 8\n> + line 9\n> + +line 1\n> + +line 2\n> + +line 3\n> + +line 14\n> + +line 15\n> + +line 16\n> + line 10\n> + line 11\n> + line 12\n> + line 13\n> + -line 14\n> + -line 15\n> + -line 16\n> + EOF\n> + test_cmp expected actual &&\n> +\n> + test_config diff.colorMoved allbounds &&\n> + git diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n> + cat <<-\\EOF >expected &&\n> + diff --git a/lines.txt b/lines.txt\n> + index 47ea9c3..ba96a38 100644\n> + --- a/lines.txt\n> + +++ b/lines.txt\n> + @@ -1,16 +1,16 @@\n> + -line 1\n> + -line 2\n> + -line 3\n> + line 4\n> + line 5\n> + line 6\n> + line 7\n> + line 8\n> + line 9\n> + +line 1\n> + +line 2\n> + +line 3\n> + +line 14\n> + +line 15\n> + +line 16\n> + line 10\n> + line 11\n> + line 12\n> + line 13\n> + -line 14\n> + -line 15\n> + -line 16\n> + EOF\n> + test_cmp expected actual\n> +'\n> +\n> +test_expect_success 'move detection does not mess up colored words' '\n> + cat <<-\\EOF >text.txt &&\n> + Lorem Ipsum is simply dummy text of the printing and typesetting \n> industry.\n> + EOF\n> + git add text.txt &&\n> + git commit -a -m \"clean state\" &&\n> + cat <<-\\EOF >text.txt &&\n> + simply Lorem Ipsum dummy is text of the typesetting and printing \n> industry.\n> + EOF\n> + git diff --color-moved --word-diff >actual &&\n> + git diff --word-diff >expect &&\n> + test_cmp expect actual\n> +'\n> +\n> +test_expect_success 'move detection with submodules' '\n> + test_create_repo bananas &&\n> + echo ripe >bananas/recipe &&\n> + git -C bananas add recipe &&\n> + test_commit fruit &&\n> + test_commit -C bananas recipe &&\n> + git submodule add ./bananas &&\n> + git add bananas &&\n> + git commit -a -m \"bananas are like a heavy library?\" &&\n> + echo foul >bananas/recipe &&\n> + echo ripe >fruit.t &&\n> +\n> + git diff --submodule=diff --color-moved >actual &&\n> +\n> + # no move detection as the moved line is across repository boundaries.\n> + test_decode_color decoded_actual &&\n> + ! grep BGREEN decoded_actual &&\n> + ! grep BRED decoded_actual &&\n> +\n> + # nor did we mess with it another way\n> + git diff --submodule=diff | test_decode_color >expect &&\n> + test_cmp expect decoded_actual\n> +'\n> +\n> test_done\n> -- \n> 2.13.0.17.gab62347cd9\n> \n\n"},{"id":"321083","messageId":"CAGZ79kbq3XiP8W_01FV133aMjZP9_GvpEg86N=XC2rTy24ZZGQ@mail.gmail.com","threadId":"46038","inReplyTo":"39D67E58AC02490FB6D6124BDF3EE2A9@PhilipOakley","subject":"Re: [PATCH 1/1] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-30T21:33:02Z","receivedAt":"2017-05-30T21:33:14Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Sat, May 27, 2017 at 12:05 AM, Philip Oakley wrote:\n> a couple of mispellings in the doc parts:\n> s/on location/one location/\n> [code not checked]\n\nThanks for proofreading the documentation!\n\n>\n> s/on location/one location/\n>\n>> + in another location will be colored with 'color.diff.newmoved'.\n>> + Any line that is removed in on location and was added\n>\n> s/on location/one location/\n\nFor a moment I was worried that my 'e' key is stuck as it happens\nto be the same misspelling here. However then I realized I copied\nover part of the sentence and just replaced added/removed.\n\nStepping back a bit, maybe the second sentence can be partially\nelided to be more concise. Maybe:\n\n Similarly 'color.diff.oldmoved' will be used for removed lines\n that are added somewhere else in the diff.\n\nThanks,\nStefan\n"},{"id":"321227","messageId":"20170601002429.3470-1-sbeller@google.com","threadId":"46038","inReplyTo":"CAGZ79kbq3XiP8W_01FV133aMjZP9_GvpEg86N=XC2rTy24ZZGQ@mail.gmail.com","subject":"[PATCH] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-01T00:24:29Z","receivedAt":"2017-06-01T00:24:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a patch consists mostly of moving blocks of code around, it can\nbe quite tedious to ensure that the blocks are moved verbatim, and not\nundesirably modified in the move. To that end, color blocks that are\nmoved within the same patch differently. For example (OM, del, add,\nand NM are different colors):\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OM] - if (!is_authorized_user())\n [OM] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OM] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NM] + sensitive_stuff(spanning,\n [NM] + multiple,\n [NM] + lines);\n [NM] +}\n\nHowever adjacent blocks may be problematic. For example, in this\npotentially malicious patch, the swapping of blocks can be spotted:\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OMA] - if (!is_authorized_user())\n [OMA] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OMA] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NMA] + sensitive_stuff(spanning,\n [NMA] + multiple,\n [NMA] + lines);\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NMA] +}\n\nIf the moved code is larger, it is easier to hide some permutation in the\ncode, which is why some alternative coloring is needed.\n\nAs the reviewers attention should be brought to the places, where the\ndifference is introduced to the moved code, we cannot just have one new\ncolor for all of moved code.\n\nFirst I implemented an alternative design, which would try to fingerprint\na line by its neighbors to detect if we are in a block or at the boundary.\nThis idea iss error prone as it inspected each line and its neighboring\nlines to determine if the line was (a) moved and (b) if was deep inside\na hunk by having matching neighboring lines. This is unreliable as the\nwe can construct hunks which have equal neighbors that just exceed the\nnumber of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter\nas a line, that is permutated to AXYZCXYZBXYZD..').\n\nInstead this provides a dynamic programming greedy algorithm that finds\nthe largest moved hunk and then has several modes on highlighting bounds.\n\nA note on the options '--submodule=diff' and '--color-words/--word-diff':\nIn the conversion to use emit_line in the prior patches both submodules\nas well as word diff output carefully chose to call emit_line with sign=0.\nAll output with sign=0 is ignored for move detection purposes in this\npatch, such that no weird looking output will be generated for these\ncases. This leads to another thought: We could pass on '--color-moved' to\nsubmodules such that they color up moved lines for themselves. If we'd do\nso only line moves within a repository boundary are marked up.\n\nHelped-by: Jonathan Tan \nSigned-off-by: Stefan Beller \n---\n\n Replacing the top commit in origin/sb/diff-color-move, \n this has the spelling fixes by Philip.\n \n Also a minor fix for the 'alternate' mode, to go back to the default\n after empty lines. Thanks to Jacob.\n \n Thanks,\n Stefan\n\n Documentation/config.txt | 10 +-\n Documentation/diff-options.txt | 32 ++++\n color.h | 2 +\n diff.c | 343 +++++++++++++++++++++++++++++++++++--\n diff.h | 15 +-\n t/t4015-diff-whitespace.sh | 373 +++++++++++++++++++++++++++++++++++++++++\n 6 files changed, 761 insertions(+), 14 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 475e874d51..73511a4603 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1051,14 +1051,20 @@ This does not affect linkgit:git-format-patch[1] or the\n 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n command line with the `--color[=]` option.\n \n+diff.colorMoved::\n+\tIf set moved lines in a diff are colored differently,\n+\tfor details see '--color-moved' in linkgit:git-diff[1].\n+\n color.diff.::\n \tUse customized color for diff colorization. `` specifies\n \twhich part of the patch to use the specified color, and is one\n \tof `context` (context text - `plain` is a historical synonym),\n \t`meta` (metainformation), `frag`\n \t(hunk header), 'func' (function in hunk header), `old` (removed lines),\n-\t`new` (added lines), `commit` (commit headers), or `whitespace`\n-\t(highlighting whitespace errors).\n+\t`new` (added lines), `commit` (commit headers), `whitespace`\n+\t(highlighting whitespace errors), `oldMoved`, `newMoved`,\n+\t`oldMovedAlternative` and `newMovedAlternative` (See the ''\n+\tsetting of '--color-moved' in linkgit:git-diff[1] for details).\n \n color.decorate.::\n \tUse customized color for 'git log --decorate' output. `` is one\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex 89cc0f48de..69bf061c5c 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -231,6 +231,38 @@ ifdef::git-diff[]\n endif::git-diff[]\n \tIt is the same as `--color=never`.\n \n+--color-moved[=]::\n+\tMoved lines of code are colored differently.\n+ifdef::git-diff[]\n+\tIt can be changed by the `diff.colorMoved` configuration setting.\n+endif::git-diff[]\n+\tThe defaults to 'no' if the option is not given\n+\tand to 'adjacentbounds' if the option with no mode is given.\n+\tThe mode must be one of:\n++\n+--\n+no::\n+\tMoved lines are not highlighted.\n+nobounds::\n+\tAny line that is added in one location and was removed\n+\tin another location will be colored with 'color.diff.newmoved'.\n+\tSimilarly 'color.diff.oldmoved' will be used for removed lines\n+\tthat are added somewhere else in the diff.\n+allbounds::\n+\tBased on 'nobounds'. Additionally blocks of moved code are\n+\tdetected and the first and last line of a block will be highlighted\n+\tusing 'color.diff.newMovedAlternate' or\n+\t'color.diff.oldMovedAlternate'.\n+adjacentbounds::\n+\tThe same as 'allbounds' except that highlighting is only performed\n+\tat adjacent block boundaries of blocks that have the same sign.\n+alternate::\n+\tBased on 'nobounds'. Additionally blocks of moved code are\n+\tdetected. If moved blocks are adjacent mark one of them with the\n+\talternative move color using 'color.diff.newMovedAlternate' or\n+\t'color.diff.oldMovedAlternate'.\n+--\n+\n --word-diff[=]::\n \tShow a word diff, using the to delimit changed words.\n \tBy default, words are delimited by whitespace; see\ndiff --git a/color.h b/color.h\nindex 90627650fc..04b3b87929 100644\n--- a/color.h\n+++ b/color.h\n@@ -42,6 +42,8 @@ struct strbuf;\n #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n+#define GIT_COLOR_DI_IT_CYAN\t\"\\033[2;3;36m\"\n+#define GIT_COLOR_DI_IT_MAGENTA\t\"\\033[2;3;35m\"\n \n /* A special value meaning \"no color selected\" */\n #define GIT_COLOR_NIL \"NIL\"\ndiff --git a/diff.c b/diff.c\nindex a3c16ef827..a1f919ba57 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -31,6 +31,7 @@ static int diff_indent_heuristic; /* experimental */\n static int diff_rename_limit_default = 400;\n static int diff_suppress_blank_empty;\n static int diff_use_color_default = -1;\n+static int diff_color_moved_default;\n static int diff_context_default = 3;\n static int diff_interhunk_context_default;\n static const char *diff_word_regex_cfg;\n@@ -55,6 +56,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n+\tGIT_COLOR_DI_IT_MAGENTA,/* OLD_MOVED */\n+\tGIT_COLOR_BG_RED,\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_DI_IT_CYAN,\t/* NEW_MOVED */\n+\tGIT_COLOR_BG_GREEN,\t/* NEW_MOVED ALTERNATIVE */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -80,6 +85,14 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_WHITESPACE;\n \tif (!strcasecmp(var, \"func\"))\n \t\treturn DIFF_FUNCINFO;\n+\tif (!strcasecmp(var, \"oldmoved\"))\n+\t\treturn DIFF_FILE_OLD_MOVED;\n+\tif (!strcasecmp(var, \"oldmovedalternative\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoved\"))\n+\t\treturn DIFF_FILE_NEW_MOVED;\n+\tif (!strcasecmp(var, \"newmovedalternative\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT;\n \treturn -1;\n }\n \n@@ -228,12 +241,35 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)\n \treturn 0;\n }\n \n+static int parse_color_moved(const char *arg)\n+{\n+\tif (!strcmp(arg, \"no\"))\n+\t\treturn MOVED_LINES_NO;\n+\telse if (!strcmp(arg, \"nobounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_NO;\n+\telse if (!strcmp(arg, \"allbounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_ALL;\n+\telse if (!strcmp(arg, \"adjacentbounds\"))\n+\t\treturn MOVED_LINES_BOUNDARY_ADJACENT;\n+\telse if (!strcmp(arg, \"alternate\"))\n+\t\treturn MOVED_LINES_ALTERNATE;\n+\telse\n+\t\treturn -1;\n+}\n+\n int git_diff_ui_config(const char *var, const char *value, void *cb)\n {\n \tif (!strcmp(var, \"diff.color\") || !strcmp(var, \"color.diff\")) {\n \t\tdiff_use_color_default = git_config_colorbool(var, value);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"diff.colormoved\")) {\n+\t\tint cm = parse_color_moved(value);\n+\t\tif (cm < 0)\n+\t\t\treturn -1;\n+\t\tdiff_color_moved_default = cm;\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"diff.context\")) {\n \t\tdiff_context_default = git_config_int(var, value);\n \t\tif (diff_context_default < 0)\n@@ -354,6 +390,88 @@ int git_diff_basic_config(const char *var, const char *value, void *cb)\n \treturn git_default_config(var, value, cb);\n }\n \n+struct moved_entry {\n+\tstruct hashmap_entry ent;\n+\tconst struct diff_line *line;\n+\tstruct moved_entry *next_line;\n+};\n+\n+static void get_ws_cleaned_string(const struct diff_line *l,\n+\t\t\t\t struct strbuf *out)\n+{\n+\tint i;\n+\tfor (i = 0; i < l->len; i++) {\n+\t\tif (isspace(l->line[i]))\n+\t\t\tcontinue;\n+\t\tstrbuf_addch(out, l->line[i]);\n+\t}\n+}\n+\n+static int diff_line_cmp_no_ws(const struct diff_line *a,\n+\t\t\t\t\t const struct diff_line *b,\n+\t\t\t\t\t const void *keydata)\n+{\n+\tint ret;\n+\tstruct strbuf sba = STRBUF_INIT;\n+\tstruct strbuf sbb = STRBUF_INIT;\n+\n+\tget_ws_cleaned_string(a, &sba);\n+\tget_ws_cleaned_string(b, &sbb);\n+\tret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);\n+\n+\tstrbuf_release(&sba);\n+\tstrbuf_release(&sbb);\n+\treturn ret;\n+}\n+\n+static int diff_line_cmp(const struct diff_line *a,\n+\t\t\t\t const struct diff_line *b,\n+\t\t\t\t const void *keydata)\n+{\n+\treturn a->len != b->len || strncmp(a->line, b->line, a->len);\n+}\n+\n+static int moved_entry_cmp(const struct moved_entry *a,\n+\t\t\t const struct moved_entry *b,\n+\t\t\t const void *keydata)\n+{\n+\treturn diff_line_cmp(a->line, b->line, keydata);\n+}\n+\n+static int moved_entry_cmp_no_ws(const struct moved_entry *a,\n+\t\t\t\t const struct moved_entry *b,\n+\t\t\t\t const void *keydata)\n+{\n+\treturn diff_line_cmp_no_ws(a->line, b->line, keydata);\n+}\n+\n+static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)\n+{\n+\tstatic struct strbuf sb = STRBUF_INIT;\n+\n+\tif (ignore_ws) {\n+\t\tstrbuf_reset(&sb);\n+\t\tget_ws_cleaned_string(line, &sb);\n+\t\treturn memhash(sb.buf, sb.len);\n+\t} else {\n+\t\treturn memhash(line->line, line->len);\n+\t}\n+}\n+\n+static struct moved_entry *prepare_entry(struct diff_options *o,\n+\t\t\t\t\t int line_no)\n+{\n+\tstruct moved_entry *ret = xmalloc(sizeof(*ret));\n+\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\tstruct diff_line *l = &o->line_buffer[line_no];\n+\n+\tret->ent.hash = get_line_hash(l, ignore_ws);\n+\tret->line = l;\n+\tret->next_line = NULL;\n+\n+\treturn ret;\n+}\n+\n static char *quote_two(const char *one, const char *two)\n {\n \tint need_one = quote_c_style(one, NULL, NULL, 1);\n@@ -516,6 +634,180 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,\n \tecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n }\n \n+static void add_lines_to_move_detection(struct diff_options *o,\n+\t\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry *prev_line = NULL;\n+\n+\tint n;\n+\tfor (n = 0; n < o->line_buffer_nr; n++) {\n+\t\tint sign = 0;\n+\t\tstruct hashmap *hm;\n+\t\tstruct moved_entry *key;\n+\n+\t\tswitch (o->line_buffer[n].sign) {\n+\t\tcase '+':\n+\t\t\tsign = '+';\n+\t\t\thm = add_lines;\n+\t\t\tbreak;\n+\t\tcase '-':\n+\t\t\tsign = '-';\n+\t\t\thm = del_lines;\n+\t\t\tbreak;\n+\t\tcase ' ':\n+\t\tdefault:\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tkey = prepare_entry(o, n);\n+\t\tif (prev_line &&\n+\t\t prev_line->line->sign == sign)\n+\t\t\tprev_line->next_line = key;\n+\n+\t\thashmap_add(hm, key);\n+\t\tprev_line = key;\n+\t}\n+}\n+\n+static void mark_color_as_moved_single_line(struct diff_options *o,\n+\t\t\t\t\t struct diff_line *l, int alt_color)\n+{\n+\tswitch (l->sign) {\n+\tcase '+':\n+\t\tl->set = diff_get_color_opt(o,\n+\t\t\tDIFF_FILE_NEW_MOVED + alt_color);\n+\t\tbreak;\n+\tcase '-':\n+\t\tl->set = diff_get_color_opt(o,\n+\t\t\tDIFF_FILE_OLD_MOVED + alt_color);\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: we should have continued earlier?\");\n+\t}\n+}\n+\n+static void mark_color_as_moved(struct diff_options *o,\n+\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry **pmb = NULL; /* potentially moved blocks */\n+\tstruct diff_line *prev_line = NULL;\n+\tint pmb_nr = 0, pmb_alloc = 0;\n+\tint n, flipped_block = 0;\n+\n+\tfor (n = 0; n < o->line_buffer_nr; n++) {\n+\t\tstruct hashmap *hm = NULL;\n+\t\tstruct moved_entry *key;\n+\t\tstruct moved_entry *match = NULL;\n+\t\tstruct diff_line *l = &o->line_buffer[n];\n+\t\tint i, lp, rp, adjacent_blocks = 0;\n+\n+\t\t/* Check for any match to color it as a move. */\n+\t\tswitch (l->sign) {\n+\t\tcase '+':\n+\t\t\thm = del_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tcase '-':\n+\t\t\thm = add_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tdefault: ;\n+\t\t\tflipped_block = 0;\n+\t\t}\n+\n+\t\tif (!match) {\n+\t\t\tpmb_nr = 0;\n+\t\t\tif (prev_line &&\n+\t\t\t o->color_moved == MOVED_LINES_BOUNDARY_ALL)\n+\t\t\t\tmark_color_as_moved_single_line(o, prev_line, 1);\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (o->color_moved == MOVED_LINES_BOUNDARY_NO) {\n+\t\t\tmark_color_as_moved_single_line(o, l, 0);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Check any potential block runs, advance each or nullify */\n+\t\tfor (i = 0; i < pmb_nr; i++) {\n+\t\t\tstruct moved_entry *p = pmb[i];\n+\t\t\tstruct moved_entry *pnext = (p && p->next_line) ?\n+\t\t\t\t\tp->next_line : NULL;\n+\t\t\tif (pnext &&\n+\t\t\t !diff_line_cmp(pnext->line, l, o)) {\n+\t\t\t\tpmb[i] = p->next_line;\n+\t\t\t} else {\n+\t\t\t\tpmb[i] = NULL;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Shrink the set of potential block to the remaining running */\n+\t\tfor (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n+\t\t\twhile (lp < pmb_nr && pmb[lp])\n+\t\t\t\tlp++;\n+\t\t\t/* lp points at the first NULL now */\n+\n+\t\t\twhile (rp > -1 && !pmb[rp])\n+\t\t\t\trp--;\n+\t\t\t/* rp points at the last non-NULL */\n+\n+\t\t\tif (lp < pmb_nr && rp > -1 && lp < rp) {\n+\t\t\t\tpmb[lp] = pmb[rp];\n+\t\t\t\tpmb[rp] = NULL;\n+\t\t\t\trp--;\n+\t\t\t\tlp++;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Remember the number of running sets */\n+\t\tpmb_nr = rp + 1;\n+\n+\t\tif (pmb_nr == 0) {\n+\t\t\t/*\n+\t\t\t * This line is the start of a new block.\n+\t\t\t * Setup the set of potential blocks.\n+\t\t\t */\n+\t\t\tfor (; match; match = hashmap_get_next(hm, match)) {\n+\t\t\t\tALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n+\t\t\t\tpmb[pmb_nr++] = match;\n+\t\t\t}\n+\n+\t\t\tif (o->color_moved == MOVED_LINES_BOUNDARY_ALL) {\n+\t\t\t\tadjacent_blocks = 1;\n+\t\t\t} else {\n+\t\t\t\t/* Check if two blocks are adjacent */\n+\t\t\t\tadjacent_blocks = prev_line &&\n+\t\t\t\t\t\t prev_line->sign == l->sign;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (o->color_moved == MOVED_LINES_ALTERNATE) {\n+\t\t\tif (adjacent_blocks)\n+\t\t\t\tflipped_block = (flipped_block + 1) % 2;\n+\t\t\tmark_color_as_moved_single_line(o, l, flipped_block);\n+\t\t} else {\n+\t\t\t/* MOVED_LINES_BOUNDARY_{ADJACENT, ALL} */\n+\t\t\tmark_color_as_moved_single_line(o, l, adjacent_blocks);\n+\t\t\tif (adjacent_blocks && prev_line)\n+\t\t\t\tprev_line->set = l->set;\n+\t\t}\n+\n+\t\tprev_line = l;\n+\t}\n+\tif (prev_line && o->color_moved == MOVED_LINES_BOUNDARY_ALL)\n+\t\tmark_color_as_moved_single_line(o, prev_line, 1);\n+\n+\tfree(pmb);\n+}\n+\n static void emit_diff_line(struct diff_options *o,\n \t\t\t struct diff_line *e)\n {\n@@ -3518,6 +3810,8 @@ void diff_setup(struct diff_options *options)\n \toptions->line_buffer = NULL;\n \toptions->line_buffer_nr = 0;\n \toptions->line_buffer_alloc = 0;\n+\n+\toptions->color_moved = diff_color_moved_default;\n }\n \n void diff_setup_done(struct diff_options *options)\n@@ -3627,6 +3921,9 @@ void diff_setup_done(struct diff_options *options)\n \n \tif (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)\n \t\tdie(_(\"--follow requires exactly one pathspec\"));\n+\n+\tif (!options->use_color || external_diff())\n+\t\toptions->color_moved = 0;\n }\n \n static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)\n@@ -4051,7 +4348,19 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n-\telse if (!strcmp(arg, \"--color-words\")) {\n+\telse if (!strcmp(arg, \"--color-moved\"))\n+\t\tif (diff_color_moved_default)\n+\t\t\toptions->color_moved = diff_color_moved_default;\n+\t\telse\n+\t\t\toptions->color_moved = MOVED_LINES_BOUNDARY_ADJACENT;\n+\telse if (!strcmp(arg, \"--no-color-moved\"))\n+\t\toptions->color_moved = MOVED_LINES_NO;\n+\telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n+\t\tint cm = parse_color_moved(arg);\n+\t\tif (cm < 0)\n+\t\t\tdie(\"bad --color-moved argument: %s\", arg);\n+\t\toptions->color_moved = cm;\n+\t} else if (!strcmp(arg, \"--color-words\")) {\n \t\toptions->use_color = 1;\n \t\toptions->word_diff = DIFF_WORDS_COLOR;\n \t}\n@@ -4856,16 +5165,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n-\t/*\n-\t * For testing purposes we want to make sure the diff machinery\n-\t * works completely with the buffer. If there is anything emitted\n-\t * outside the emit_diff_line, then the order is screwed\n-\t * up and the tests will fail.\n-\t *\n-\t * TODO (later in this series):\n-\t * We'll unset this flag in a later patch.\n-\t */\n-\to->use_buffer = 1;\n+\n+\tif (o->color_moved)\n+\t\to->use_buffer = 1;\n \n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n@@ -4874,6 +5176,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \t}\n \n \tif (o->use_buffer) {\n+\t\tif (o->color_moved) {\n+\t\t\tstruct hashmap add_lines, del_lines;\n+\t\t\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\n+\t\t\thashmap_init(&del_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\t\t\thashmap_init(&add_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\n+\t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n+\t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\n+\t\t\thashmap_free(&add_lines, 0);\n+\t\t\thashmap_free(&del_lines, 0);\n+\t\t}\n+\n \t\tfor (i = 0; i < o->line_buffer_nr; i++)\n \t\t\temit_diff_line(o, &o->line_buffer[i]);\n \n@@ -4962,6 +5282,7 @@ void diff_flush(struct diff_options *options)\n \t\tif (!options->file)\n \t\t\tdie_errno(\"Could not open /dev/null\");\n \t\toptions->close_file = 1;\n+\t\toptions->color_moved = 0;\n \t\tfor (i = 0; i < q->nr; i++) {\n \t\t\tstruct diff_filepair *p = q->queue[i];\n \t\t\tif (check_pair_status(p))\ndiff --git a/diff.h b/diff.h\nindex be51e8f867..d9fbafd383 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -7,6 +7,7 @@\n #include \"tree-walk.h\"\n #include \"pathspec.h\"\n #include \"object.h\"\n+#include \"hashmap.h\"\n \n struct rev_info;\n struct diff_options;\n@@ -228,6 +229,14 @@ struct diff_options {\n \n \tstruct diff_line *line_buffer;\n \tint line_buffer_nr, line_buffer_alloc;\n+\n+\tenum {\n+\t\tMOVED_LINES_NO = 0,\n+\t\tMOVED_LINES_BOUNDARY_NO = 1,\n+\t\tMOVED_LINES_BOUNDARY_ALL = 2,\n+\t\tMOVED_LINES_BOUNDARY_ADJACENT = 3,\n+\t\tMOVED_LINES_ALTERNATE = 4,\n+\t} color_moved;\n };\n \n /* Emit [line_prefix] [set] line [reset] */\n@@ -243,7 +252,11 @@ enum color_diff {\n \tDIFF_FILE_NEW = 5,\n \tDIFF_COMMIT = 6,\n \tDIFF_WHITESPACE = 7,\n-\tDIFF_FUNCINFO = 8\n+\tDIFF_FUNCINFO = 8,\n+\tDIFF_FILE_OLD_MOVED = 9,\n+\tDIFF_FILE_OLD_MOVED_ALT = 10,\n+\tDIFF_FILE_NEW_MOVED = 11,\n+\tDIFF_FILE_NEW_MOVED_ALT = 12\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 289806d0c7..e7b821be0a 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -972,4 +972,377 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '\n \n '\n \n+test_expect_success 'detect moved code, complete file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >test.c &&\n+\t#include\n+\tmain()\n+\t{\n+\tprintf(\"Hello World\");\n+\t}\n+\tEOF\n+\tgit add test.c &&\n+\tgit commit -m \"add main function\" &&\n+\tgit mv test.c main.c &&\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tdiff --git a/main.c b/main.c\n+\tnew file mode 100644\n+\tindex 0000000..a986c57\n+\t--- /dev/null\n+\t+++ b/main.c\n+\t@@ -0,0 +1,5 @@\n+\t+#include\n+\t+main()\n+\t+{\n+\t+printf(\"Hello World\");\n+\t+}\n+\tdiff --git a/test.c b/test.c\n+\tdeleted file mode 100644\n+\tindex a986c57..0000000\n+\t--- a/test.c\n+\t+++ /dev/null\n+\t@@ -1,5 +0,0 @@\n+\t-#include\n+\t-main()\n+\t-{\n+\t-printf(\"Hello World\");\n+\t-}\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect moved code, inside file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit add main.c test.c &&\n+\tgit commit -m \"add main and test file\" &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"bold red\" &&\n+\ttest_config color.diff.newMovedAlternative \"bold green\" &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..e34eb69 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+foo(u);\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect permutations inside moved code' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\tEOF\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\tEOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\n+\n+\tgit diff HEAD --no-renames --color-moved=nobounds| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\t\tdiff --git a/lines.txt b/lines.txt\n+\t\tindex 47ea9c3..ba96a38 100644\n+\t\t--- a/lines.txt\n+\t\t+++ b/lines.txt\n+\t\t@@ -1,16 +1,16 @@\n+\t\t-line 1\n+\t\t-line 2\n+\t\t-line 3\n+\t\t line 4\n+\t\t line 5\n+\t\t line 6\n+\t\t line 7\n+\t\t line 8\n+\t\t line 9\n+\t\t+line 1\n+\t\t+line 2\n+\t\t+line 3\n+\t\t+line 14\n+\t\t+line 15\n+\t\t+line 16\n+\t\t line 10\n+\t\t line 11\n+\t\t line 12\n+\t\t line 13\n+\t\t-line 14\n+\t\t-line 15\n+\t\t-line 16\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\tgit diff HEAD --no-renames --color-moved=adjacentbounds| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\ttest_config diff.colorMoved alternate &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\ttest_config diff.colorMoved allbounds &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'move detection does not mess up colored words' '\n+\tcat <<-\\EOF >text.txt &&\n+\tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n+\tEOF\n+\tgit add text.txt &&\n+\tgit commit -a -m \"clean state\" &&\n+\tcat <<-\\EOF >text.txt &&\n+\tsimply Lorem Ipsum dummy is text of the typesetting and printing industry.\n+\tEOF\n+\tgit diff --color-moved --word-diff >actual &&\n+\tgit diff --word-diff >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'move detection with submodules' '\n+\ttest_create_repo bananas &&\n+\techo ripe >bananas/recipe &&\n+\tgit -C bananas add recipe &&\n+\ttest_commit fruit &&\n+\ttest_commit -C bananas recipe &&\n+\tgit submodule add ./bananas &&\n+\tgit add bananas &&\n+\tgit commit -a -m \"bananas are like a heavy library?\" &&\n+\techo foul >bananas/recipe &&\n+\techo ripe >fruit.t &&\n+\n+\tgit diff --submodule=diff --color-moved >actual &&\n+\n+\t# no move detection as the moved line is across repository boundaries.\n+\ttest_decode_color decoded_actual &&\n+\t! grep BGREEN decoded_actual &&\n+\t! grep BRED decoded_actual &&\n+\n+\t# nor did we mess with it another way\n+\tgit diff --submodule=diff | test_decode_color >expect &&\n+\ttest_cmp expect decoded_actual\n+'\n+\n test_done\n-- \n2.13.0.17.gab62347cd9\n\n"},{"id":"322159","messageId":"20170613145457.2e888c27@twelve2.svl.corp.google.com","threadId":"46038","inReplyTo":"20170524214036.29623-5-sbeller@google.com","subject":"Re: [PATCHv5 04/17] diff: introduce more flexible emit function","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-06-13T21:54:57Z","receivedAt":"2017-06-13T21:55:09Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Wed, 24 May 2017 14:40:23 -0700\nStefan Beller wrote:\n\n> Currently, diff output is written either through the emit_line_0\n> function or through the FILE * in struct diff_options directly. To\n> make it easier to teach diff to buffer its output (which will be done\n> in a subsequent commit), introduce a more flexible emit_line() function.\n> In this commit, direct usages of emit_line_0() are replaced with\n> emit_line(); subsequent commits will also replace usages of the\n> FILE * with emit().\n\nCheck the names of the functions in this paragraph.\n\n> diff --git a/diff.c b/diff.c\n> index 2f9722b382..3569857818 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -516,36 +516,30 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,\n> \tecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n> }\n> \n> -static void emit_line_0(struct diff_options *o, const char *set, const char *reset,\n> -\t\t\tint first, const char *line, int len)\n> +static void emit_line(struct diff_options *o, const char *set, const char *reset,\n> +\t\t int add_line_prefix, int sign, const char *line, int len)\n\nIn the future, this function is going to be used even to emit partial\nlines - could this be called emit() instead?\n"},{"id":"322163","messageId":"20170613150722.0bd8fa56@twelve2.svl.corp.google.com","threadId":"46038","inReplyTo":"20170524214036.29623-17-sbeller@google.com","subject":"Re: [PATCHv5 16/17] diff: buffer all output if asked to","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-06-13T22:07:22Z","receivedAt":"2017-06-13T22:07:33Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Wed, 24 May 2017 14:40:35 -0700\nStefan Beller wrote:\n\n> diff --git a/diff.h b/diff.h\n> index 85948ed65a..fad1258556 100644\n> --- a/diff.h\n> +++ b/diff.h\n> @@ -115,6 +115,42 @@ enum diff_submodule_format {\n> \tDIFF_SUBMODULE_INLINE_DIFF\n> };\n> \n> +/*\n> + * This struct is used when we need to buffer the output of the diff output.\n> + *\n> + * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer\n> + * into the pre/post image file. This pointer could be a union with the\n> + * line pointer. By storing an offset into the file instead of the literal line,\n> + * we can decrease the memory footprint for the buffered output. At first we\n> + * may want to only have indirection for the content lines, but we could also\n> + * enhance the state for emitting prefabricated lines, e.g. the similarity\n> + * score line or hunk/file headers would only need to store a number or path\n> + * and then the output can be constructed later on depending on state.\n> + */\n> +struct diff_line {\n\nProbably should be called diff_emission (or just emission), since these\nmay not be full lines.\n\nAlso, can this definition be in the .c file? Callers should use the\ndiff_emit_line() below, and not need to know how it is implemented\ninternally.\n\n> +\tconst char *set;\n> +\tconst char *reset;\n> +\tconst char *line;\n> +\tint len;\n> +\tint sign;\n> +\tint add_line_prefix;\n> +\tenum {\n> +\t\t/*\n> +\t\t * Emits [lineprefix][set][sign][reset] and then calls\n> +\t\t * ws_check_emit which will output \"line\", marked up\n> +\t\t * according to ws_rule.\n> +\t\t */\n> +\t\tDIFF_LINE_WS,\n> +\n> +\t\t/* Emits [lineprefix][set][sign] line [reset] */\n> +\t\tDIFF_LINE_ASIS,\n> +\n> +\t\t/* Reloads the ws_rule; line contains the file name */\n> +\t\tDIFF_LINE_RELOAD_WS_RULE\n> +\t} state;\n> +};\n> +#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}\n\nShould be DIFF_LINE_INIT (capitalization), and {NULL} is sufficient, I\nthink.\n"},{"id":"322167","messageId":"20170613155158.2454d9c7@twelve2.svl.corp.google.com","threadId":"46038","inReplyTo":"20170601002429.3470-1-sbeller@google.com","subject":"Re: [PATCH] diff.c: color moved lines differently","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-06-13T22:51:58Z","receivedAt":"2017-06-13T22:52:08Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Wed, 31 May 2017 17:24:29 -0700\nStefan Beller wrote:\n\n> When a patch consists mostly of moving blocks of code around, it can\n> be quite tedious to ensure that the blocks are moved verbatim, and not\n> undesirably modified in the move. To that end, color blocks that are\n> moved within the same patch differently. For example (OM, del, add,\n> and NM are different colors):\n\n[snip]\n\nJunio asks \"are we happy with these changes\" [1] and my answer is, in\ngeneral, yes - this seems like a very useful feature to have, and I'm OK\nwith the current design.\n\nI do feel a bit of unease at how the emitted strings are collected\nwithout many guarantees as to their contents (e.g. whether they are full\nlines or even whether they originate from the text of a file), but this\nis already true for the existing code. The potential danger is that we\nare now relying more on the format of these strings, but we don't plan\nto do anything other than to color them, so this seems fine.\n\nI would also prefer if there was only one coloring method, to ease\ntesting, but I can tolerate the current multiplicity of options.\n\nI think there was some discussion at trying to avoid indicating that\nsmall blocks (consisting of few non-whitespace characters) are moved,\nbut I think that that can be added later.\n\n[1] https://public-inbox.org/git/CAGZ79kY2Z-fJYxczbzheu1hChLkKkdjEcDMwsP-hkN0TjUBotQ@mail.gmail.com/\n\n> --- a/Documentation/diff-options.txt\n> +++ b/Documentation/diff-options.txt\n> @@ -231,6 +231,38 @@ ifdef::git-diff[]\n> endif::git-diff[]\n> \tIt is the same as `--color=never`.\n> \n> +--color-moved[=]::\n> +\tMoved lines of code are colored differently.\n> +ifdef::git-diff[]\n> +\tIt can be changed by the `diff.colorMoved` configuration setting.\n> +endif::git-diff[]\n> +\tThe defaults to 'no' if the option is not given\n> +\tand to 'adjacentbounds' if the option with no mode is given.\n> +\tThe mode must be one of:\n> ++\n> +--\n> +no::\n> +\tMoved lines are not highlighted.\n> +nobounds::\n> +\tAny line that is added in one location and was removed\n> +\tin another location will be colored with 'color.diff.newmoved'.\n> +\tSimilarly 'color.diff.oldmoved' will be used for removed lines\n\nProbably best to consistently capitalize newMoved and oldMoved.\n\n> +static unsigned get_line_hash(struct diff_line *line, unsigned ignore_ws)\n> +{\n> +\tstatic struct strbuf sb = STRBUF_INIT;\n> +\n> +\tif (ignore_ws) {\n> +\t\tstrbuf_reset(&sb);\n> +\t\tget_ws_cleaned_string(line, &sb);\n> +\t\treturn memhash(sb.buf, sb.len);\n> +\t} else {\n> +\t\treturn memhash(line->line, line->len);\n> +\t}\n> +}\n\nCan be written without the \"else\".\n\n> +test_expect_success 'move detection does not mess up colored words' '\n\nProbably name this test \"--color-moved has no effect when used with\n--word-diff\".\n"},{"id":"322170","messageId":"CAGZ79kY9ByaHe_fx+S3neT6+1LcA_Q13bbVha8AKHUUO32FgeA@mail.gmail.com","threadId":"46038","inReplyTo":"20170613145457.2e888c27@twelve2.svl.corp.google.com","subject":"Re: [PATCHv5 04/17] diff: introduce more flexible emit function","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-13T23:41:57Z","receivedAt":"2017-06-13T23:42:03Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Jun 13, 2017 at 2:54 PM, Jonathan Tan wrote:\n> On Wed, 24 May 2017 14:40:23 -0700\n> Stefan Beller wrote:\n>\n>> Currently, diff output is written either through the emit_line_0\n>> function or through the FILE * in struct diff_options directly. To\n>> make it easier to teach diff to buffer its output (which will be done\n>> in a subsequent commit), introduce a more flexible emit_line() function.\n>> In this commit, direct usages of emit_line_0() are replaced with\n>> emit_line(); subsequent commits will also replace usages of the\n>> FILE * with emit().\n>\n> Check the names of the functions in this paragraph.\n\nOnly the very last word needed replacement of /s/emit/emit_line/.\n\n>\n>> diff --git a/diff.c b/diff.c\n>> index 2f9722b382..3569857818 100644\n>> --- a/diff.c\n>> +++ b/diff.c\n>> @@ -516,36 +516,30 @@ static void check_blank_at_eof(mmfile_t *mf1, mmfile_t *mf2,\n>> ecbdata->blank_at_eof_in_postimage = (at - l2) + 1;\n>> }\n>>\n>> -static void emit_line_0(struct diff_options *o, const char *set, const char *reset,\n>> - int first, const char *line, int len)\n>> +static void emit_line(struct diff_options *o, const char *set, const char *reset,\n>> + int add_line_prefix, int sign, const char *line, int len)\n>\n> In the future, this function is going to be used even to emit partial\n> lines\n\nYes.\n\n> - could this be called emit() instead?\n\nDespite having good IDEs available some (including me)\nvery much like working with raw text, and then having a function\nnamed as a common string doesn't help.\n\nAfter this patch\n\n $ git grep emit_line |wc -l\n 16\n # not all are this function, there is\n emit_line_checked as well. But 16 is not too much.\n\nBut if renamed to emit():\n\n $ git grep emit -- diff.c |wc -l\n 60\n\nYou could argue I'd just have to grep\nfor \"emit (\" instead, but that then I would have\nrely on correct whitespacing or use a regex already.\nComplexity which I would not like.\n\nSo I am not sure if this is helping a reader. (Not the casual\nreader, but the one grepping for this function)\n\nMaybe we can settle on a different name though,\nsuch as emit_string which is not a prefix of a dozen\ndifferent other functions?\n\nThanks,\nStefan\n"},{"id":"322172","messageId":"20170613164618.30804543@twelve2.svl.corp.google.com","threadId":"46038","inReplyTo":"CAGZ79kY9ByaHe_fx+S3neT6+1LcA_Q13bbVha8AKHUUO32FgeA@mail.gmail.com","subject":"Re: [PATCHv5 04/17] diff: introduce more flexible emit function","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-06-13T23:46:18Z","receivedAt":"2017-06-13T23:46:33Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Tue, 13 Jun 2017 16:41:57 -0700\nStefan Beller wrote:\n\n> On Tue, Jun 13, 2017 at 2:54 PM, Jonathan Tan wrote:\n> > - could this be called emit() instead?\n> \n> Despite having good IDEs available some (including me)\n> very much like working with raw text, and then having a function\n> named as a common string doesn't help.\n> \n> After this patch\n> \n> $ git grep emit_line |wc -l\n> 16\n> # not all are this function, there is\n> emit_line_checked as well. But 16 is not too much.\n> \n> But if renamed to emit():\n> \n> $ git grep emit -- diff.c |wc -l\n> 60\n> \n> You could argue I'd just have to grep\n> for \"emit (\" instead, but that then I would have\n> rely on correct whitespacing or use a regex already.\n> Complexity which I would not like.\n> \n> So I am not sure if this is helping a reader. (Not the casual\n> reader, but the one grepping for this function)\n> \n> Maybe we can settle on a different name though,\n> such as emit_string which is not a prefix of a dozen\n> different other functions?\n\nemit_string sounds good to me.\n"},{"id":"322183","messageId":"CAGZ79kZEcURkeBv2T4D3cFOQ7Tnzc-GVTfk_QT6K9iW1hvA55g@mail.gmail.com","threadId":"46038","inReplyTo":"20170613150722.0bd8fa56@twelve2.svl.corp.google.com","subject":"Re: [PATCHv5 16/17] diff: buffer all output if asked to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-14T02:52:52Z","receivedAt":"2017-06-14T02:52:58Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Jun 13, 2017 at 3:07 PM, Jonathan Tan wrote:\n\n>> +struct diff_line {\n>\n> Probably should be called diff_emission (or just emission), since these\n> may not be full lines.\n\nI think emitted_string would do as well?\n\n>\n> Also, can this definition be in the .c file? Callers should use the\n> diff_emit_line() below, and not need to know how it is implemented\n> internally.\n\ndone\n\n>> +#define diff_line_INIT {NULL, NULL, NULL, 0, 0, 0}\n>\n> Should be DIFF_LINE_INIT (capitalization), and {NULL} is sufficient, I\n> think.\n\ndone.\n"},{"id":"322284","messageId":"CAGZ79kaqjQYmkt77kk5m=fdBfbZAvwd0YhhT7=O5b-FkQmDfHg@mail.gmail.com","threadId":"46038","inReplyTo":"20170613155158.2454d9c7@twelve2.svl.corp.google.com","subject":"Re: [PATCH] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-14T18:55:12Z","receivedAt":"2017-06-14T18:55:28Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Jun 13, 2017 at 3:51 PM, Jonathan Tan wrote:\n> On Wed, 31 May 2017 17:24:29 -0700\n> Stefan Beller wrote:\n>\n>> When a patch consists mostly of moving blocks of code around, it can\n>> be quite tedious to ensure that the blocks are moved verbatim, and not\n>> undesirably modified in the move. To that end, color blocks that are\n>> moved within the same patch differently. For example (OM, del, add,\n>> and NM are different colors):\n>\n> [snip]\n>\n> Junio asks \"are we happy with these changes\" [1] and my answer is, in\n> general, yes - this seems like a very useful feature to have, and I'm OK\n> with the current design.\n>\n> I do feel a bit of unease at how the emitted strings are collected\n> without many guarantees as to their contents (e.g. whether they are full\n> lines or even whether they originate from the text of a file), but this\n> is already true for the existing code. The potential danger is that we\n> are now relying more on the format of these strings, but we don't plan\n> to do anything other than to color them, so this seems fine.\n\nI will add comments into the code for that.\n\n>\n> I would also prefer if there was only one coloring method, to ease\n> testing, but I can tolerate the current multiplicity of options.\n\nI *think* by now everyone involved in the discussion agrees that we\nwant Zebra + optional aggressive dimming (inside blocks as well as\nat bounds that are not adjacent to other blocks, i.e. anything\nnon-adjacent to a different block)\n"},{"id":"322643","messageId":"20170620024816.20021-1-sbeller@google.com","threadId":"46038","inReplyTo":"CAGZ79kaqjQYmkt77kk5m=fdBfbZAvwd0YhhT7=O5b-FkQmDfHg@mail.gmail.com","subject":"[PATCH 00/26] reroll of sb/diff-color-moved","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:50Z","receivedAt":"2017-06-20T02:48:34Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This is a complete rewrite of the series. Highlights:\n* instead of buffering partial lines, we'll pretend all diff output\n follows a well defined grammar, and we emit symbols thereof.\n (The difference is mostly mental, though by this trick we reduce\n the memory footprint for storing one of these symbols from 7 variables\n (3 pointers, 3 ints, one state (also int) down to 4 variables\n (one pointer, 2 ints, one state).\n* The algorithm for color painting was detangled:\n -> different functions for block detection and dimming\n -> The last patch (not to be applied) is an RFC that shows\n how we would approach non-colored, but machine parseable highlighting\n of moved lines.\n\nThanks,\nStefan\n\nStefan Beller (26):\n diff.c: readability fix\n diff.c: move line ending check into emit_hunk_header\n diff.c: factor out diff_flush_patch_all_file_pairs\n diff.c: introduce emit_diff_symbol\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF\n diff.c: migrate emit_line_checked to use emit_diff_symbol\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS{_PORCELAIN}\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF\n submodule.c: migrate diff output to use emit_diff_symbol\n diff.c: convert emit_binary_diff_body to use emit_diff_symbol\n diff.c: convert show_stats to use emit_diff_symbol\n diff.c: convert word diffing to use emit_diff_symbol\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY\n diff.c: buffer all output if asked to\n diff.c: color moved lines differently\n diff.c: color moved lines differently, plain mode\n diff.c: add dimming to moved line detection\n diff: document the new --color-moved setting\n WIP/RFC: diff.c: have a \"machine parseable\" move coloring\n\n Documentation/config.txt | 12 +-\n Documentation/diff-options.txt | 27 +\n cache.h | 1 +\n color.h | 2 +\n diff.c | 1283 ++++++++++++++++++++++++++++++++--------\n diff.h | 39 +-\n submodule.c | 85 ++-\n submodule.h | 13 +-\n t/t4015-diff-whitespace.sh | 369 ++++++++++++\n 9 files changed, 1515 insertions(+), 316 deletions(-)\n\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322644","messageId":"20170620024816.20021-3-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 02/26] diff.c: move line ending check into emit_hunk_header","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:52Z","receivedAt":"2017-06-20T02:48:36Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The emit_hunk_header() function is responsible for assembling a\nhunk header and calling emit_line() to send the hunk header\nto the output file. Its only caller fn_out_consume() needs\nto prepare for a case where the function emits an incomplete\nline and add the terminating LF.\n\nInstead make sure emit_hunk_header() to always send a\ncompleted line to emit_line().\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 3f5bf8b5a4..c2ed605cd0 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -677,6 +677,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t}\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n+\tstrbuf_complete_line(&msgbuf);\n+\n \temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n@@ -1315,8 +1317,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\tlen = sane_truncate_line(ecbdata, line, len);\n \t\tfind_lno(line, ecbdata);\n \t\temit_hunk_header(ecbdata, line, len);\n-\t\tif (line[len-1] != '\\n')\n-\t\t\tputc('\\n', o->file);\n \t\treturn;\n \t}\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322645","messageId":"20170620024816.20021-4-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 03/26] diff.c: factor out diff_flush_patch_all_file_pairs","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:53Z","receivedAt":"2017-06-20T02:48:37Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we want to do more things before and after all filepairs\nare flushed. So factor flushing out all file pairs into its own function\nthat the new code can be plugged in easily.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 17 ++++++++++++-----\n 1 file changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex c2ed605cd0..2f9722b382 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4737,6 +4737,17 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n \t\twarning(_(rename_limit_advice), varname, needed);\n }\n \n+static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n+{\n+\tint i;\n+\tstruct diff_queue_struct *q = &diff_queued_diff;\n+\tfor (i = 0; i < q->nr; i++) {\n+\t\tstruct diff_filepair *p = q->queue[i];\n+\t\tif (check_pair_status(p))\n+\t\t\tdiff_flush_patch(p, o);\n+\t}\n+}\n+\n void diff_flush(struct diff_options *options)\n {\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n@@ -4831,11 +4842,7 @@ void diff_flush(struct diff_options *options)\n \t\t\t}\n \t\t}\n \n-\t\tfor (i = 0; i < q->nr; i++) {\n-\t\t\tstruct diff_filepair *p = q->queue[i];\n-\t\t\tif (check_pair_status(p))\n-\t\t\t\tdiff_flush_patch(p, options);\n-\t\t}\n+\t\tdiff_flush_patch_all_file_pairs(options);\n \t}\n \n \tif (output_format & DIFF_FORMAT_CALLBACK)\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322646","messageId":"20170620024816.20021-7-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 06/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:56Z","receivedAt":"2017-06-20T02:48:39Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 3af07fa659..a1804a7785 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -562,6 +562,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n enum diff_symbol {\n \tDIFF_SYMBOL_SEPARATOR,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n+\tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n };\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n@@ -579,6 +580,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line(o, context, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -704,8 +708,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n-\n-\temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n+\temit_diff_symbol(ecbdata->opt,\n+\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322647","messageId":"20170620024816.20021-8-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 07/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:57Z","receivedAt":"2017-06-20T02:48:40Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 19 +++++++++++--------\n 1 file changed, 11 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex a1804a7785..28be3226c2 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -563,11 +563,13 @@ enum diff_symbol {\n \tDIFF_SYMBOL_SEPARATOR,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n+\tDIFF_SYMBOL_NO_LF_EOF,\n };\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len)\n {\n+\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_SEPARATOR:\n@@ -583,6 +585,13 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_NO_LF_EOF:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tputc('\\n', o->file);\n+\t\temit_line_0(o, context, reset, '\\\\',\n+\t\t\t nneof, strlen(nneof));\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -750,7 +759,6 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\t\t int prefix, const char *data, int size)\n {\n \tconst char *endp = NULL;\n-\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);\n \n \twhile (0 < size) {\n@@ -768,13 +776,8 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tsize -= len;\n \t\tdata += len;\n \t}\n-\tif (!endp) {\n-\t\tconst char *context = diff_get_color(ecb->color_diff,\n-\t\t\t\t\t\t DIFF_CONTEXT);\n-\t\tputc('\\n', ecb->opt->file);\n-\t\temit_line_0(ecb->opt, context, reset, '\\\\',\n-\t\t\t nneof, strlen(nneof));\n-\t}\n+\tif (!endp)\n+\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);\n }\n \n static void emit_rewrite_diff(const char *name_a,\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322648","messageId":"20170620024816.20021-9-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:58Z","receivedAt":"2017-06-20T02:48:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a new flags field to emit_diff_symbol, that will be used by\ncontext lines for:\n* white space rules that are applicable (The first 12 bits)\n* how the rules are evaluated (actually this double encodes the sign\n of the line, but the code is easier to keep this way, bits 13,14,15)\n* if the line a blank line at EOF (bit 16)\n\n\nSigned-off-by: Stefan Beller \n---\n cache.h | 1 +\n diff.c | 112 ++++++++++++++++++++++++++++++++++++++++------------------------\n diff.h | 6 ++--\n 3 files changed, 75 insertions(+), 44 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex e1f0e182ad..4d63c44f07 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -2168,6 +2168,7 @@ void shift_tree_by(const struct object_id *, const struct object_id *, struct ob\n #define WS_TRAILING_SPACE (WS_BLANK_AT_EOL|WS_BLANK_AT_EOF)\n #define WS_DEFAULT_RULE (WS_TRAILING_SPACE|WS_SPACE_BEFORE_TAB|8)\n #define WS_TAB_WIDTH_MASK 077\n+#define WS_RULE_MASK 07777\n extern unsigned whitespace_rule_cfg;\n extern unsigned whitespace_rule(const char *);\n extern unsigned parse_whitespace_rule(const char *);\ndiff --git a/diff.c b/diff.c\nindex 28be3226c2..42c9f48dc2 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -564,13 +564,50 @@ enum diff_symbol {\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_NO_LF_EOF,\n+\tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_PLUS,\n+\tDIFF_SYMBOL_MINUS,\n };\n+/*\n+ * Flags for content lines:\n+ * 0..12 are whitespace rules\n+ * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT\n+ * 16 is marking if the line is blank at EOF\n+ */\n+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n+#define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n+\n+static void emit_line_ws_markup(struct diff_options *o,\n+\t\t\t\tconst char *set, const char *reset,\n+\t\t\t\tconst char *line, int len, char sign,\n+\t\t\t\tunsigned ws_rule, int blank_at_eof)\n+{\n+\tconst char *ws = NULL;\n+\n+\tif (o->ws_error_highlight & ws_rule) {\n+\t\tws = diff_get_color_opt(o, DIFF_WHITESPACE);\n+\t\tif (!*ws)\n+\t\t\tws = NULL;\n+\t}\n+\n+\tif (!ws)\n+\t\temit_line_0(o, set, reset, sign, line, len);\n+\telse if (blank_at_eof)\n+\t\t/* Blank line at EOF - paint '+' as well */\n+\t\temit_line_0(o, ws, reset, sign, line, len);\n+\telse {\n+\t\t/* Emit just the prefix, then the rest. */\n+\t\temit_line_0(o, set, reset, sign, \"\", 0);\n+\t\tws_check_emit(line, len, ws_rule,\n+\t\t\t o->file, set, reset, ws);\n+\t}\n+}\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n-\t\t\t const char *line, int len)\n+\t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset;\n+\tconst char *context, *reset, *set;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n@@ -592,6 +629,25 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_0(o, context, reset, '\\\\',\n \t\t\t nneof, strlen(nneof));\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT:\n+\t\tset = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, ' ',\n+\t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_PLUS:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, '+',\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_MINUS:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, '-',\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -608,57 +664,31 @@ static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line\n \treturn ws_blank_line(line, len, ecbdata->ws_rule);\n }\n \n-static void emit_line_checked(const char *reset,\n-\t\t\t struct emit_callback *ecbdata,\n-\t\t\t const char *line, int len,\n-\t\t\t enum color_diff color,\n-\t\t\t unsigned ws_error_highlight,\n-\t\t\t char sign)\n-{\n-\tconst char *set = diff_get_color(ecbdata->color_diff, color);\n-\tconst char *ws = NULL;\n-\n-\tif (ecbdata->opt->ws_error_highlight & ws_error_highlight) {\n-\t\tws = diff_get_color(ecbdata->color_diff, DIFF_WHITESPACE);\n-\t\tif (!*ws)\n-\t\t\tws = NULL;\n-\t}\n-\n-\tif (!ws)\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, line, len);\n-\telse if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))\n-\t\t/* Blank line at EOF - paint '+' as well */\n-\t\temit_line_0(ecbdata->opt, ws, reset, sign, line, len);\n-\telse {\n-\t\t/* Emit just the prefix, then the rest. */\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, \"\", 0);\n-\t\tws_check_emit(line, len, ecbdata->ws_rule,\n-\t\t\t ecbdata->opt->file, set, reset, ws);\n-\t}\n-}\n-\n static void emit_add_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_FILE_NEW, WSEH_NEW, '+');\n+\tunsigned flags = WSEH_NEW | ecbdata->ws_rule;\n+\tif (new_blank_line_at_eof(ecbdata, line, len))\n+\t\tflags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;\n+\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);\n }\n \n static void emit_del_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_FILE_OLD, WSEH_OLD, '-');\n+\tunsigned flags = WSEH_OLD | ecbdata->ws_rule;\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_MINUS, line, len, flags);\n }\n \n static void emit_context_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_CONTEXT, WSEH_CONTEXT, ' ');\n+\tunsigned flags = WSEH_CONTEXT | ecbdata->ws_rule;\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_CONTEXT, line, len, flags);\n }\n \n static void emit_hunk_header(struct emit_callback *ecbdata,\n@@ -682,7 +712,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n \t\temit_diff_symbol(ecbdata->opt,\n-\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len);\n+\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len, 0);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n@@ -718,7 +748,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n \temit_diff_symbol(ecbdata->opt,\n-\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);\n+\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len, 0);\n \tstrbuf_release(&msgbuf);\n }\n \n@@ -777,7 +807,7 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tdata += len;\n \t}\n \tif (!endp)\n-\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);\n+\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0, 0);\n }\n \n static void emit_rewrite_diff(const char *name_a,\n@@ -4866,7 +4896,7 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\ndiff --git a/diff.h b/diff.h\nindex 5be1ee77a7..8483ca0991 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -148,9 +148,9 @@ struct diff_options {\n \tint abbrev;\n \tint ita_invisible_in_index;\n /* white-space error highlighting */\n-#define WSEH_NEW 1\n-#define WSEH_CONTEXT 2\n-#define WSEH_OLD 4\n+#define WSEH_NEW (1<<12)\n+#define WSEH_CONTEXT (1<<13)\n+#define WSEH_OLD (1<<14)\n \tunsigned ws_error_highlight;\n \tconst char *prefix;\n \tint prefix_length;\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322649","messageId":"20170620024816.20021-12-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:01Z","receivedAt":"2017-06-20T02:48:47Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 42 +++++++++++++++++++++---------------------\n 1 file changed, 21 insertions(+), 21 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex e6029e8213..7c56150b56 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -570,6 +570,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_MINUS,\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n \tDIFF_SYMBOL_WORDS,\n+\tDIFF_SYMBOL_FILEPAIR,\n };\n /*\n * Flags for content lines:\n@@ -610,7 +611,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset, *set;\n+\tconst char *context, *reset, *set, *meta;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n@@ -676,6 +677,14 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t}\n \t\temit_line(o, context, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_FILEPAIR:\n+\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tfprintf(o->file, \"%s%s%s%s%s%s\\n\", diff_line_prefix(o), meta,\n+\t\t\tflags ? \"+++ \" : \"--- \",\n+\t\t\tline, reset,\n+\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -847,8 +856,6 @@ static void emit_rewrite_diff(const char *name_a,\n \t\t\t struct diff_options *o)\n {\n \tint lc_a, lc_b;\n-\tconst char *name_a_tab, *name_b_tab;\n-\tconst char *metainfo = diff_get_color(o->use_color, DIFF_METAINFO);\n \tconst char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n \tconst char *reset = diff_get_color(o->use_color, DIFF_RESET);\n \tstatic struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;\n@@ -868,8 +875,6 @@ static void emit_rewrite_diff(const char *name_a,\n \n \tname_a += (*name_a == '/');\n \tname_b += (*name_b == '/');\n-\tname_a_tab = strchr(name_a, ' ') ? \"\\t\" : \"\";\n-\tname_b_tab = strchr(name_b, ' ') ? \"\\t\" : \"\";\n \n \tstrbuf_reset(&a_name);\n \tstrbuf_reset(&b_name);\n@@ -896,11 +901,11 @@ static void emit_rewrite_diff(const char *name_a,\n \n \tlc_a = count_lines(data_one, size_one);\n \tlc_b = count_lines(data_two, size_two);\n-\tfprintf(o->file,\n-\t\t\"%s%s--- %s%s%s\\n%s%s+++ %s%s%s\\n%s%s@@ -\",\n-\t\tline_prefix, metainfo, a_name.buf, name_a_tab, reset,\n-\t\tline_prefix, metainfo, b_name.buf, name_b_tab, reset,\n-\t\tline_prefix, fraginfo);\n+\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, a_name.buf, a_name.len, 0);\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, b_name.buf, b_name.len, 1);\n+\n+\tfprintf(o->file, \"%s%s@@ -\", line_prefix, fraginfo);\n \tif (!o->irreversible_delete)\n \t\tprint_line_count(o->file, lc_a);\n \telse\n@@ -1369,10 +1374,8 @@ static void find_lno(const char *line, struct emit_callback *ecbdata)\n static void fn_out_consume(void *priv, char *line, unsigned long len)\n {\n \tstruct emit_callback *ecbdata = priv;\n-\tconst char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n-\tconst char *line_prefix = diff_line_prefix(o);\n \n \to->found_changes = 1;\n \n@@ -1383,15 +1386,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->label_path[0]) {\n-\t\tconst char *name_a_tab, *name_b_tab;\n-\n-\t\tname_a_tab = strchr(ecbdata->label_path[0], ' ') ? \"\\t\" : \"\";\n-\t\tname_b_tab = strchr(ecbdata->label_path[1], ' ') ? \"\\t\" : \"\";\n-\n-\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);\n-\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR,\n+\t\t\t\t ecbdata->label_path[0],\n+\t\t\t\t strlen(ecbdata->label_path[0]), 0);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR,\n+\t\t\t\t ecbdata->label_path[1],\n+\t\t\t\t strlen(ecbdata->label_path[1]), 1);\n \t\tecbdata->label_path[0] = ecbdata->label_path[1] = NULL;\n \t}\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322650","messageId":"20170620024816.20021-5-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:54Z","receivedAt":"2017-06-20T02:48:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 22 +++++++++++++++++++---\n 1 file changed, 19 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2f9722b382..89466018e5 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n \temit_line_0(o, set, reset, line[0], line+1, len-1);\n }\n \n+enum diff_symbol {\n+\tDIFF_SYMBOL_SEPARATOR,\n+};\n+\n+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n+\t\t\t const char *line, int len)\n+{\n+\tswitch (s) {\n+\tcase DIFF_SYMBOL_SEPARATOR:\n+\t\tfprintf(o->file, \"%s%c\",\n+\t\t\tdiff_line_prefix(o),\n+\t\t\to->line_termination);\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: unknown diff symbol\");\n+\t}\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -4833,9 +4851,7 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\tfprintf(options->file, \"%s%c\",\n-\t\t\t\tdiff_line_prefix(options),\n-\t\t\t\toptions->line_termination);\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322651","messageId":"20170620024816.20021-16-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 15/26] submodule.c: migrate diff output to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:05Z","receivedAt":"2017-06-20T02:48:51Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"As the submodule process is no longer attached to the same stdout as\nthe superprojects process, we need to pass coloring explicitly.\n\nRemove the colors from the function signatures, as all the coloring\ndecisions will be made either inside the child process or the final\nemit_diff_symbol.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++---------\n diff.h | 9 +++++++\n submodule.c | 85 ++++++++++++++++++++++++++--------------------------------\n submodule.h | 13 +++------\n 4 files changed, 128 insertions(+), 68 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 96ce53c5cf..42a9020d95 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -574,6 +574,13 @@ enum diff_symbol {\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_REWRITE_DIFF,\n+\tDIFF_SYMBOL_SUBMODULE_ADD,\n+\tDIFF_SYMBOL_SUBMODULE_DEL,\n+\tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\tDIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\tDIFF_SYMBOL_SUBMODULE_HEADER,\n+\tDIFF_SYMBOL_SUBMODULE_ERROR,\n+\tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n };\n /*\n * Flags for content lines:\n@@ -700,11 +707,77 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line(o, fraginfo, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_ADD:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, set, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_DEL:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, set, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_UNTRACKED:\n+\t\tfprintf(o->file, \"%sSubmodule %s contains untracked content\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_MODIFIED:\n+\t\tfprintf(o->file, \"%sSubmodule %s contains modified content\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n+\t\tfprintf(o->file, \"%s%s\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n }\n \n+void diff_emit_submodule_del(struct diff_options *o, const char *line)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);\n+}\n+\n+void diff_emit_submodule_add(struct diff_options *o, const char *line)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ADD, line, strlen(line), 0);\n+}\n+\n+void diff_emit_submodule_untracked(struct diff_options *o, const char *path)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\t\t\t path, strlen(path), 0);\n+}\n+\n+void diff_emit_submodule_modified(struct diff_options *o, const char *path)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\t\t\t path, strlen(path), 0);\n+}\n+\n+void diff_emit_submodule_header(struct diff_options *o, const char *header)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_HEADER,\n+\t\t\t header, strlen(header), 0);\n+}\n+\n+void diff_emit_submodule_error(struct diff_options *o, const char *err)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ERROR, err, strlen(err), 0);\n+}\n+\n+void diff_emit_submodule_pipethrough(struct diff_options *o,\n+\t\t\t\t const char *line, int len)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_PIPETHROUGH, line, len, 0);\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -2465,24 +2538,16 @@ static void builtin_diff(const char *name_a,\n \tif (o->submodule_format == DIFF_SUBMODULE_LOG &&\n \t (!one->mode || S_ISGITLINK(one->mode)) &&\n \t (!two->mode || S_ISGITLINK(two->mode))) {\n-\t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n-\t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_summary(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_summary(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n-\t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset);\n+\t\t\t\ttwo->dirty_submodule);\n \t\treturn;\n \t} else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&\n \t\t (!one->mode || S_ISGITLINK(one->mode)) &&\n \t\t (!two->mode || S_ISGITLINK(two->mode))) {\n-\t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n-\t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_inline_diff(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_inline_diff(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n-\t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset, o);\n+\t\t\t\ttwo->dirty_submodule);\n \t\treturn;\n \t}\n \ndiff --git a/diff.h b/diff.h\nindex 8483ca0991..2ee0ef3908 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,15 @@ struct diff_options {\n \tint diff_path_counter;\n };\n \n+void diff_emit_submodule_del(struct diff_options *o, const char *line);\n+void diff_emit_submodule_add(struct diff_options *o, const char *line);\n+void diff_emit_submodule_untracked(struct diff_options *o, const char *path);\n+void diff_emit_submodule_modified(struct diff_options *o, const char *path);\n+void diff_emit_submodule_header(struct diff_options *o, const char *header);\n+void diff_emit_submodule_error(struct diff_options *o, const char *err);\n+void diff_emit_submodule_pipethrough(struct diff_options *o,\n+\t\t\t\t const char *line, int len);\n+\n enum color_diff {\n \tDIFF_RESET = 0,\n \tDIFF_CONTEXT = 1,\ndiff --git a/submodule.c b/submodule.c\nindex d3299e29c0..71471f3241 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -362,9 +362,7 @@ static int prepare_submodule_summary(struct rev_info *rev, const char *path,\n \treturn prepare_revision_walk(rev);\n }\n \n-static void print_submodule_summary(struct rev_info *rev, FILE *f,\n-\t\tconst char *line_prefix,\n-\t\tconst char *del, const char *add, const char *reset)\n+static void print_submodule_summary(struct rev_info *rev, struct diff_options *o)\n {\n \tstatic const char format[] = \" %m %s\";\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -375,18 +373,12 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \t\tctx.date_mode = rev->date_mode;\n \t\tctx.output_encoding = get_log_output_encoding();\n \t\tstrbuf_setlen(&sb, 0);\n-\t\tstrbuf_addstr(&sb, line_prefix);\n-\t\tif (commit->object.flags & SYMMETRIC_LEFT) {\n-\t\t\tif (del)\n-\t\t\t\tstrbuf_addstr(&sb, del);\n-\t\t}\n-\t\telse if (add)\n-\t\t\tstrbuf_addstr(&sb, add);\n \t\tformat_commit_message(commit, format, &sb, &ctx);\n-\t\tif (reset)\n-\t\t\tstrbuf_addstr(&sb, reset);\n \t\tstrbuf_addch(&sb, '\\n');\n-\t\tfprintf(f, \"%s\", sb.buf);\n+\t\tif (commit->object.flags & SYMMETRIC_LEFT)\n+\t\t\tdiff_emit_submodule_del(o, sb.buf);\n+\t\telse\n+\t\t\tdiff_emit_submodule_add(o, sb.buf);\n \t}\n \tstrbuf_release(&sb);\n }\n@@ -413,11 +405,9 @@ void prepare_submodule_repo_env(struct argv_array *out)\n * attempt to lookup both the left and right commits and put them into the\n * left and right pointers.\n */\n-static void show_submodule_header(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+static void show_submodule_header(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *reset,\n+\t\tunsigned dirty_submodule,\n \t\tstruct commit **left, struct commit **right,\n \t\tstruct commit_list **merge_bases)\n {\n@@ -426,11 +416,10 @@ static void show_submodule_header(FILE *f, const char *path,\n \tint fast_forward = 0, fast_backward = 0;\n \n \tif (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED)\n-\t\tfprintf(f, \"%sSubmodule %s contains untracked content\\n\",\n-\t\t\tline_prefix, path);\n+\t\tdiff_emit_submodule_untracked(o, path);\n+\n \tif (dirty_submodule & DIRTY_SUBMODULE_MODIFIED)\n-\t\tfprintf(f, \"%sSubmodule %s contains modified content\\n\",\n-\t\t\tline_prefix, path);\n+\t\tdiff_emit_submodule_modified(o, path);\n \n \tif (is_null_oid(one))\n \t\tmessage = \"(new submodule)\";\n@@ -472,31 +461,29 @@ static void show_submodule_header(FILE *f, const char *path,\n \t}\n \n output_header:\n-\tstrbuf_addf(&sb, \"%s%sSubmodule %s \", line_prefix, meta, path);\n+\tstrbuf_addf(&sb, \"Submodule %s \", path);\n \tstrbuf_add_unique_abbrev(&sb, one->hash, DEFAULT_ABBREV);\n \tstrbuf_addstr(&sb, (fast_backward || fast_forward) ? \"..\" : \"...\");\n \tstrbuf_add_unique_abbrev(&sb, two->hash, DEFAULT_ABBREV);\n \tif (message)\n-\t\tstrbuf_addf(&sb, \" %s%s\\n\", message, reset);\n+\t\tstrbuf_addf(&sb, \" %s\\n\", message);\n \telse\n-\t\tstrbuf_addf(&sb, \"%s:%s\\n\", fast_backward ? \" (rewind)\" : \"\", reset);\n-\tfwrite(sb.buf, sb.len, 1, f);\n+\t\tstrbuf_addf(&sb, \"%s:\\n\", fast_backward ? \" (rewind)\" : \"\");\n+\tdiff_emit_submodule_header(o, sb.buf);\n \n \tstrbuf_release(&sb);\n }\n \n-void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset)\n+\t\tunsigned dirty_submodule)\n {\n \tstruct rev_info rev;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n-\t\t\t meta, reset, &left, &right, &merge_bases);\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n+\t\t\t &left, &right, &merge_bases);\n \n \t/*\n \t * If we don't have both a left and a right pointer, there is no\n@@ -508,11 +495,11 @@ void show_submodule_summary(FILE *f, const char *path,\n \n \t/* Treat revision walker failure the same as missing commits */\n \tif (prepare_submodule_summary(&rev, path, left, right, merge_bases)) {\n-\t\tfprintf(f, \"%s(revision walker failed)\\n\", line_prefix);\n+\t\tdiff_emit_submodule_error(o, \"(revision walker failed)\\n\");\n \t\tgoto out;\n \t}\n \n-\tprint_submodule_summary(&rev, f, line_prefix, del, add, reset);\n+\tprint_submodule_summary(&rev, o);\n \n out:\n \tif (merge_bases)\n@@ -521,21 +508,18 @@ void show_submodule_summary(FILE *f, const char *path,\n \tclear_commit_marks(right, ~0);\n }\n \n-void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *o)\n+\t\tunsigned dirty_submodule)\n {\n \tconst struct object_id *old = &empty_tree_oid, *new = &empty_tree_oid;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n-\tstruct strbuf submodule_dir = STRBUF_INIT;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n-\t\t\t meta, reset, &left, &right, &merge_bases);\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n+\t\t\t &left, &right, &merge_bases);\n \n \t/* We need a valid left and right commit to display a difference */\n \tif (!(left || is_null_oid(one)) ||\n@@ -547,15 +531,16 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \tif (right)\n \t\tnew = two;\n \n-\tfflush(f);\n \tcp.git_cmd = 1;\n \tcp.dir = path;\n-\tcp.out = dup(fileno(f));\n+\tcp.out = -1;\n \tcp.no_stdin = 1;\n \n \t/* TODO: other options may need to be passed here. */\n \targv_array_push(&cp.args, \"diff\");\n-\targv_array_pushf(&cp.args, \"--line-prefix=%s\", line_prefix);\n+\targv_array_pushf(&cp.args, \"--color=%s\", want_color(o->use_color) ?\n+\t\t\t \"always\" : \"never\");\n+\n \tif (DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\targv_array_pushf(&cp.args, \"--src-prefix=%s%s/\",\n \t\t\t\t o->b_prefix, path);\n@@ -578,11 +563,17 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \t\targv_array_push(&cp.args, oid_to_hex(new));\n \n \tprepare_submodule_repo_env(&cp.env_array);\n-\tif (run_command(&cp))\n-\t\tfprintf(f, \"(diff failed)\\n\");\n+\tif (start_command(&cp))\n+\t\tdiff_emit_submodule_error(o, \"(diff failed)\\n\");\n+\n+\twhile (strbuf_getwholeline_fd(&sb, cp.out, '\\n') != EOF)\n+\t\tdiff_emit_submodule_pipethrough(o, sb.buf, sb.len);\n+\n+\tif (finish_command(&cp))\n+\t\tdiff_emit_submodule_error(o, \"(diff failed)\\n\");\n \n done:\n-\tstrbuf_release(&submodule_dir);\n+\tstrbuf_release(&sb);\n \tif (merge_bases)\n \t\tfree_commit_list(merge_bases);\n \tif (left)\ndiff --git a/submodule.h b/submodule.h\nindex 1277480add..e7358863e9 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -53,17 +53,12 @@ extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n extern void handle_ignore_submodules_arg(struct diff_options *, const char *);\n-extern void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+extern void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset);\n-extern void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+\t\tunsigned dirty_submodule);\n+extern void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *opt);\n+\t\tunsigned dirty_submodule);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n /* Check if we want to update any submodule.*/\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322652","messageId":"20170620024816.20021-20-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 19/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:09Z","receivedAt":"2017-06-20T02:48:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 10 +++++++---\n 1 file changed, 7 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 5b46baa12a..18bfc4720b 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -591,6 +591,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n \tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_WORD_DIFF,\n+\tDIFF_SYMBOL_STAT_SEP,\n };\n /*\n * Flags for content lines:\n@@ -776,6 +777,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_WORD_DIFF:\n \t\tfprintf(o->file, \"%.*s\", len, line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_STAT_SEP:\n+\t\tfputs(o->stat_sep, o->file);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -5085,10 +5089,10 @@ void diff_flush(struct diff_options *options)\n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n \t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);\n-\t\t\tif (options->stat_sep) {\n+\t\t\tif (options->stat_sep)\n \t\t\t\t/* attach patch instead of inline */\n-\t\t\t\tfputs(options->stat_sep, options->file);\n-\t\t\t}\n+\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STAT_SEP,\n+\t\t\t\t\t\t NULL, 0, 0);\n \t\t}\n \n \t\tdiff_flush_patch_all_file_pairs(options);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322653","messageId":"20170620024816.20021-13-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 12/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:02Z","receivedAt":"2017-06-20T02:48:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 28 ++++++++++++++++++++--------\n 1 file changed, 20 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 7c56150b56..95f11ea86f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -571,6 +571,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n \tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_FILEPAIR,\n+\tDIFF_SYMBOL_HEADER,\n };\n /*\n * Flags for content lines:\n@@ -685,6 +686,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tline, reset,\n \t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n+\tcase DIFF_SYMBOL_HEADER:\n+\t\tfprintf(o->file, \"%s\", line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1380,7 +1384,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \to->found_changes = 1;\n \n \tif (ecbdata->header) {\n-\t\tfprintf(o->file, \"%s\", ecbdata->header->buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t ecbdata->header->buf, ecbdata->header->len, 0);\n \t\tstrbuf_reset(ecbdata->header);\n \t\tecbdata->header = NULL;\n \t}\n@@ -2514,7 +2519,8 @@ static void builtin_diff(const char *name_a,\n \t\tif (complete_rewrite &&\n \t\t (textconv_one || !diff_filespec_is_binary(one)) &&\n \t\t (textconv_two || !diff_filespec_is_binary(two))) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tstrbuf_reset(&header);\n \t\t\temit_rewrite_diff(name_a, name_b, one, two,\n \t\t\t\t\t\ttextconv_one, textconv_two, o);\n@@ -2524,7 +2530,8 @@ static void builtin_diff(const char *name_a,\n \t}\n \n \tif (o->irreversible_delete && lbl[1][0] == '/') {\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf,\n+\t\t\t\t header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tgoto free_ab_and_return;\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n@@ -2535,10 +2542,13 @@ static void builtin_diff(const char *name_a,\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n \t\t\tif (!oidcmp(&one->oid, &two->oid)) {\n \t\t\t\tif (must_show_header)\n-\t\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t\t\t header.buf, header.len,\n+\t\t\t\t\t\t\t 0);\n \t\t\t\tgoto free_ab_and_return;\n \t\t\t}\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n \t\t\t\tline_prefix, lbl[0], lbl[1]);\n \t\t\tgoto free_ab_and_return;\n@@ -2549,10 +2559,11 @@ static void builtin_diff(const char *name_a,\n \t\tif (mf1.size == mf2.size &&\n \t\t !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {\n \t\t\tif (must_show_header)\n-\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n@@ -2570,7 +2581,8 @@ static void builtin_diff(const char *name_a,\n \t\tconst struct userdiff_funcname *pe;\n \n \t\tif (must_show_header) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tstrbuf_reset(&header);\n \t\t}\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322654","messageId":"20170620024816.20021-18-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 17/26] diff.c: convert show_stats to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:07Z","receivedAt":"2017-06-20T02:48:59Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We call print_stat_summary from builtin/apply, so we still\nneed the version with a file pointer, so introduce\nprint_stat_summary_0 that uses emit_string machinery and\nkeep print_stat_summary with the same arguments around.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 118 +++++++++++++++++++++++++++++++++++++++++------------------------\n diff.h | 4 +--\n 2 files changed, 77 insertions(+), 45 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex e6ade5fde0..0a5ff2500f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -586,7 +586,10 @@ enum diff_symbol {\n \tDIFF_SYMBOL_SUBMODULE_HEADER,\n \tDIFF_SYMBOL_SUBMODULE_ERROR,\n \tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n-\n+\tDIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n+\tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n+\tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n+\tDIFF_SYMBOL_STATS_LINE,\n };\n /*\n * Flags for content lines:\n@@ -628,6 +631,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n+\tstruct strbuf sb = STRBUF_INIT;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n@@ -756,9 +760,22 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:\n+\t\tfprintf(o->file, \" 0 files changed\\n\");\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n+\t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_LINE:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n+\tstrbuf_release(&sb);\n }\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line)\n@@ -1712,20 +1729,14 @@ static int scale_linear(int it, int width, int max_change)\n \treturn 1 + (it * (width - 1) / max_change);\n }\n \n-static void show_name(FILE *file,\n-\t\t const char *prefix, const char *name, int len)\n-{\n-\tfprintf(file, \" %s%-*s |\", prefix, len, name);\n-}\n-\n-static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)\n+static void show_graph(struct strbuf *out, char ch, int cnt,\n+\t\t const char *set, const char *reset)\n {\n \tif (cnt <= 0)\n \t\treturn;\n-\tfprintf(file, \"%s\", set);\n-\twhile (cnt--)\n-\t\tputc(ch, file);\n-\tfprintf(file, \"%s\", reset);\n+\tstrbuf_addstr(out, set);\n+\tstrbuf_addchars(out, ch, cnt);\n+\tstrbuf_addstr(out, reset);\n }\n \n static void fill_print_name(struct diffstat_file *file)\n@@ -1749,14 +1760,16 @@ static void fill_print_name(struct diffstat_file *file)\n \tfile->print_name = pname;\n }\n \n-int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n+static void print_stat_summary_inserts_deletes(struct diff_options *options,\n+\t\tint files, int insertions, int deletions)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint ret;\n \n \tif (!files) {\n \t\tassert(insertions == 0 && deletions == 0);\n-\t\treturn fprintf(fp, \"%s\\n\", \" 0 files changed\");\n+\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n+\t\t\t\t NULL, 0, 0);\n+\t\treturn;\n \t}\n \n \tstrbuf_addf(&sb,\n@@ -1783,9 +1796,19 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n \t\t\t deletions);\n \t}\n \tstrbuf_addch(&sb, '\\n');\n-\tret = fputs(sb.buf, fp);\n+\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n+\t\t\t sb.buf, sb.len, 0);\n \tstrbuf_release(&sb);\n-\treturn ret;\n+}\n+\n+void print_stat_summary(FILE *fp, int files,\n+\t\t\tint insertions, int deletions)\n+{\n+\tstruct diff_options o;\n+\tmemset(&o, 0, sizeof(o));\n+\to.file = fp;\n+\n+\tprint_stat_summary_inserts_deletes(&o, files, insertions, deletions);\n }\n \n static void show_stats(struct diffstat_t *data, struct diff_options *options)\n@@ -1795,13 +1818,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \tint total_files = data->nr, count;\n \tint width, name_width, graph_width, number_width = 0, bin_width = 0;\n \tconst char *reset, *add_c, *del_c;\n-\tconst char *line_prefix = \"\";\n \tint extra_shown = 0;\n+\tconst char *line_prefix = diff_line_prefix(options);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (data->nr == 0)\n \t\treturn;\n \n-\tline_prefix = diff_line_prefix(options);\n \tcount = options->stat_count ? options->stat_count : data->nr;\n \n \treset = diff_get_color_opt(options, DIFF_RESET);\n@@ -1955,26 +1978,32 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t}\n \n \t\tif (file->is_binary) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" %*s\", number_width, \"Bin\");\n+\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\t\tstrbuf_addf(&out, \" %*s\", number_width, \"Bin\");\n \t\t\tif (!added && !deleted) {\n-\t\t\t\tputc('\\n', options->file);\n+\t\t\t\tstrbuf_addch(&out, '\\n');\n+\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\t\tstrbuf_reset(&out);\n \t\t\t\tcontinue;\n \t\t\t}\n-\t\t\tfprintf(options->file, \" %s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addf(&out, \" %s%\"PRIuMAX\"%s\",\n \t\t\t\tdel_c, deleted, reset);\n-\t\t\tfprintf(options->file, \" -> \");\n-\t\t\tfprintf(options->file, \"%s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addstr(&out, \" -> \");\n+\t\t\tstrbuf_addf(&out, \"%s%\"PRIuMAX\"%s\",\n \t\t\t\tadd_c, added, reset);\n-\t\t\tfprintf(options->file, \" bytes\");\n-\t\t\tfprintf(options->file, \"\\n\");\n+\t\t\tstrbuf_addstr(&out, \" bytes\\n\");\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \t\telse if (file->is_unmerged) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" Unmerged\\n\");\n+\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\t\tstrbuf_addstr(&out, \" Unmerged\\n\");\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \n@@ -1997,14 +2026,16 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t\t\tadd = total - del;\n \t\t\t}\n \t\t}\n-\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\tshow_name(options->file, prefix, name, len);\n-\t\tfprintf(options->file, \" %*\"PRIuMAX\"%s\",\n+\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\tstrbuf_addf(&out, \" %*\"PRIuMAX\"%s\",\n \t\t\tnumber_width, added + deleted,\n \t\t\tadded + deleted ? \" \" : \"\");\n-\t\tshow_graph(options->file, '+', add, add_c, reset);\n-\t\tshow_graph(options->file, '-', del, del_c, reset);\n-\t\tfprintf(options->file, \"\\n\");\n+\t\tshow_graph(&out, '+', add, add_c, reset);\n+\t\tshow_graph(&out, '-', del, del_c, reset);\n+\t\tstrbuf_addch(&out, '\\n');\n+\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t out.buf, out.len, 0);\n+\t\tstrbuf_reset(&out);\n \t}\n \n \tfor (i = 0; i < data->nr; i++) {\n@@ -2025,11 +2056,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\tif (i < count)\n \t\t\tcontinue;\n \t\tif (!extra_shown)\n-\t\t\tfprintf(options->file, \"%s ...\\n\", line_prefix);\n+\t\t\temit_diff_symbol(options,\n+\t\t\t\t\t DIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n+\t\t\t\t\t NULL, 0, 0);\n \t\textra_shown = 1;\n \t}\n-\tfprintf(options->file, \"%s\", line_prefix);\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\n+\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n }\n \n static void show_shortstats(struct diffstat_t *data, struct diff_options *options)\n@@ -2041,7 +2074,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \n \tfor (i = 0; i < data->nr; i++) {\n \t\tint added = data->files[i]->added;\n-\t\tint deleted= data->files[i]->deleted;\n+\t\tint deleted = data->files[i]->deleted;\n \n \t\tif (data->files[i]->is_unmerged ||\n \t\t (!data->files[i]->is_interesting && (added + deleted == 0))) {\n@@ -2051,8 +2084,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \t\t\tdels += deleted;\n \t\t}\n \t}\n-\tfprintf(options->file, \"%s\", diff_line_prefix(options));\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n }\n \n static void show_numstat(struct diffstat_t *data, struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex 2ee0ef3908..b73f7a1e75 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -398,8 +398,8 @@ extern int parse_rename_score(const char **cp_p);\n \n extern long parse_algorithm_value(const char *value);\n \n-extern int print_stat_summary(FILE *fp, int files,\n-\t\t\t int insertions, int deletions);\n+extern void print_stat_summary(FILE *fp, int files,\n+\t\t\t int insertions, int deletions);\n extern void setup_diff_pager(struct diff_options *);\n \n #endif /* DIFF_H */\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322655","messageId":"20170620024816.20021-19-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 18/26] diff.c: convert word diffing to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:08Z","receivedAt":"2017-06-20T02:49:01Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The word diffing is not line oriented and would need some serious\neffort to be transformed into a line oriented approach, so\njust go with a symbol DIFF_SYMBOL_WORD_DIFF that is a partial line.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 79 ++++++++++++++++++++++++++++++++++++++----------------------------\n 1 file changed, 46 insertions(+), 33 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 0a5ff2500f..5b46baa12a 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -590,6 +590,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n \tDIFF_SYMBOL_STATS_LINE,\n+\tDIFF_SYMBOL_WORD_DIFF,\n };\n /*\n * Flags for content lines:\n@@ -772,6 +773,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_WORD_DIFF:\n+\t\tfprintf(o->file, \"%.*s\", len, line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1100,37 +1104,49 @@ struct diff_words_data {\n \tstruct diff_words_style *style;\n };\n \n-static int fn_out_diff_words_write_helper(FILE *fp,\n+static int fn_out_diff_words_write_helper(struct diff_options *o,\n \t\t\t\t\t struct diff_words_style_elem *st_el,\n \t\t\t\t\t const char *newline,\n-\t\t\t\t\t size_t count, const char *buf,\n-\t\t\t\t\t const char *line_prefix)\n+\t\t\t\t\t size_t count, const char *buf)\n {\n \tint print = 0;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n \twhile (count) {\n \t\tchar *p = memchr(buf, '\\n', count);\n \t\tif (print)\n-\t\t\tfputs(line_prefix, fp);\n+\t\t\tstrbuf_addstr(&sb, diff_line_prefix(o));\n+\n \t\tif (p != buf) {\n-\t\t\tif (st_el->color && fputs(st_el->color, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (fputs(st_el->prefix, fp) < 0 ||\n-\t\t\t fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||\n-\t\t\t fputs(st_el->suffix, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (st_el->color && *st_el->color\n-\t\t\t && fputs(GIT_COLOR_RESET, fp) < 0)\n-\t\t\t\treturn -1;\n+\t\t\tconst char *reset = st_el->color && *st_el->color ?\n+\t\t\t\t\t GIT_COLOR_RESET : NULL;\n+\t\t\tif (st_el->color && *st_el->color)\n+\t\t\t\tstrbuf_addstr(&sb, st_el->color);\n+\t\t\tstrbuf_addstr(&sb, st_el->prefix);\n+\t\t\tstrbuf_add(&sb, buf, p ? p - buf : count);\n+\t\t\tstrbuf_addstr(&sb, st_el->suffix);\n+\t\t\tif (reset)\n+\t\t\t\tstrbuf_addstr(&sb, reset);\n \t\t}\n \t\tif (!p)\n-\t\t\treturn 0;\n-\t\tif (fputs(newline, fp) < 0)\n-\t\t\treturn -1;\n+\t\t\tgoto out;\n+\n+\t\tstrbuf_addstr(&sb, newline);\n \t\tcount -= p + 1 - buf;\n \t\tbuf = p + 1;\n \t\tprint = 1;\n+\t\tif (count) {\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_reset(&sb);\n+\t\t}\n \t}\n+\n+out:\n+\tif (sb.len)\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\tstrbuf_release(&sb);\n \treturn 0;\n }\n \n@@ -1212,24 +1228,20 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)\n \t\tfputs(line_prefix, diff_words->opt->file);\n \t}\n \tif (diff_words->current_plus != plus_begin) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->ctx, style->newline,\n \t\t\t\tplus_begin - diff_words->current_plus,\n-\t\t\t\tdiff_words->current_plus, line_prefix);\n-\t\tif (*(plus_begin - 1) == '\\n')\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n+\t\t\t\tdiff_words->current_plus);\n \t}\n \tif (minus_begin != minus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->old, style->newline,\n-\t\t\t\tminus_end - minus_begin, minus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tminus_end - minus_begin, minus_begin);\n \t}\n \tif (plus_begin != plus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->new, style->newline,\n-\t\t\t\tplus_end - plus_begin, plus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tplus_end - plus_begin, plus_begin);\n \t}\n \n \tdiff_words->current_plus = plus_end;\n@@ -1323,11 +1335,12 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \n \t/* special case: only removal */\n \tif (!diff_words->plus.text.size) {\n-\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\temit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t line_prefix, strlen(line_prefix), 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->old, style->newline,\n \t\t\tdiff_words->minus.text.size,\n-\t\t\tdiff_words->minus.text.ptr, line_prefix);\n+\t\t\tdiff_words->minus.text.ptr);\n \t\tdiff_words->minus.text.size = 0;\n \t\treturn;\n \t}\n@@ -1350,12 +1363,12 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \tif (diff_words->current_plus != diff_words->plus.text.ptr +\n \t\t\tdiff_words->plus.text.size) {\n \t\tif (color_words_output_graph_prefix(diff_words))\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\t\temit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t\t line_prefix, strlen(line_prefix), 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->ctx, style->newline,\n \t\t\tdiff_words->plus.text.ptr + diff_words->plus.text.size\n-\t\t\t- diff_words->current_plus, diff_words->current_plus,\n-\t\t\tline_prefix);\n+\t\t\t- diff_words->current_plus, diff_words->current_plus);\n \t}\n \tdiff_words->minus.text.size = diff_words->plus.text.size = 0;\n }\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322656","messageId":"20170620024816.20021-26-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 25/26] diff: document the new --color-moved setting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:15Z","receivedAt":"2017-06-20T02:49:05Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n Documentation/config.txt | 12 ++++++++++--\n Documentation/diff-options.txt | 27 +++++++++++++++++++++++++++\n 2 files changed, 37 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 475e874d51..29e0b9fa69 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1051,14 +1051,22 @@ This does not affect linkgit:git-format-patch[1] or the\n 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n command line with the `--color[=]` option.\n \n+diff.colorMoved::\n+\tIf set moved lines in a diff are colored differently,\n+\tfor details see '--color-moved' in linkgit:git-diff[1].\n+\n color.diff.::\n \tUse customized color for diff colorization. `` specifies\n \twhich part of the patch to use the specified color, and is one\n \tof `context` (context text - `plain` is a historical synonym),\n \t`meta` (metainformation), `frag`\n \t(hunk header), 'func' (function in hunk header), `old` (removed lines),\n-\t`new` (added lines), `commit` (commit headers), or `whitespace`\n-\t(highlighting whitespace errors).\n+\t`new` (added lines), `commit` (commit headers), `whitespace`\n+\t(highlighting whitespace errors), `oldMoved` (deleted lines),\n+\t`newMoved` (added lines), `oldMovedDimmed`, `oldMovedAlternative`,\n+\t`oldMovedAlternativeDimmed`, `newMovedDimmed`, `newMovedAlternative`\n+\tand `newMovedAlternativeDimmed` (See the ''\n+\tsetting of '--color-moved' in linkgit:git-diff[1] for details).\n \n color.decorate.::\n \tUse customized color for 'git log --decorate' output. `` is one\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex 89cc0f48de..058c8014ed 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -231,6 +231,33 @@ ifdef::git-diff[]\n endif::git-diff[]\n \tIt is the same as `--color=never`.\n \n+--color-moved[=]::\n+\tMoved lines of code are colored differently.\n+ifdef::git-diff[]\n+\tIt can be changed by the `diff.colorMoved` configuration setting.\n+endif::git-diff[]\n+\tThe defaults to 'no' if the option is not given\n+\tand to 'dimmed_zebra' if the option with no mode is given.\n+\tThe mode must be one of:\n++\n+--\n+no::\n+\tMoved lines are not highlighted.\n+plain::\n+\tAny line that is added in one location and was removed\n+\tin another location will be colored with 'color.diff.newMoved'.\n+\tSimilarly 'color.diff.oldMoved' will be used for removed lines\n+\tthat are added somewhere else in the diff.\n+zebra::\n+\tBlocks of moved code are detected. The detected blocks are\n+\tpainted using the 'color.diff.{old,new}Moved' alternating with\n+\t'color.diff.{old,new}MovedAlternative'.\n+dimmed_zebra::\n+\tSimilar to 'zebra', but additional dimming of uninteresting parts\n+\tof moved code is performed. The bordering lines of two adjacent\n+\tblocks are considered interesting, the rest is uninteresting.\n+--\n+\n --word-diff[=]::\n \tShow a word diff, using the to delimit changed words.\n \tBy default, words are delimited by whitespace; see\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322657","messageId":"20170620024816.20021-15-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 14/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:04Z","receivedAt":"2017-06-20T02:49:07Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 35 +++++++++++++++++++++--------------\n 1 file changed, 21 insertions(+), 14 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex e56962b844..96ce53c5cf 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -573,6 +573,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_FILEPAIR,\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_BINARY_FILES,\n+\tDIFF_SYMBOL_REWRITE_DIFF,\n };\n /*\n * Flags for content lines:\n@@ -613,7 +614,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset, *set, *meta;\n+\tconst char *context, *reset, *set, *meta, *fraginfo;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n@@ -694,6 +695,11 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\tfprintf(o->file, \"%sBinary files %s differ\\n\",\n \t\t\tdiff_line_prefix(o), line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_REWRITE_DIFF:\n+\t\tfraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, fraginfo, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -816,17 +822,17 @@ static void remove_tempfile(void)\n \t}\n }\n \n-static void print_line_count(FILE *file, int count)\n+static void add_line_count(struct strbuf *out, int count)\n {\n \tswitch (count) {\n \tcase 0:\n-\t\tfprintf(file, \"0,0\");\n+\t\tstrbuf_addstr(out, \"0,0\");\n \t\tbreak;\n \tcase 1:\n-\t\tfprintf(file, \"1\");\n+\t\tstrbuf_addstr(out, \"1\");\n \t\tbreak;\n \tdefault:\n-\t\tfprintf(file, \"1,%d\", count);\n+\t\tstrbuf_addf(out, \"1,%d\", count);\n \t\tbreak;\n \t}\n }\n@@ -865,14 +871,12 @@ static void emit_rewrite_diff(const char *name_a,\n \t\t\t struct diff_options *o)\n {\n \tint lc_a, lc_b;\n-\tconst char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n-\tconst char *reset = diff_get_color(o->use_color, DIFF_RESET);\n \tstatic struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;\n \tconst char *a_prefix, *b_prefix;\n \tchar *data_one, *data_two;\n \tsize_t size_one, size_two;\n \tstruct emit_callback ecbdata;\n-\tconst char *line_prefix = diff_line_prefix(o);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\ta_prefix = o->b_prefix;\n@@ -914,14 +918,17 @@ static void emit_rewrite_diff(const char *name_a,\n \temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, a_name.buf, a_name.len, 0);\n \temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, b_name.buf, b_name.len, 1);\n \n-\tfprintf(o->file, \"%s%s@@ -\", line_prefix, fraginfo);\n+\tstrbuf_addstr(&out, \"@@ -\");\n \tif (!o->irreversible_delete)\n-\t\tprint_line_count(o->file, lc_a);\n+\t\tadd_line_count(&out, lc_a);\n \telse\n-\t\tfprintf(o->file, \"?,?\");\n-\tfprintf(o->file, \" +\");\n-\tprint_line_count(o->file, lc_b);\n-\tfprintf(o->file, \" @@%s\\n\", reset);\n+\t\tstrbuf_addstr(&out, \"?,?\");\n+\tstrbuf_addstr(&out, \" +\");\n+\tadd_line_count(&out, lc_b);\n+\tstrbuf_addstr(&out, \" @@\\n\");\n+\temit_diff_symbol(o, DIFF_SYMBOL_REWRITE_DIFF, out.buf, out.len, 0);\n+\tstrbuf_release(&out);\n+\n \tif (lc_a && !o->irreversible_delete)\n \t\temit_rewrite_lines(&ecbdata, '-', data_one, size_one);\n \tif (lc_b)\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322658","messageId":"20170620024816.20021-27-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[showing-off RFC/PATCH 26/26] diff.c: have a \"machine parseable\" move coloring","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:16Z","receivedAt":"2017-06-20T02:49:09Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Ævar asked for it, this is how you would do it.\n(plus documentation, tests, CLI knobs, options)\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 15 +++++++++++----\n diff.h | 2 ++\n 2 files changed, 13 insertions(+), 4 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 7756f7610c..61caa057ff 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n \tstruct strbuf sb = STRBUF_INIT;\n+\tint sign;\n \n \tenum diff_symbol s = eds->s;\n \tconst char *line = eds->line;\n@@ -1058,8 +1059,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n \t\t}\n+\t\tsign = '+';\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)\n+\t\t\tsign = '*';\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n-\t\temit_line_ws_markup(o, set, reset, line, len, '+',\n+\t\temit_line_ws_markup(o, set, reset, line, len, sign,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n@@ -1086,8 +1090,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n \t\t}\n+\t\tsign = '-';\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)\n+\t\t\tsign = '_';\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n-\t\temit_line_ws_markup(o, set, reset, line, len, '-',\n+\t\temit_line_ws_markup(o, set, reset, line, len, sign,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n \t\tbreak;\n \tcase DIFF_SYMBOL_WORDS_PORCELAIN:\n@@ -5475,7 +5482,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \tstatic struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n \n-\tif (o->color_moved)\n+\tif (o->color_moved || o->machine_readable_moves)\n \t\to->emitted_symbols = &esm;\n \n \tfor (i = 0; i < q->nr; i++) {\n@@ -5485,7 +5492,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \t}\n \n \tif (o->emitted_symbols) {\n-\t\tif (o->color_moved) {\n+\t\tif (o->color_moved || o->machine_readable_moves) {\n \t\t\tstruct hashmap add_lines, del_lines;\n \t\t\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n \ndiff --git a/diff.h b/diff.h\nindex 98abd75521..b6c4d7ab0f 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -194,6 +194,8 @@ struct diff_options {\n \t\tCOLOR_MOVED_ZEBRA = 2,\n \t\tCOLOR_MOVED_ZEBRA_DIM = 3,\n \t} color_moved;\n+\n+\tint machine_readable_moves;\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322659","messageId":"20170620024816.20021-21-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 20/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:10Z","receivedAt":"2017-06-20T02:49:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 73 +++++++++++++++++++++++++++++++++++++++---------------------------\n 1 file changed, 43 insertions(+), 30 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 18bfc4720b..f2f7a4da79 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -592,6 +592,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_WORD_DIFF,\n \tDIFF_SYMBOL_STAT_SEP,\n+\tDIFF_SYMBOL_SUMMARY,\n };\n /*\n * Flags for content lines:\n@@ -780,6 +781,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_STAT_SEP:\n \t\tfputs(o->stat_sep, o->file);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUMMARY:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -4727,67 +4731,76 @@ static void flush_one_pair(struct diff_filepair *p, struct diff_options *opt)\n \t}\n }\n \n-static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)\n+static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tif (fs->mode)\n-\t\tfprintf(file, \" %s mode %06o \", newdelete, fs->mode);\n+\t\tstrbuf_addf(&sb, \" %s mode %06o \", newdelete, fs->mode);\n \telse\n-\t\tfprintf(file, \" %s \", newdelete);\n-\twrite_name_quoted(fs->path, file, '\\n');\n-}\n+\t\tstrbuf_addf(&sb, \" %s \", newdelete);\n \n+\tquote_c_style(fs->path, &sb, NULL, 0);\n+\tstrbuf_addch(&sb, '\\n');\n+\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t sb.buf, sb.len, 0);\n+\tstrbuf_release(&sb);\n+}\n \n-static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,\n-\t\tconst char *line_prefix)\n+static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,\n+\t\tint show_name)\n {\n \tif (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {\n-\t\tfprintf(file, \"%s mode change %06o => %06o%c\", line_prefix, p->one->mode,\n-\t\t\tp->two->mode, show_name ? ' ' : '\\n');\n+\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\tstrbuf_addf(&sb, \" mode change %06o => %06o\",\n+\t\t\t p->one->mode, p->two->mode);\n \t\tif (show_name) {\n-\t\t\twrite_name_quoted(p->two->path, file, '\\n');\n+\t\t\tstrbuf_addch(&sb, ' ');\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n \t\t}\n+\t\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\t\tstrbuf_release(&sb);\n \t}\n }\n \n-static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,\n-\t\t\tconst char *line_prefix)\n+static void show_rename_copy(struct diff_options *opt, const char *renamecopy,\n+\t\tstruct diff_filepair *p)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tchar *names = pprint_rename(p->one->path, p->two->path);\n-\n-\tfprintf(file, \" %s %s (%d%%)\\n\", renamecopy, names, similarity_index(p));\n+\tstrbuf_addf(&sb, \" %s %s (%d%%)\\n\",\n+\t\t\trenamecopy, names, similarity_index(p));\n \tfree(names);\n-\tshow_mode_change(file, p, 0, line_prefix);\n+\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\tshow_mode_change(opt, p, 0);\n }\n \n static void diff_summary(struct diff_options *opt, struct diff_filepair *p)\n {\n-\tFILE *file = opt->file;\n-\tconst char *line_prefix = diff_line_prefix(opt);\n-\n \tswitch(p->status) {\n \tcase DIFF_STATUS_DELETED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"delete\", p->one);\n+\t\tshow_file_mode_name(opt, \"delete\", p->one);\n \t\tbreak;\n \tcase DIFF_STATUS_ADDED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"create\", p->two);\n+\t\tshow_file_mode_name(opt, \"create\", p->two);\n \t\tbreak;\n \tcase DIFF_STATUS_COPIED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"copy\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"copy\", p);\n \t\tbreak;\n \tcase DIFF_STATUS_RENAMED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"rename\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"rename\", p);\n \t\tbreak;\n \tdefault:\n \t\tif (p->score) {\n-\t\t\tfprintf(file, \"%s rewrite \", line_prefix);\n-\t\t\twrite_name_quoted(p->two->path, file, ' ');\n-\t\t\tfprintf(file, \"(%d%%)\\n\", similarity_index(p));\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addstr(&sb, \" rewrite \");\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n+\t\t\tstrbuf_addf(&sb, \" (%d%%)\\n\", similarity_index(p));\n+\t\t\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n \t\t}\n-\t\tshow_mode_change(file, p, !p->score, line_prefix);\n+\t\tshow_mode_change(opt, p, !p->score);\n \t\tbreak;\n \t}\n }\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322660","messageId":"20170620024816.20021-22-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 21/26] diff.c: buffer all output if asked to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:11Z","receivedAt":"2017-06-20T02:49:14Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Introduce a new option 'emitted_symbols' in the struct diff_options which\ncontrols whether all output is buffered up until all output is available.\nIt is set internally in diff.c when necessary.\n\nWe'll have a new struct 'emitted_string' in diff.c which will be used to\nbuffer each line. The emitted_string will duplicate the memory of the\nline to buffer as that is easiest to reason about for now. In a future\npatch we may want to decrease the memory usage by not duplicating all\noutput for buffering but rather we may want to store offsets into the\nfile or in case of hunk descriptions such as the similarity score, we\ncould just store the relevant number and reproduce the text later on.\n\nThis approach was chosen as a first step because it is quite simple\ncompared to the alternative with less memory footprint.\n\nemit_diff_symbol factors out the emission part and depending on the\ndiff_options->emitted_symbols the emission will be performed directly\nwhen calling emit_diff_symbol or after the whole process is done, i.e.\nby buffering we have add the possibility for a second pass over the\nwhole output before doing the actual output.\n\nIn 6440d34 (2012-03-14, diff: tweak a _copy_ of diff_options with\nword-diff) we introduced a duplicate diff options struct for word\nemissions as we may have different regex settings in there.\nWhen buffering the output, we need to operate on just one buffer,\nso we have to copy back the emissions of the word buffer into the\nmain buffer.\n\nUnconditionally enable output via buffer in this patch as it yields\na great opportunity for testing, i.e. all the diff tests from the\ntest suite pass without having reordering issues (i.e. only parts\nof the output got buffered, and we forgot to buffer other parts).\nThe test suite passes, which gives confidence that we converted all\nfunctions to use emit_string for output.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--\n diff.h | 2 ++\n 2 files changed, 109 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex f2f7a4da79..35b5924ff2 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -603,6 +603,47 @@ enum diff_symbol {\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n+/*\n+ * This struct is used when we need to buffer the output of the diff output.\n+ *\n+ * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer\n+ * into the pre/post image file. This pointer could be a union with the\n+ * line pointer. By storing an offset into the file instead of the literal line,\n+ * we can decrease the memory footprint for the buffered output. At first we\n+ * may want to only have indirection for the content lines, but we could also\n+ * enhance the state for emitting prefabricated lines, e.g. the similarity\n+ * score line or hunk/file headers would only need to store a number or path\n+ * and then the output can be constructed later on depending on state.\n+ */\n+struct emitted_diff_symbol {\n+\tconst char *line;\n+\tint len;\n+\tint flags;\n+\tenum diff_symbol s;\n+};\n+#define EMITTED_DIFF_SYMBOL_INIT {NULL}\n+\n+struct emitted_diff_symbols {\n+\tstruct emitted_diff_symbol *buf;\n+\tint nr, alloc;\n+};\n+#define EMITTED_DIFF_SYMBOLS_INIT {NULL, 0, 0}\n+\n+static void append_emitted_diff_symbol(struct diff_options *o,\n+\t\t\t\t struct emitted_diff_symbol *e)\n+{\n+\tstruct emitted_diff_symbol *f;\n+\n+\tALLOC_GROW(o->emitted_symbols->buf,\n+\t\t o->emitted_symbols->nr + 1,\n+\t\t o->emitted_symbols->alloc);\n+\tf = &o->emitted_symbols->buf[o->emitted_symbols->nr++];\n+\n+\tmemcpy(f, e, sizeof(struct emitted_diff_symbol));\n+\tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n+}\n+\n+\n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n \t\t\t\tconst char *line, int len, char sign,\n@@ -629,12 +670,18 @@ static void emit_line_ws_markup(struct diff_options *o,\n \t}\n }\n \n-static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n-\t\t\t const char *line, int len, unsigned flags)\n+static void emit_diff_symbol_from_struct(struct diff_options *o,\n+\t\t\t\t\t struct emitted_diff_symbol *eds)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n \tstruct strbuf sb = STRBUF_INIT;\n+\n+\tenum diff_symbol s = eds->s;\n+\tconst char *line = eds->line;\n+\tint len = eds->len;\n+\tunsigned flags = eds->flags;\n+\n \tswitch (s) {\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n@@ -790,6 +837,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tstrbuf_release(&sb);\n }\n \n+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n+\t\t\t const char *line, int len, unsigned flags)\n+{\n+\tstruct emitted_diff_symbol e = {line, len, flags, s};\n+\n+\tif (o->emitted_symbols)\n+\t\tappend_emitted_diff_symbol(o, &e);\n+\telse\n+\t\temit_diff_symbol_from_struct(o, &e);\n+}\n+\n void diff_emit_submodule_del(struct diff_options *o, const char *line)\n {\n \temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);\n@@ -1384,9 +1442,29 @@ static void diff_words_show(struct diff_words_data *diff_words)\n /* In \"color-words\" mode, show word-diff of words accumulated in the buffer */\n static void diff_words_flush(struct emit_callback *ecbdata)\n {\n+\tstruct diff_options *wo = ecbdata->diff_words->opt;\n+\n \tif (ecbdata->diff_words->minus.text.size ||\n \t ecbdata->diff_words->plus.text.size)\n \t\tdiff_words_show(ecbdata->diff_words);\n+\n+\tif (wo->emitted_symbols) {\n+\t\tstruct diff_options *o = ecbdata->opt;\n+\t\tstruct emitted_diff_symbols *wol = wo->emitted_symbols;\n+\t\tint i;\n+\n+\t\t/*\n+\t\t * NEEDSWORK:\n+\t\t * Instead of appending each, concat all words to a line?\n+\t\t */\n+\t\tfor (i = 0; i < wol->nr; i++)\n+\t\t\tappend_emitted_diff_symbol(o, &wol->buf[i]);\n+\n+\t\tfor (i = 0; i < wol->nr; i++)\n+\t\t\tfree((void *)wol->buf[i].line);\n+\n+\t\twol->nr = 0;\n+\t}\n }\n \n static void diff_filespec_load_driver(struct diff_filespec *one)\n@@ -1422,6 +1500,11 @@ static void init_diff_words_data(struct emit_callback *ecbdata,\n \t\txcalloc(1, sizeof(struct diff_words_data));\n \tecbdata->diff_words->type = o->word_diff;\n \tecbdata->diff_words->opt = o;\n+\n+\tif (orig_opts->emitted_symbols)\n+\t\to->emitted_symbols =\n+\t\t\txcalloc(1, sizeof(struct emitted_diff_symbols));\n+\n \tif (!o->word_regex)\n \t\to->word_regex = userdiff_word_regex(one);\n \tif (!o->word_regex)\n@@ -1456,6 +1539,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)\n {\n \tif (ecbdata->diff_words) {\n \t\tdiff_words_flush(ecbdata);\n+\t\tfree (ecbdata->diff_words->opt->emitted_symbols);\n \t\tfree (ecbdata->diff_words->opt);\n \t\tfree (ecbdata->diff_words->minus.text.ptr);\n \t\tfree (ecbdata->diff_words->minus.orig);\n@@ -5008,12 +5092,33 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n+\tstatic struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n+\t/*\n+\t * For testing purposes we want to make sure the diff machinery\n+\t * works completely with the buffer. If there is anything emitted\n+\t * outside the emit_string, then the order is screwed\n+\t * up and the tests will fail.\n+\t *\n+\t * TODO (later in this series):\n+\t * We'll unset this pointer in a later patch.\n+\t */\n+\to->emitted_symbols = &esm;\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tif (check_pair_status(p))\n \t\t\tdiff_flush_patch(p, o);\n \t}\n+\n+\tif (o->emitted_symbols) {\n+\t\tfor (i = 0; i < esm.nr; i++)\n+\t\t\temit_diff_symbol_from_struct(o, &esm.buf[i]);\n+\n+\t\tfor (i = 0; i < esm.nr; i++)\n+\t\t\tfree((void *)esm.buf[i].line);\n+\t}\n+\tesm.nr = 0;\n }\n \n void diff_flush(struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex b73f7a1e75..65fc9dbb4b 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -186,6 +186,8 @@ struct diff_options {\n \tvoid *output_prefix_data;\n \n \tint diff_path_counter;\n+\n+\tstruct emitted_diff_symbols *emitted_symbols;\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322661","messageId":"20170620024816.20021-25-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 24/26] diff.c: add dimming to moved line detection","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:14Z","receivedAt":"2017-06-20T02:49:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Any lines inside a moved block of code are not interesting. Boundaries\nof blocks are only interesting if they are next to another block of moved\ncode.\n\nSigned-off-by: Stefan Beller \n---\n color.h | 2 +\n diff.c | 139 ++++++++++++++++++++++++++++++++++++++++-----\n diff.h | 9 ++-\n t/t4015-diff-whitespace.sh | 124 ++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 259 insertions(+), 15 deletions(-)\n\ndiff --git a/color.h b/color.h\nindex 90627650fc..0e091b0cf5 100644\n--- a/color.h\n+++ b/color.h\n@@ -42,6 +42,8 @@ struct strbuf;\n #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n+#define GIT_COLOR_DI\t\t\"\\033[2m\"\n+#define GIT_COLOR_DI_IT\t\t\"\\033[2;3m\"\n \n /* A special value meaning \"no color selected\" */\n #define GIT_COLOR_NIL \"NIL\"\ndiff --git a/diff.c b/diff.c\nindex 0d41a53b76..7756f7610c 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -57,10 +57,14 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n-\tGIT_COLOR_MAGENTA,\t/* OLD_MOVED */\n-\tGIT_COLOR_BLUE,\t\t/* OLD_MOVED ALTERNATIVE */\n-\tGIT_COLOR_CYAN,\t\t/* NEW_MOVED */\n-\tGIT_COLOR_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n+\tGIT_COLOR_BOLD_MAGENTA,\t/* OLD_MOVED */\n+\tGIT_COLOR_BOLD_BLUE,\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_DI,\t\t/* OLD_MOVED_DIM */\n+\tGIT_COLOR_DI_IT,\t/* OLD_MOVED_ALTERNATIVE_DIM */\n+\tGIT_COLOR_BOLD_CYAN,\t/* NEW_MOVED */\n+\tGIT_COLOR_BOLD_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n+\tGIT_COLOR_DI,\t\t/* NEW_MOVED_DIM */\n+\tGIT_COLOR_DI_IT,\t/* NEW_MOVED_ALTERNATIVE_DIM */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -90,10 +94,18 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_FILE_OLD_MOVED;\n \tif (!strcasecmp(var, \"oldmovedalternative\"))\n \t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"oldmoveddimmed\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_DIM;\n+\tif (!strcasecmp(var, \"oldmovedalternativedimmed\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT_DIM;\n \tif (!strcasecmp(var, \"newmoved\"))\n \t\treturn DIFF_FILE_NEW_MOVED;\n \tif (!strcasecmp(var, \"newmovedalternative\"))\n \t\treturn DIFF_FILE_NEW_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoveddimmed\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_DIM;\n+\tif (!strcasecmp(var, \"newmovedalternativedimmed\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT_DIM;\n \treturn -1;\n }\n \n@@ -250,6 +262,8 @@ static int parse_color_moved(const char *arg)\n \t\treturn COLOR_MOVED_PLAIN;\n \telse if (!strcmp(arg, \"zebra\"))\n \t\treturn COLOR_MOVED_ZEBRA;\n+\telse if (!strcmp(arg, \"dimmed_zebra\"))\n+\t\treturn COLOR_MOVED_ZEBRA_DIM;\n \telse\n \t\treturn -1;\n }\n@@ -636,6 +650,7 @@ enum diff_symbol {\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n #define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n #define DIFF_SYMBOL_MOVED_LINE_ZEBRA\t\t(1<<18)\n+#define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING\t(1<<19)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n /*\n@@ -889,6 +904,67 @@ static void mark_color_as_moved(struct diff_options *o,\n \tfree(pmb);\n }\n \n+#define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \\\n+ (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ZEBRA)\n+static void dim_moved_lines(struct diff_options *o)\n+{\n+\tint n;\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct emitted_diff_symbol *prev = (n != 0) ?\n+\t\t\t\t&o->emitted_symbols->buf[n - 1] : NULL;\n+\t\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];\n+\t\tstruct emitted_diff_symbol *next =\n+\t\t\t\t(n < o->emitted_symbols->nr - 1) ?\n+\t\t\t\t&o->emitted_symbols->buf[n + 1] : NULL;\n+\n+\t\t/* Not a plus or minus line? */\n+\t\tif (l->s != DIFF_SYMBOL_PLUS && l->s != DIFF_SYMBOL_MINUS)\n+\t\t\tcontinue;\n+\n+\t\t/* Not a moved line? */\n+\t\tif (!(l->flags & DIFF_SYMBOL_MOVED_LINE))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * If prev or next are not a plus or minus line,\n+\t\t * pretend they don't exist\n+\t\t */\n+\t\tif (prev && prev->s != DIFF_SYMBOL_PLUS &&\n+\t\t\t prev->s != DIFF_SYMBOL_MINUS)\n+\t\t\tprev = NULL;\n+\t\tif (next && next->s != DIFF_SYMBOL_PLUS &&\n+\t\t\t next->s != DIFF_SYMBOL_MINUS)\n+\t\t\tnext = NULL;\n+\n+\t\t/* Inside a block? */\n+\t\tif ((prev &&\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK)) &&\n+\t\t (next &&\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK))) {\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Check if we are at an interesting bound: */\n+\t\tif (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA))\n+\t\t\tcontinue;\n+\t\tif (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * The boundary to prev and next are not interesting,\n+\t\t * so this line is not interesting as a whole\n+\t\t */\n+\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;\n+\t}\n+}\n+\n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n \t\t\t\tconst char *line, int len, char sign,\n@@ -960,24 +1036,56 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\temit_line(o, context, reset, line, len);\n \t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n-\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA)\n+\t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n-\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);\n-\t\telse\n+\t\t\tbreak;\n+\t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\t}\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '+',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n-\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA)\n+\t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n-\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);\n-\t\telse\n+\t\t\tbreak;\n+\t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\t}\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n@@ -4543,9 +4651,12 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n-\telse if (!strcmp(arg, \"--color-moved\"))\n-\t\toptions->color_moved = COLOR_MOVED_ZEBRA;\n-\telse if (!strcmp(arg, \"--no-color-moved\"))\n+\telse if (!strcmp(arg, \"--color-moved\")) {\n+\t\tif (diff_color_moved_default)\n+\t\t\toptions->color_moved = diff_color_moved_default;\n+\t\tif (options->color_moved == COLOR_MOVED_NO)\n+\t\t\toptions->color_moved = COLOR_MOVED_ZEBRA_DIM;\n+\t} else if (!strcmp(arg, \"--no-color-moved\"))\n \t\toptions->color_moved = COLOR_MOVED_NO;\n \telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n \t\tint cm = parse_color_moved(arg);\n@@ -5387,6 +5498,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \n \t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n \t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\t\t\tif (o->color_moved == COLOR_MOVED_ZEBRA_DIM)\n+\t\t\t\tdim_moved_lines(o);\n \n \t\t\thashmap_free(&add_lines, 0);\n \t\t\thashmap_free(&del_lines, 0);\ndiff --git a/diff.h b/diff.h\nindex 1aae8738ca..98abd75521 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -192,6 +192,7 @@ struct diff_options {\n \t\tCOLOR_MOVED_NO = 0,\n \t\tCOLOR_MOVED_PLAIN = 1,\n \t\tCOLOR_MOVED_ZEBRA = 2,\n+\t\tCOLOR_MOVED_ZEBRA_DIM = 3,\n \t} color_moved;\n };\n \n@@ -216,8 +217,12 @@ enum color_diff {\n \tDIFF_FUNCINFO = 8,\n \tDIFF_FILE_OLD_MOVED = 9,\n \tDIFF_FILE_OLD_MOVED_ALT = 10,\n-\tDIFF_FILE_NEW_MOVED = 11,\n-\tDIFF_FILE_NEW_MOVED_ALT = 12\n+\tDIFF_FILE_OLD_MOVED_DIM = 11,\n+\tDIFF_FILE_OLD_MOVED_ALT_DIM = 12,\n+\tDIFF_FILE_NEW_MOVED = 13,\n+\tDIFF_FILE_NEW_MOVED_ALT = 14,\n+\tDIFF_FILE_NEW_MOVED_DIM = 15,\n+\tDIFF_FILE_NEW_MOVED_ALT_DIM = 16\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 1ca16435d6..ae8c686f3c 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -1179,6 +1179,130 @@ test_expect_success 'plain moved code, inside file' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'detect permutations inside moved code -- dimmed_zebra' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\tEOF\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\tEOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\ttest_config color.diff.oldMovedDimmed \"normal magenta\" &&\n+\ttest_config color.diff.newMovedDimmed \"normal cyan\" &&\n+\ttest_config color.diff.oldMovedAlternativeDimmed \"normal blue\" &&\n+\ttest_config color.diff.newMovedAlternativeDimmed \"normal yellow\" &&\n+\tgit diff HEAD --no-renames --color-moved=dimmed_zebra| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'cmd option assumes configured colored-moved' '\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\ttest_config color.diff.oldMovedDimmed \"normal magenta\" &&\n+\ttest_config color.diff.newMovedDimmed \"normal cyan\" &&\n+\ttest_config color.diff.oldMovedAlternativeDimmed \"normal blue\" &&\n+\ttest_config color.diff.newMovedAlternativeDimmed \"normal yellow\" &&\n+\ttest_config diff.colorMoved zebra &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'no effect from --color-moved with --word-diff' '\n \tcat <<-\\EOF >text.txt &&\n \tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322662","messageId":"20170620024816.20021-17-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 16/26] diff.c: convert emit_binary_diff_body to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:06Z","receivedAt":"2017-06-20T02:49:19Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++-----------------\n 1 file changed, 49 insertions(+), 17 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 42a9020d95..e6ade5fde0 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -573,6 +573,11 @@ enum diff_symbol {\n \tDIFF_SYMBOL_FILEPAIR,\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_BINARY_FILES,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n+\tDIFF_SYMBOL_BINARY_DIFF_BODY,\n+\tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n \tDIFF_SYMBOL_REWRITE_DIFF,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n@@ -581,6 +586,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_SUBMODULE_HEADER,\n \tDIFF_SYMBOL_SUBMODULE_ERROR,\n \tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n+\n };\n /*\n * Flags for content lines:\n@@ -702,6 +708,22 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\tfprintf(o->file, \"%sBinary files %s differ\\n\",\n \t\t\tdiff_line_prefix(o), line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER:\n+\t\tfprintf(o->file, \"%sGIT binary patch\\n\", diff_line_prefix(o));\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA:\n+\t\tfprintf(o->file, \"%sdelta %s\\n\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:\n+\t\tfprintf(o->file, \"%sliteral %s\\n\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_FOOTER:\n+\t\tfputs(diff_line_prefix(o), o->file);\n+\t\tfputc('\\n', o->file);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_REWRITE_DIFF:\n \t\tfraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -2394,8 +2416,8 @@ static unsigned char *deflate_it(char *data,\n \treturn deflated;\n }\n \n-static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t\t const char *prefix)\n+static void emit_binary_diff_body(struct diff_options *o,\n+\t\t\t\t mmfile_t *one, mmfile_t *two)\n {\n \tvoid *cp;\n \tvoid *delta;\n@@ -2424,13 +2446,18 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t}\n \n \tif (delta && delta_size < deflate_size) {\n-\t\tfprintf(file, \"%sdelta %lu\\n\", prefix, orig_size);\n+\t\tchar *s = xstrfmt(\"%lu\", orig_size);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n+\t\t\t\t s, strlen(s), 0);\n+\t\tfree(s);\n \t\tfree(deflated);\n \t\tdata = delta;\n \t\tdata_size = delta_size;\n-\t}\n-\telse {\n-\t\tfprintf(file, \"%sliteral %lu\\n\", prefix, two->size);\n+\t} else {\n+\t\tchar *s = xstrfmt(\"%lu\", two->size);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n+\t\t\t\t s, strlen(s), 0);\n+\t\tfree(s);\n \t\tfree(delta);\n \t\tdata = deflated;\n \t\tdata_size = deflate_size;\n@@ -2439,8 +2466,9 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t/* emit data encoded in base85 */\n \tcp = data;\n \twhile (data_size) {\n+\t\tint len;\n \t\tint bytes = (52 < data_size) ? 52 : data_size;\n-\t\tchar line[70];\n+\t\tchar line[71];\n \t\tdata_size -= bytes;\n \t\tif (bytes <= 26)\n \t\t\tline[0] = bytes + 'A' - 1;\n@@ -2448,20 +2476,24 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t\t\tline[0] = bytes - 26 + 'a' - 1;\n \t\tencode_85(line + 1, cp, bytes);\n \t\tcp = (char *) cp + bytes;\n-\t\tfprintf(file, \"%s\", prefix);\n-\t\tfputs(line, file);\n-\t\tfputc('\\n', file);\n+\n+\t\tlen = strlen(line);\n+\t\tline[len++] = '\\n';\n+\t\tline[len] = '\\0';\n+\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_BODY,\n+\t\t\t\t line, len, 0);\n \t}\n-\tfprintf(file, \"%s\\n\", prefix);\n+\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_FOOTER, NULL, 0, 0);\n \tfree(data);\n }\n \n-static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t const char *prefix)\n+static void emit_binary_diff(struct diff_options *o,\n+\t\t\t mmfile_t *one, mmfile_t *two)\n {\n-\tfprintf(file, \"%sGIT binary patch\\n\", prefix);\n-\temit_binary_diff_body(file, one, two, prefix);\n-\temit_binary_diff_body(file, two, one, prefix);\n+\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER, NULL, 0, 0);\n+\temit_binary_diff_body(o, one, two);\n+\temit_binary_diff_body(o, two, one);\n }\n \n int diff_filespec_is_binary(struct diff_filespec *one)\n@@ -2646,7 +2678,7 @@ static void builtin_diff(const char *name_a,\n \t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n-\t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n+\t\t\temit_binary_diff(o, &mf1, &mf2);\n \t\telse {\n \t\t\tstrbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n \t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322663","messageId":"20170620024816.20021-24-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 23/26] diff.c: color moved lines differently, plain mode","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:13Z","receivedAt":"2017-06-20T02:49:22Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add the 'plain' mode for move detection of code.\nThis omits the checking for adjacent blocks, so it is not as useful.\nIf you have a lot of the same blocks moved in the same patch, the 'Zebra'\nwould end up slow as it is O(n^2) (n is number of same blocks).\nSo this may be useful there and is generally easy to add\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 5 +++++\n diff.h | 1 +\n t/t4015-diff-whitespace.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++-\n 3 files changed, 56 insertions(+), 1 deletion(-)\n\ndiff --git a/diff.c b/diff.c\nindex 20c1f9b99f..0d41a53b76 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -246,6 +246,8 @@ static int parse_color_moved(const char *arg)\n {\n \tif (!strcmp(arg, \"no\"))\n \t\treturn COLOR_MOVED_NO;\n+\telse if (!strcmp(arg, \"plain\"))\n+\t\treturn COLOR_MOVED_PLAIN;\n \telse if (!strcmp(arg, \"zebra\"))\n \t\treturn COLOR_MOVED_ZEBRA;\n \telse\n@@ -830,6 +832,9 @@ static void mark_color_as_moved(struct diff_options *o,\n \n \t\tl->flags |= DIFF_SYMBOL_MOVED_LINE;\n \n+\t\tif (o->color_moved == COLOR_MOVED_PLAIN)\n+\t\t\tcontinue;\n+\n \t\t/* Check any potential block runs, advance each or nullify */\n \t\tfor (i = 0; i < pmb_nr; i++) {\n \t\t\tstruct moved_entry *p = pmb[i];\ndiff --git a/diff.h b/diff.h\nindex 7726ad255c..1aae8738ca 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -190,6 +190,7 @@ struct diff_options {\n \tstruct emitted_diff_symbols *emitted_symbols;\n \tenum {\n \t\tCOLOR_MOVED_NO = 0,\n+\t\tCOLOR_MOVED_PLAIN = 1,\n \t\tCOLOR_MOVED_ZEBRA = 2,\n \t} color_moved;\n };\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 4a03766f1f..1ca16435d6 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -986,7 +986,7 @@ test_expect_success 'detect moved code, complete file' '\n \tgit mv test.c main.c &&\n \ttest_config color.diff.oldMoved \"normal red\" &&\n \ttest_config color.diff.newMoved \"normal green\" &&\n-\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tgit diff HEAD --color-moved=zebra --no-renames | test_decode_color >actual &&\n \tcat >expected <<-\\EOF &&\n \tdiff --git a/main.c b/main.c\n \tnew file mode 100644\n@@ -1130,6 +1130,55 @@ test_expect_success 'detect malicious moved code, inside file' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'plain moved code, inside file' '\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\t# needs previous test as setup\n+\tgit diff HEAD --no-renames --color-moved=plain| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'no effect from --color-moved with --word-diff' '\n \tcat <<-\\EOF >text.txt &&\n \tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322664","messageId":"20170620024816.20021-10-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 09/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS{_PORCELAIN}","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:59Z","receivedAt":"2017-06-20T02:49:27Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 42 ++++++++++++++++++++++++++----------------\n 1 file changed, 26 insertions(+), 16 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 42c9f48dc2..7096457c8a 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -567,6 +567,8 @@ enum diff_symbol {\n \tDIFF_SYMBOL_CONTEXT,\n \tDIFF_SYMBOL_PLUS,\n \tDIFF_SYMBOL_MINUS,\n+\tDIFF_SYMBOL_WORDS_PORCELAIN,\n+\tDIFF_SYMBOL_WORDS,\n };\n /*\n * Flags for content lines:\n@@ -648,6 +650,26 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_WORDS_PORCELAIN:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tfputs(\"~\\n\", o->file);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_WORDS:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\t/*\n+\t\t * Skip the prefix character, if any. With\n+\t\t * diff_suppress_blank_empty, there may be\n+\t\t * none.\n+\t\t */\n+\t\tif (line[0] != '\\n') {\n+\t\t\tline++;\n+\t\t\tlen--;\n+\t\t}\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1342,7 +1364,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n {\n \tstruct emit_callback *ecbdata = priv;\n \tconst char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);\n-\tconst char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n \tconst char *line_prefix = diff_line_prefix(o);\n@@ -1384,6 +1405,9 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->diff_words) {\n+\t\tenum diff_symbol s =\n+\t\t\tecbdata->diff_words->type == DIFF_WORDS_PORCELAIN ?\n+\t\t\tDIFF_SYMBOL_WORDS_PORCELAIN : DIFF_SYMBOL_WORDS;\n \t\tif (line[0] == '-') {\n \t\t\tdiff_words_append(line, len,\n \t\t\t\t\t &ecbdata->diff_words->minus);\n@@ -1403,21 +1427,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t\treturn;\n \t\t}\n \t\tdiff_words_flush(ecbdata);\n-\t\tif (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {\n-\t\t\temit_line(o, context, reset, line, len);\n-\t\t\tfputs(\"~\\n\", o->file);\n-\t\t} else {\n-\t\t\t/*\n-\t\t\t * Skip the prefix character, if any. With\n-\t\t\t * diff_suppress_blank_empty, there may be\n-\t\t\t * none.\n-\t\t\t */\n-\t\t\tif (line[0] != '\\n') {\n-\t\t\t line++;\n-\t\t\t len--;\n-\t\t\t}\n-\t\t\temit_line(o, context, reset, line, len);\n-\t\t}\n+\t\temit_diff_symbol(o, s, line, len, 0);\n \t\treturn;\n \t}\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322665","messageId":"20170620024816.20021-23-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 22/26] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:12Z","receivedAt":"2017-06-20T02:49:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a patch consists mostly of moving blocks of code around, it can\nbe quite tedious to ensure that the blocks are moved verbatim, and not\nundesirably modified in the move. To that end, color blocks that are\nmoved within the same patch differently. For example (OM, del, add,\nand NM are different colors):\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OM] - if (!is_authorized_user())\n [OM] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OM] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NM] + sensitive_stuff(spanning,\n [NM] + multiple,\n [NM] + lines);\n [NM] +}\n\nHowever adjacent blocks may be problematic. For example, in this\npotentially malicious patch, the swapping of blocks can be spotted:\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OMA] - if (!is_authorized_user())\n [OMA] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OMA] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NMA] + sensitive_stuff(spanning,\n [NMA] + multiple,\n [NMA] + lines);\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NMA] +}\n\nIf the moved code is larger, it is easier to hide some permutation in the\ncode, which is why some alternative coloring is needed.\n\nThis patch implements the first mode:\n* basic alternating 'Zebra' mode\n This conveys all information needed to the user. Defer customization to\n later patches.\n\nFirst I implemented an alternative design, which would try to fingerprint\na line by its neighbors to detect if we are in a block or at the boundary.\nThis idea iss error prone as it inspected each line and its neighboring\nlines to determine if the line was (a) moved and (b) if was deep inside\na hunk by having matching neighboring lines. This is unreliable as the\nwe can construct hunks which have equal neighbors that just exceed the\nnumber of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter\nas a line, that is permutated to AXYZCXYZBXYZD..').\n\nInstead this provides a dynamic programming greedy algorithm that finds\nthe largest moved hunk and then has several modes on highlighting bounds.\n\nA note on the options '--submodule=diff' and '--color-words/--word-diff':\nIn the conversion to use emit_line in the prior patches both submodules\nas well as word diff output carefully chose to call emit_line with sign=0.\nAll output with sign=0 is ignored for move detection purposes in this\npatch, such that no weird looking output will be generated for these\ncases. This leads to another thought: We could pass on '--color-moved' to\nsubmodules such that they color up moved lines for themselves. If we'd do\nso only line moves within a repository boundary are marked up.\n\nHelped-by: Jonathan Tan \nSigned-off-by: Stefan Beller \n---\n diff.c | 304 ++++++++++++++++++++++++++++++++++++++++++---\n diff.h | 10 +-\n t/t4015-diff-whitespace.sh | 196 +++++++++++++++++++++++++++++\n 3 files changed, 495 insertions(+), 15 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 35b5924ff2..20c1f9b99f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -15,6 +15,7 @@\n #include \"userdiff.h\"\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n+#include \"hashmap.h\"\n #include \"ll-merge.h\"\n #include \"string-list.h\"\n #include \"argv-array.h\"\n@@ -31,6 +32,7 @@ static int diff_indent_heuristic; /* experimental */\n static int diff_rename_limit_default = 400;\n static int diff_suppress_blank_empty;\n static int diff_use_color_default = -1;\n+static int diff_color_moved_default;\n static int diff_context_default = 3;\n static int diff_interhunk_context_default;\n static const char *diff_word_regex_cfg;\n@@ -55,6 +57,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n+\tGIT_COLOR_MAGENTA,\t/* OLD_MOVED */\n+\tGIT_COLOR_BLUE,\t\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_CYAN,\t\t/* NEW_MOVED */\n+\tGIT_COLOR_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -80,6 +86,14 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_WHITESPACE;\n \tif (!strcasecmp(var, \"func\"))\n \t\treturn DIFF_FUNCINFO;\n+\tif (!strcasecmp(var, \"oldmoved\"))\n+\t\treturn DIFF_FILE_OLD_MOVED;\n+\tif (!strcasecmp(var, \"oldmovedalternative\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoved\"))\n+\t\treturn DIFF_FILE_NEW_MOVED;\n+\tif (!strcasecmp(var, \"newmovedalternative\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT;\n \treturn -1;\n }\n \n@@ -228,12 +242,29 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)\n \treturn 0;\n }\n \n+static int parse_color_moved(const char *arg)\n+{\n+\tif (!strcmp(arg, \"no\"))\n+\t\treturn COLOR_MOVED_NO;\n+\telse if (!strcmp(arg, \"zebra\"))\n+\t\treturn COLOR_MOVED_ZEBRA;\n+\telse\n+\t\treturn -1;\n+}\n+\n int git_diff_ui_config(const char *var, const char *value, void *cb)\n {\n \tif (!strcmp(var, \"diff.color\") || !strcmp(var, \"color.diff\")) {\n \t\tdiff_use_color_default = git_config_colorbool(var, value);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"diff.colormoved\")) {\n+\t\tint cm = parse_color_moved(value);\n+\t\tif (cm < 0)\n+\t\t\treturn -1;\n+\t\tdiff_color_moved_default = cm;\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"diff.context\")) {\n \t\tdiff_context_default = git_config_int(var, value);\n \t\tif (diff_context_default < 0)\n@@ -600,7 +631,9 @@ enum diff_symbol {\n * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT\n * 16 is marking if the line is blank at EOF\n */\n-#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n+#define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n+#define DIFF_SYMBOL_MOVED_LINE_ZEBRA\t\t(1<<18)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n /*\n@@ -643,6 +676,213 @@ static void append_emitted_diff_symbol(struct diff_options *o,\n \tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n }\n \n+struct moved_entry {\n+\tstruct hashmap_entry ent;\n+\tconst struct emitted_diff_symbol *es;\n+\tstruct moved_entry *next_line;\n+};\n+\n+static void get_ws_cleaned_string(const struct emitted_diff_symbol *l,\n+\t\t\t\t struct strbuf *out)\n+{\n+\tint i;\n+\tfor (i = 0; i < l->len; i++) {\n+\t\tif (isspace(l->line[i]))\n+\t\t\tcontinue;\n+\t\tstrbuf_addch(out, l->line[i]);\n+\t}\n+}\n+\n+static int emitted_symbol_cmp_no_ws(const struct emitted_diff_symbol *a,\n+\t\t\t\t const struct emitted_diff_symbol *b,\n+\t\t\t\t const void *keydata)\n+{\n+\tint ret;\n+\tstruct strbuf sba = STRBUF_INIT;\n+\tstruct strbuf sbb = STRBUF_INIT;\n+\n+\tget_ws_cleaned_string(a, &sba);\n+\tget_ws_cleaned_string(b, &sbb);\n+\tret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);\n+\n+\tstrbuf_release(&sba);\n+\tstrbuf_release(&sbb);\n+\treturn ret;\n+}\n+\n+static int emitted_symbol_cmp(const struct emitted_diff_symbol *a,\n+\t\t\t const struct emitted_diff_symbol *b,\n+\t\t\t const void *keydata)\n+{\n+\treturn a->len != b->len || strncmp(a->line, b->line, a->len);\n+}\n+\n+static int moved_entry_cmp(const struct moved_entry *a,\n+\t\t\t const struct moved_entry *b,\n+\t\t\t const void *keydata)\n+{\n+\treturn emitted_symbol_cmp(a->es, b->es, keydata);\n+}\n+\n+static int moved_entry_cmp_no_ws(const struct moved_entry *a,\n+\t\t\t\t const struct moved_entry *b,\n+\t\t\t\t const void *keydata)\n+{\n+\treturn emitted_symbol_cmp_no_ws(a->es, b->es, keydata);\n+}\n+\n+static unsigned get_string_hash(struct emitted_diff_symbol *es, unsigned ignore_ws)\n+{\n+\tstatic struct strbuf sb = STRBUF_INIT;\n+\n+\tif (ignore_ws) {\n+\t\tstrbuf_reset(&sb);\n+\t\tget_ws_cleaned_string(es, &sb);\n+\t\treturn memhash(sb.buf, sb.len);\n+\t} else {\n+\t\treturn memhash(es->line, es->len);\n+\t}\n+}\n+\n+static struct moved_entry *prepare_entry(struct diff_options *o,\n+\t\t\t\t\t int line_no)\n+{\n+\tstruct moved_entry *ret = xmalloc(sizeof(*ret));\n+\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];\n+\n+\tret->ent.hash = get_string_hash(l, ignore_ws);\n+\tret->es = l;\n+\tret->next_line = NULL;\n+\n+\treturn ret;\n+}\n+\n+static void add_lines_to_move_detection(struct diff_options *o,\n+\t\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry *prev_line = NULL;\n+\n+\tint n;\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct hashmap *hm;\n+\t\tstruct moved_entry *key;\n+\n+\t\tswitch (o->emitted_symbols->buf[n].s) {\n+\t\tcase DIFF_SYMBOL_PLUS:\n+\t\t\thm = add_lines;\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MINUS:\n+\t\t\thm = del_lines;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tkey = prepare_entry(o, n);\n+\t\tif (prev_line && prev_line->es->s == o->emitted_symbols->buf[n].s)\n+\t\t\tprev_line->next_line = key;\n+\n+\t\thashmap_add(hm, key);\n+\t\tprev_line = key;\n+\t}\n+}\n+\n+/* Find blocks of moved code, delegate actual coloring decision to helper */\n+static void mark_color_as_moved(struct diff_options *o,\n+\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry **pmb = NULL; /* potentially moved blocks */\n+\tint pmb_nr = 0, pmb_alloc = 0;\n+\tint n, flipped_block = 1;\n+\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct hashmap *hm = NULL;\n+\t\tstruct moved_entry *key;\n+\t\tstruct moved_entry *match = NULL;\n+\t\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];\n+\t\tint i, lp, rp;\n+\n+\t\tswitch (l->s) {\n+\t\tcase DIFF_SYMBOL_PLUS:\n+\t\t\thm = del_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MINUS:\n+\t\t\thm = add_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tflipped_block = 1;\n+\t\t}\n+\n+\t\tif (!match) {\n+\t\t\tpmb_nr = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE;\n+\n+\t\t/* Check any potential block runs, advance each or nullify */\n+\t\tfor (i = 0; i < pmb_nr; i++) {\n+\t\t\tstruct moved_entry *p = pmb[i];\n+\t\t\tstruct moved_entry *pnext = (p && p->next_line) ?\n+\t\t\t\t\tp->next_line : NULL;\n+\t\t\tif (pnext &&\n+\t\t\t !emitted_symbol_cmp(pnext->es, l, o)) {\n+\t\t\t\tpmb[i] = p->next_line;\n+\t\t\t} else {\n+\t\t\t\tpmb[i] = NULL;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Shrink the set of potential block to the remaining running */\n+\t\tfor (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n+\t\t\twhile (lp < pmb_nr && pmb[lp])\n+\t\t\t\tlp++;\n+\t\t\t/* lp points at the first NULL now */\n+\n+\t\t\twhile (rp > -1 && !pmb[rp])\n+\t\t\t\trp--;\n+\t\t\t/* rp points at the last non-NULL */\n+\n+\t\t\tif (lp < pmb_nr && rp > -1 && lp < rp) {\n+\t\t\t\tpmb[lp] = pmb[rp];\n+\t\t\t\tpmb[rp] = NULL;\n+\t\t\t\trp--;\n+\t\t\t\tlp++;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Remember the number of running sets */\n+\t\tpmb_nr = rp + 1;\n+\n+\t\tif (pmb_nr == 0) {\n+\t\t\t/*\n+\t\t\t * The current line is the start of a new block.\n+\t\t\t * Setup the set of potential blocks.\n+\t\t\t */\n+\t\t\tfor (; match; match = hashmap_get_next(hm, match)) {\n+\t\t\t\tALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n+\t\t\t\tpmb[pmb_nr++] = match;\n+\t\t\t}\n+\n+\t\t\tflipped_block = (flipped_block + 1) % 2;\n+\t\t}\n+\n+\t\tif (flipped_block)\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_ZEBRA;\n+\t}\n+\n+\tfree(pmb);\n+}\n \n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n@@ -715,14 +955,24 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\temit_line(o, context, reset, line, len);\n \t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n-\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n+\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);\n+\t\telse\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '+',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n-\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n+\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);\n+\t\telse\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n@@ -3750,6 +4000,8 @@ void diff_setup(struct diff_options *options)\n \t\toptions->a_prefix = \"a/\";\n \t\toptions->b_prefix = \"b/\";\n \t}\n+\n+\toptions->color_moved = diff_color_moved_default;\n }\n \n void diff_setup_done(struct diff_options *options)\n@@ -3859,6 +4111,9 @@ void diff_setup_done(struct diff_options *options)\n \n \tif (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)\n \t\tdie(_(\"--follow requires exactly one pathspec\"));\n+\n+\tif (!options->use_color || external_diff())\n+\t\toptions->color_moved = 0;\n }\n \n static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)\n@@ -4283,7 +4538,16 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n-\telse if (!strcmp(arg, \"--color-words\")) {\n+\telse if (!strcmp(arg, \"--color-moved\"))\n+\t\toptions->color_moved = COLOR_MOVED_ZEBRA;\n+\telse if (!strcmp(arg, \"--no-color-moved\"))\n+\t\toptions->color_moved = COLOR_MOVED_NO;\n+\telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n+\t\tint cm = parse_color_moved(arg);\n+\t\tif (cm < 0)\n+\t\t\tdie(\"bad --color-moved argument: %s\", arg);\n+\t\toptions->color_moved = cm;\n+\t} else if (!strcmp(arg, \"--color-words\")) {\n \t\toptions->use_color = 1;\n \t\toptions->word_diff = DIFF_WORDS_COLOR;\n \t}\n@@ -5094,16 +5358,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \tint i;\n \tstatic struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n-\t/*\n-\t * For testing purposes we want to make sure the diff machinery\n-\t * works completely with the buffer. If there is anything emitted\n-\t * outside the emit_string, then the order is screwed\n-\t * up and the tests will fail.\n-\t *\n-\t * TODO (later in this series):\n-\t * We'll unset this pointer in a later patch.\n-\t */\n-\to->emitted_symbols = &esm;\n+\n+\tif (o->color_moved)\n+\t\to->emitted_symbols = &esm;\n \n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n@@ -5112,6 +5369,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \t}\n \n \tif (o->emitted_symbols) {\n+\t\tif (o->color_moved) {\n+\t\t\tstruct hashmap add_lines, del_lines;\n+\t\t\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\n+\t\t\thashmap_init(&del_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\t\t\thashmap_init(&add_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\n+\t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n+\t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\n+\t\t\thashmap_free(&add_lines, 0);\n+\t\t\thashmap_free(&del_lines, 0);\n+\t\t}\n+\n \t\tfor (i = 0; i < esm.nr; i++)\n \t\t\temit_diff_symbol_from_struct(o, &esm.buf[i]);\n \n@@ -5195,6 +5470,7 @@ void diff_flush(struct diff_options *options)\n \t\tif (!options->file)\n \t\t\tdie_errno(\"Could not open /dev/null\");\n \t\toptions->close_file = 1;\n+\t\toptions->color_moved = 0;\n \t\tfor (i = 0; i < q->nr; i++) {\n \t\t\tstruct diff_filepair *p = q->queue[i];\n \t\t\tif (check_pair_status(p))\ndiff --git a/diff.h b/diff.h\nindex 65fc9dbb4b..7726ad255c 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,10 @@ struct diff_options {\n \tint diff_path_counter;\n \n \tstruct emitted_diff_symbols *emitted_symbols;\n+\tenum {\n+\t\tCOLOR_MOVED_NO = 0,\n+\t\tCOLOR_MOVED_ZEBRA = 2,\n+\t} color_moved;\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n@@ -208,7 +212,11 @@ enum color_diff {\n \tDIFF_FILE_NEW = 5,\n \tDIFF_COMMIT = 6,\n \tDIFF_WHITESPACE = 7,\n-\tDIFF_FUNCINFO = 8\n+\tDIFF_FUNCINFO = 8,\n+\tDIFF_FILE_OLD_MOVED = 9,\n+\tDIFF_FILE_OLD_MOVED_ALT = 10,\n+\tDIFF_FILE_NEW_MOVED = 11,\n+\tDIFF_FILE_NEW_MOVED_ALT = 12\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 289806d0c7..4a03766f1f 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -972,4 +972,200 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '\n \n '\n \n+test_expect_success 'detect moved code, complete file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >test.c &&\n+\t#include\n+\tmain()\n+\t{\n+\tprintf(\"Hello World\");\n+\t}\n+\tEOF\n+\tgit add test.c &&\n+\tgit commit -m \"add main function\" &&\n+\tgit mv test.c main.c &&\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tdiff --git a/main.c b/main.c\n+\tnew file mode 100644\n+\tindex 0000000..a986c57\n+\t--- /dev/null\n+\t+++ b/main.c\n+\t@@ -0,0 +1,5 @@\n+\t+#include\n+\t+main()\n+\t+{\n+\t+printf(\"Hello World\");\n+\t+}\n+\tdiff --git a/test.c b/test.c\n+\tdeleted file mode 100644\n+\tindex a986c57..0000000\n+\t--- a/test.c\n+\t+++ /dev/null\n+\t@@ -1,5 +0,0 @@\n+\t-#include\n+\t-main()\n+\t-{\n+\t-printf(\"Hello World\");\n+\t-}\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect malicious moved code, inside file' '\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit add main.c test.c &&\n+\tgit commit -m \"add main and test file\" &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tfoo(u);\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit diff HEAD --no-renames --color-moved=zebra| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'no effect from --color-moved with --word-diff' '\n+\tcat <<-\\EOF >text.txt &&\n+\tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n+\tEOF\n+\tgit add text.txt &&\n+\tgit commit -a -m \"clean state\" &&\n+\tcat <<-\\EOF >text.txt &&\n+\tsimply Lorem Ipsum dummy is text of the typesetting and printing industry.\n+\tEOF\n+\tgit diff --color-moved --word-diff >actual &&\n+\tgit diff --word-diff >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'move detection with submodules' '\n+\ttest_create_repo bananas &&\n+\techo ripe >bananas/recipe &&\n+\tgit -C bananas add recipe &&\n+\ttest_commit fruit &&\n+\ttest_commit -C bananas recipe &&\n+\tgit submodule add ./bananas &&\n+\tgit add bananas &&\n+\tgit commit -a -m \"bananas are like a heavy library?\" &&\n+\techo foul >bananas/recipe &&\n+\techo ripe >fruit.t &&\n+\n+\tgit diff --submodule=diff --color-moved >actual &&\n+\n+\t# no move detection as the moved line is across repository boundaries.\n+\ttest_decode_color decoded_actual &&\n+\t! grep BGREEN decoded_actual &&\n+\t! grep BRED decoded_actual &&\n+\n+\t# nor did we mess with it another way\n+\tgit diff --submodule=diff | test_decode_color >expect &&\n+\ttest_cmp expect decoded_actual\n+'\n+\n test_done\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322666","messageId":"20170620024816.20021-6-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 05/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:55Z","receivedAt":"2017-06-20T02:49:36Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 10 +++++++++-\n 1 file changed, 9 insertions(+), 1 deletion(-)\n\ndiff --git a/diff.c b/diff.c\nindex 89466018e5..3af07fa659 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,17 +561,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n \n enum diff_symbol {\n \tDIFF_SYMBOL_SEPARATOR,\n+\tDIFF_SYMBOL_CONTEXT_MARKER,\n };\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len)\n {\n+\tconst char *context, *reset;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n \t\t\tdiff_line_prefix(o),\n \t\t\to->line_termination);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT_MARKER:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -661,7 +668,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tif (len < 10 ||\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n-\t\temit_line(ecbdata->opt, context, reset, line, len);\n+\t\temit_diff_symbol(ecbdata->opt,\n+\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322667","messageId":"20170620024816.20021-11-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 10/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:00Z","receivedAt":"2017-06-20T02:49:38Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 10 ++++++++--\n 1 file changed, 8 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 7096457c8a..e6029e8213 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -565,6 +565,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_NO_LF_EOF,\n \tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_CONTEXT_INCOMPLETE,\n \tDIFF_SYMBOL_PLUS,\n \tDIFF_SYMBOL_MINUS,\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n@@ -637,6 +638,11 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_ws_markup(o, set, reset, line, len, ' ',\n \t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT_INCOMPLETE:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n \t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -1448,8 +1454,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \tdefault:\n \t\t/* incomplete line at the end */\n \t\tecbdata->lno_in_preimage++;\n-\t\temit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),\n-\t\t\t reset, line, len);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_CONTEXT_INCOMPLETE,\n+\t\t\t\t line, len, 0);\n \t\tbreak;\n \t}\n }\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322668","messageId":"20170620024816.20021-14-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:48:03Z","receivedAt":"2017-06-20T02:49:40Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 21 ++++++++++++++++-----\n 1 file changed, 16 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 95f11ea86f..e56962b844 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -572,6 +572,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_FILEPAIR,\n \tDIFF_SYMBOL_HEADER,\n+\tDIFF_SYMBOL_BINARY_FILES,\n };\n /*\n * Flags for content lines:\n@@ -689,6 +690,10 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_FILES:\n+\t\tfprintf(o->file, \"%sBinary files %s differ\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -2537,6 +2542,7 @@ static void builtin_diff(const char *name_a,\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n \t ( (!textconv_one && diff_filespec_is_binary(one)) ||\n \t (!textconv_two && diff_filespec_is_binary(two)) )) {\n+\t\tstruct strbuf sb = STRBUF_INIT;\n \t\tif (!one->data && !two->data &&\n \t\t S_ISREG(one->mode) && S_ISREG(two->mode) &&\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n@@ -2549,8 +2555,10 @@ static void builtin_diff(const char *name_a,\n \t\t\t}\n \t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n \t\t\t\t\t header.buf, header.len, 0);\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\t\tstrbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_release(&sb);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n \t\tif (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)\n@@ -2567,9 +2575,12 @@ static void builtin_diff(const char *name_a,\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n-\t\telse\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\telse {\n+\t\t\tstrbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_release(&sb);\n+\t\t}\n \t\to->found_changes = 1;\n \t} else {\n \t\t/* Crazy xdl interfaces.. */\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322669","messageId":"20170620024816.20021-2-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCH 01/26] diff.c: readability fix","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:47:51Z","receivedAt":"2017-06-20T02:49:42Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We already have dereferenced 'p->two' into a local variable 'two'.\nUse that.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 74283d9001..3f5bf8b5a4 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3283,8 +3283,8 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \tconst char *other;\n \tconst char *attr_path;\n \n-\tname = p->one->path;\n-\tother = (strcmp(name, p->two->path) ? p->two->path : NULL);\n+\tname = one->path;\n+\tother = (strcmp(name, two->path) ? two->path : NULL);\n \tattr_path = name;\n \tif (o->prefix_length)\n \t\tstrip_prefix(o->prefix_length, &name, &other);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"322670","messageId":"CAGZ79kYLdpaXzn0T5547fRr=EPR691n2bPfS+mzaZuS-k39ayw@mail.gmail.com","threadId":"46038","inReplyTo":"20170620024816.20021-27-sbeller@google.com","subject":"Re: [showing-off RFC/PATCH 26/26] diff.c: have a \"machine parseable\" move coloring","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T02:50:35Z","receivedAt":"2017-06-20T02:50:41Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"+ Ævar, who was not part of the email where I copied all recipients\nfrom for this series.\n\nOn Mon, Jun 19, 2017 at 7:48 PM, Stefan Beller wrote:\n> Ævar asked for it, this is how you would do it.\n> (plus documentation, tests, CLI knobs, options)\n>\n> Signed-off-by: Stefan Beller \n> ---\n> diff.c | 15 +++++++++++----\n> diff.h | 2 ++\n> 2 files changed, 13 insertions(+), 4 deletions(-)\n>\n> diff --git a/diff.c b/diff.c\n> index 7756f7610c..61caa057ff 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n> static const char *nneof = \" No newline at end of file\\n\";\n> const char *context, *reset, *set, *meta, *fraginfo;\n> struct strbuf sb = STRBUF_INIT;\n> + int sign;\n>\n> enum diff_symbol s = eds->s;\n> const char *line = eds->line;\n> @@ -1058,8 +1059,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n> default:\n> set = diff_get_color_opt(o, DIFF_FILE_NEW);\n> }\n> + sign = '+';\n> + if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)\n> + sign = '*';\n> reset = diff_get_color_opt(o, DIFF_RESET);\n> - emit_line_ws_markup(o, set, reset, line, len, '+',\n> + emit_line_ws_markup(o, set, reset, line, len, sign,\n> flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n> flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n> break;\n> @@ -1086,8 +1090,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n> default:\n> set = diff_get_color_opt(o, DIFF_FILE_OLD);\n> }\n> + sign = '-';\n> + if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)\n> + sign = '_';\n> reset = diff_get_color_opt(o, DIFF_RESET);\n> - emit_line_ws_markup(o, set, reset, line, len, '-',\n> + emit_line_ws_markup(o, set, reset, line, len, sign,\n> flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n> break;\n> case DIFF_SYMBOL_WORDS_PORCELAIN:\n> @@ -5475,7 +5482,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n> static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n> struct diff_queue_struct *q = &diff_queued_diff;\n>\n> - if (o->color_moved)\n> + if (o->color_moved || o->machine_readable_moves)\n> o->emitted_symbols = &esm;\n>\n> for (i = 0; i < q->nr; i++) {\n> @@ -5485,7 +5492,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n> }\n>\n> if (o->emitted_symbols) {\n> - if (o->color_moved) {\n> + if (o->color_moved || o->machine_readable_moves) {\n> struct hashmap add_lines, del_lines;\n> unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n>\n> diff --git a/diff.h b/diff.h\n> index 98abd75521..b6c4d7ab0f 100644\n> --- a/diff.h\n> +++ b/diff.h\n> @@ -194,6 +194,8 @@ struct diff_options {\n> COLOR_MOVED_ZEBRA = 2,\n> COLOR_MOVED_ZEBRA_DIM = 3,\n> } color_moved;\n> +\n> + int machine_readable_moves;\n> };\n>\n> void diff_emit_submodule_del(struct diff_options *o, const char *line);\n> --\n> 2.12.2.575.gb14f27f917\n>\n"},{"id":"322780","messageId":"20170620130147.75e1fb78@twelve2.svl.corp.google.com","threadId":"46038","inReplyTo":"20170620024816.20021-12-sbeller@google.com","subject":"Re: [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-06-20T20:01:47Z","receivedAt":"2017-06-20T20:01:58Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Mon, 19 Jun 2017 19:48:01 -0700\nStefan Beller wrote:\n\n> @@ -676,6 +677,14 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n> \t\t}\n> \t\temit_line(o, context, reset, line, len);\n> \t\tbreak;\n> +\tcase DIFF_SYMBOL_FILEPAIR:\n> +\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n> +\t\treset = diff_get_color_opt(o, DIFF_RESET);\n> +\t\tfprintf(o->file, \"%s%s%s%s%s%s\\n\", diff_line_prefix(o), meta,\n> +\t\t\tflags ? \"+++ \" : \"--- \",\n\nI think that a constant should be defined for this flag, just like for\ncontent lines. If not it is not immediately obvious that a flag should\nbe set for the +++ lines.\n\n> +\t\t\tline, reset,\n> +\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n> +\t\tbreak;\n> \tdefault:\n> \t\tdie(\"BUG: unknown diff symbol\");\n> \t}\n"},{"id":"322781","messageId":"20170620130903.36b1e696@twelve2.svl.corp.google.com","threadId":"46038","inReplyTo":"20170620024816.20021-16-sbeller@google.com","subject":"Re: [PATCH 15/26] submodule.c: migrate diff output to use emit_diff_symbol","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-06-20T20:09:03Z","receivedAt":"2017-06-20T20:09:14Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Mon, 19 Jun 2017 19:48:05 -0700\nStefan Beller wrote:\n\n> As the submodule process is no longer attached to the same stdout as\n> the superprojects process, we need to pass coloring explicitly.\n\nI found this confusing - what difference does the stdout make? If they\nwere the same stdout, one process could set a color for the other\nprocess to follow, but it wouldn't be able to affect color changes\ninside the output, right?\n\n> Remove the colors from the function signatures, as all the coloring\n> decisions will be made either inside the child process or the final\n> emit_diff_symbol.\n\nThis seems to be of the same pattern as the others. I would write this\nas:\n\n Create diff_emit_submodule_.* functions that make its own coloring\n decisions and migrate submodule.c to use them. This centralizes\n diff output, including coloring information, in one file.\n\nor something like that.\n"},{"id":"322782","messageId":"20170620131339.3f56bcf0@twelve2.svl.corp.google.com","threadId":"46038","inReplyTo":"20170620024816.20021-23-sbeller@google.com","subject":"Re: [PATCH 22/26] diff.c: color moved lines differently","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2017-06-20T20:13:39Z","receivedAt":"2017-06-20T20:13:49Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"I just glanced through this file, because it seems similar to the\nversions I have previously reviewed.\n\nI'll skip patches 23 onwards in this round of review because (i) I would\nbe happy if just patches 1-22 were included in the tree and (ii) those\npatches might end up changing anyway because of review comments in the\nprior patches.\n\nOn Mon, 19 Jun 2017 19:48:12 -0700\nStefan Beller wrote:\n\n> +/* Find blocks of moved code, delegate actual coloring decision to helper */\n> +static void mark_color_as_moved(struct diff_options *o,\n> +\t\t\t\tstruct hashmap *add_lines,\n> +\t\t\t\tstruct hashmap *del_lines)\n> +{\n\n[snip]\n\n> +\t\tif (flipped_block)\n> +\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_ZEBRA;\n\nThis should probably be DIFF_SYMBOL_MOVED_LINE_ALT. \"Zebra\" refers to\nboth the stripes, not just the alternate stripe.\n"},{"id":"322783","messageId":"CAGZ79kbHzmZE_uUUbFbMN-Gv8oegkEOry1i6G28GKbHmSJzmRA@mail.gmail.com","threadId":"46038","inReplyTo":"20170620131339.3f56bcf0@twelve2.svl.corp.google.com","subject":"Re: [PATCH 22/26] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-20T20:57:14Z","receivedAt":"2017-06-20T20:57:26Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Jun 20, 2017 at 1:13 PM, Jonathan Tan wrote:\n> I just glanced through this file, because it seems similar to the\n> versions I have previously reviewed.\n\nYes, this has not changed much.\nThe thing that took so long were patches 1-20 to do.\nIn these later patches (22) the UX was changed slightly.\nThe mode is zebra instead of alternating for example\n\n> I'll skip patches 23 onwards in this round of review because (i) I would\n> be happy if just patches 1-22 were included in the tree\n\npatch 22 just introduced the zebra mode, which contains all\ninformation necessary, the later \"dimming\" is purely\nUI excitement, no further information added.\n\nIn former series, I had documentation at each\npatch that added a new mode, now I rolled all of\ndocumentation in patch 25. I will add the basic docs\nto that patch in a reroll.\n\n> and (ii) those\n> patches might end up changing anyway because of review comments in the\n> prior patches.\n\nplain, dimming, docs, and this RFC for machine parsable output,\nok with me.\n\nSo I'll focus on the first 22 patches.\n\n>\n> On Mon, 19 Jun 2017 19:48:12 -0700\n> Stefan Beller wrote:\n>\n>> +/* Find blocks of moved code, delegate actual coloring decision to helper */\n>> +static void mark_color_as_moved(struct diff_options *o,\n>> + struct hashmap *add_lines,\n>> + struct hashmap *del_lines)\n>> +{\n>\n> [snip]\n>\n>> + if (flipped_block)\n>> + l->flags |= DIFF_SYMBOL_MOVED_LINE_ZEBRA;\n>\n> This should probably be DIFF_SYMBOL_MOVED_LINE_ALT. \"Zebra\" refers to\n> both the stripes, not just the alternate stripe.\n\n\nok\n"},{"id":"322851","messageId":"xmqq7f05p1z7.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170620024816.20021-5-sbeller@google.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T19:36:28Z","receivedAt":"2017-06-21T19:36:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Signed-off-by: Stefan Beller \n> ---\n> diff.c | 22 +++++++++++++++++++---\n> 1 file changed, 19 insertions(+), 3 deletions(-)\n>\n> diff --git a/diff.c b/diff.c\n> index 2f9722b382..89466018e5 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n> \temit_line_0(o, set, reset, line[0], line+1, len-1);\n> }\n> \n> +enum diff_symbol {\n> +\tDIFF_SYMBOL_SEPARATOR,\n\nDrop the last comma from enum?\n\n> +static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n> +\t\t\t const char *line, int len)\n> +{\n> +\tswitch (s) {\n> +\tcase DIFF_SYMBOL_SEPARATOR:\n> +\t\tfprintf(o->file, \"%s%c\",\n> +\t\t\tdiff_line_prefix(o),\n> +\t\t\to->line_termination);\n> +\t\tbreak;\n\nAs the first patch in the \"diff-symbol\" subseries of this topic,\nthis change must seriously be justified. Why is it so important\nthat a printing of an empty line must be moved to a helper function,\nwhich later will gain ability to show other kind of lines?\n\n> +\tdefault:\n> +\t\tdie(\"BUG: unknown diff symbol\");\n> +\t}\n> +}\n> +\n> static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n> {\n> \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n> @@ -4833,9 +4851,7 @@ void diff_flush(struct diff_options *options)\n> \n> \tif (output_format & DIFF_FORMAT_PATCH) {\n> \t\tif (separator) {\n> -\t\t\tfprintf(options->file, \"%s%c\",\n> -\t\t\t\tdiff_line_prefix(options),\n> -\t\t\t\toptions->line_termination);\n> +\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n> \t\t\tif (options->stat_sep) {\n> \t\t\t\t/* attach patch instead of inline */\n> \t\t\t\tfputs(options->stat_sep, options->file);\n"},{"id":"322853","messageId":"CAGZ79kYUpJxX-wBU=ALPgJwVaA8h_iJRtAu3T7p4J7qmy=U4dg@mail.gmail.com","threadId":"46038","inReplyTo":"xmqq7f05p1z7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-21T19:46:09Z","receivedAt":"2017-06-21T19:46:15Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jun 21, 2017 at 12:36 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> Signed-off-by: Stefan Beller \n>> ---\n>> diff.c | 22 +++++++++++++++++++---\n>> 1 file changed, 19 insertions(+), 3 deletions(-)\n>>\n>> diff --git a/diff.c b/diff.c\n>> index 2f9722b382..89466018e5 100644\n>> --- a/diff.c\n>> +++ b/diff.c\n>> @@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n>> emit_line_0(o, set, reset, line[0], line+1, len-1);\n>> }\n>>\n>> +enum diff_symbol {\n>> + DIFF_SYMBOL_SEPARATOR,\n>\n> Drop the last comma from enum?\n\nI looked through out code base and for enums this is\nactually strictly enforced, so I guess I have to play\nby the rules here as I do not want to be the first\nto deviate from an upheld standard.\n\nThis will be painful though as the next ~20 patches\nadd more symbols mostly at the end, maybe I need\nto restructure that such that the last symbol stays the same\nthroughout the series. Thanks for that thought.\n\n>\n>> +static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n>> + const char *line, int len)\n>> +{\n>> + switch (s) {\n>> + case DIFF_SYMBOL_SEPARATOR:\n>> + fprintf(o->file, \"%s%c\",\n>> + diff_line_prefix(o),\n>> + o->line_termination);\n>> + break;\n>\n> As the first patch in the \"diff-symbol\" subseries of this topic,\n> this change must seriously be justified. Why is it so important\n> that a printing of an empty line must be moved to a helper function,\n> which later will gain ability to show other kind of lines?\n\nAh yes. This got lost in comparison to the currently queued series with\ndiff_lines. The justification for the change was in the buffer patch,\nbut now we need to have the justification here.\n\nIn the old series, I had copied the same text in all these\nrefactoring patches, but thought to delete them in this series. The first\nrefactoring patch makes sense though.\n\nThanks,\nStefan\n"},{"id":"322856","messageId":"xmqq37atp0ng.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170620024816.20021-9-sbeller@google.com","subject":"Re: [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T20:05:07Z","receivedAt":"2017-06-21T20:05:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> static void emit_add_line(const char *reset,\n> \t\t\t struct emit_callback *ecbdata,\n> \t\t\t const char *line, int len)\n> {\n> -\temit_line_checked(reset, ecbdata, line, len,\n> -\t\t\t DIFF_FILE_NEW, WSEH_NEW, '+');\n> +\tunsigned flags = WSEH_NEW | ecbdata->ws_rule;\n> +\tif (new_blank_line_at_eof(ecbdata, line, len))\n> +\t\tflags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;\n> +\n> +\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);\n> }\n\nThis is a bit unsatisfactory that the original code didn't even have\nto make a call to new_blank_line_at_eof() at all when we know we are\nnot checking for / coloring whitespace errors, but the function is\ncalled unconditionally in the new code.\n\n> diff --git a/diff.h b/diff.h\n> index 5be1ee77a7..8483ca0991 100644\n> --- a/diff.h\n> +++ b/diff.h\n> @@ -148,9 +148,9 @@ struct diff_options {\n> \tint abbrev;\n> \tint ita_invisible_in_index;\n> /* white-space error highlighting */\n> -#define WSEH_NEW 1\n> -#define WSEH_CONTEXT 2\n> -#define WSEH_OLD 4\n> +#define WSEH_NEW (1<<12)\n> +#define WSEH_CONTEXT (1<<13)\n> +#define WSEH_OLD (1<<14)\n\nHopefully this is a safe conversion, as everything should come from\nparse_ws_error_highlight() that uses these constants.\n\nBut this is brittle; there must be some hint to future readers of\nthis code why these bits begin at 12th (it is linked to 07777 we saw\nearlier); otherwise when they change something that needs to widen\nthat 07777, they will forget to shift these up, which would screw\nup everything that is carefully laid out here.\n\n\n"},{"id":"322857","messageId":"xmqqvanpnlva.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170620130147.75e1fb78@twelve2.svl.corp.google.com","subject":"Re: [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T20:09:45Z","receivedAt":"2017-06-21T20:09:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Tan writes:\n\n> On Mon, 19 Jun 2017 19:48:01 -0700\n> Stefan Beller wrote:\n>\n>> @@ -676,6 +677,14 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n>> \t\t}\n>> \t\temit_line(o, context, reset, line, len);\n>> \t\tbreak;\n>> +\tcase DIFF_SYMBOL_FILEPAIR:\n>> +\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n>> +\t\treset = diff_get_color_opt(o, DIFF_RESET);\n>> +\t\tfprintf(o->file, \"%s%s%s%s%s%s\\n\", diff_line_prefix(o), meta,\n>> +\t\t\tflags ? \"+++ \" : \"--- \",\n>\n> I think that a constant should be defined for this flag, just like for\n> content lines. If not it is not immediately obvious that a flag should\n> be set for the +++ lines.\n\nTrue.\n\n>> +\t\t\tline, reset,\n>> +\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n>> +\t\tbreak;\n\nAlso this is somewhat misnamed in that a call to it only deals with\none half of a filepair, iow, the caller must call it twice to show a\nsingle filepair.\n"},{"id":"322858","messageId":"xmqqr2ydnlok.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170620024816.20021-14-sbeller@google.com","subject":"Re: [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T20:13:47Z","receivedAt":"2017-06-21T20:13:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Signed-off-by: Stefan Beller \n> ---\n> diff.c | 21 ++++++++++++++++-----\n> 1 file changed, 16 insertions(+), 5 deletions(-)\n>\n> diff --git a/diff.c b/diff.c\n> index 95f11ea86f..e56962b844 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -572,6 +572,7 @@ enum diff_symbol {\n> \tDIFF_SYMBOL_WORDS,\n> \tDIFF_SYMBOL_FILEPAIR,\n> \tDIFF_SYMBOL_HEADER,\n> +\tDIFF_SYMBOL_BINARY_FILES,\n> };\n> /*\n> * Flags for content lines:\n> @@ -689,6 +690,10 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n> \tcase DIFF_SYMBOL_HEADER:\n> \t\tfprintf(o->file, \"%s\", line);\n> \t\tbreak;\n> +\tcase DIFF_SYMBOL_BINARY_FILES:\n> +\t\tfprintf(o->file, \"%sBinary files %s differ\\n\",\n> +\t\t\tdiff_line_prefix(o), line);\n> +\t\tbreak;\n\nIt is curious why the \"line\" is defined to be \"A and B\" here. It\ncould have been defined to be the whole string \"Binary files A and B\ndiffer\" or even with the terminating LF.\n\nAnd with that it may have been able to share concepts with the\n\"HEADER\" we see above. SYMBOL_HEADER becomes \"oneline without\nprefix\", and this one (updated to make the caller prepare the whole\nsentence) becomes \"oneline with prefix\".\n\n> @@ -2549,8 +2555,10 @@ static void builtin_diff(const char *name_a,\n> \t\t\t}\n> \t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n> \t\t\t\t\t header.buf, header.len, 0);\n> -\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n> -\t\t\t\tline_prefix, lbl[0], lbl[1]);\n> +\t\t\tstrbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n> +\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n> +\t\t\t\t\t sb.buf, sb.len, 0);\n> +\t\t\tstrbuf_release(&sb);\n> \t\t\tgoto free_ab_and_return;\n> \t\t}\n> \t\tif (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)\n> @@ -2567,9 +2575,12 @@ static void builtin_diff(const char *name_a,\n> \t\tstrbuf_reset(&header);\n> \t\tif (DIFF_OPT_TST(o, BINARY))\n> \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n> -\t\telse\n> -\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n> -\t\t\t\tline_prefix, lbl[0], lbl[1]);\n> +\t\telse {\n> +\t\t\tstrbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n> +\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n> +\t\t\t\t\t sb.buf, sb.len, 0);\n> +\t\t\tstrbuf_release(&sb);\n> +\t\t}\n> \t\to->found_changes = 1;\n> \t} else {\n> \t\t/* Crazy xdl interfaces.. */\n"},{"id":"322859","messageId":"xmqqmv91nlkd.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170620024816.20021-17-sbeller@google.com","subject":"Re: [PATCH 16/26] diff.c: convert emit_binary_diff_body to use emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T20:16:18Z","receivedAt":"2017-06-21T20:16:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Signed-off-by: Stefan Beller \n> ---\n> diff.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++-----------------\n> 1 file changed, 49 insertions(+), 17 deletions(-)\n>\n> diff --git a/diff.c b/diff.c\n> index 42a9020d95..e6ade5fde0 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -573,6 +573,11 @@ enum diff_symbol {\n> \tDIFF_SYMBOL_FILEPAIR,\n> \tDIFF_SYMBOL_HEADER,\n> \tDIFF_SYMBOL_BINARY_FILES,\n> +\tDIFF_SYMBOL_BINARY_DIFF_HEADER,\n> +\tDIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n> +\tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n> +\tDIFF_SYMBOL_BINARY_DIFF_BODY,\n> +\tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n> \tDIFF_SYMBOL_REWRITE_DIFF,\n> \tDIFF_SYMBOL_SUBMODULE_ADD,\n> \tDIFF_SYMBOL_SUBMODULE_DEL,\n> @@ -581,6 +586,7 @@ enum diff_symbol {\n> \tDIFF_SYMBOL_SUBMODULE_HEADER,\n> \tDIFF_SYMBOL_SUBMODULE_ERROR,\n> \tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n> +\n\nLine noise?\n\n> };\n"},{"id":"322860","messageId":"xmqqinjpnl89.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170620024816.20021-25-sbeller@google.com","subject":"Re: [PATCH 24/26] diff.c: add dimming to moved line detection","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T20:23:34Z","receivedAt":"2017-06-21T20:23:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Any lines inside a moved block of code are not interesting. Boundaries\n> of blocks are only interesting if they are next to another block of moved\n> code.\n>\n> Signed-off-by: Stefan Beller \n> ---\n> color.h | 2 +\n> diff.c | 139 ++++++++++++++++++++++++++++++++++++++++-----\n> diff.h | 9 ++-\n> t/t4015-diff-whitespace.sh | 124 ++++++++++++++++++++++++++++++++++++++++\n> 4 files changed, 259 insertions(+), 15 deletions(-)\n>\n> diff --git a/color.h b/color.h\n> index 90627650fc..0e091b0cf5 100644\n> --- a/color.h\n> +++ b/color.h\n> @@ -42,6 +42,8 @@ struct strbuf;\n> #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n> #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n> #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n> +#define GIT_COLOR_DI\t\t\"\\033[2m\"\n> +#define GIT_COLOR_DI_IT\t\t\"\\033[2;3m\"\n\nI can and I think most naive readers understand what BOLD, BG and\nBLUE means, but I do not think the same can be said to DI and IT.\nIf you meant FAINT and ITALIC, it is better to spell them out. You\ndo not have to type them to too many places anyway.\n\n"},{"id":"322861","messageId":"xmqqefudnl2m.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"CAGZ79kYUpJxX-wBU=ALPgJwVaA8h_iJRtAu3T7p4J7qmy=U4dg@mail.gmail.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T20:26:57Z","receivedAt":"2017-06-21T20:27:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> This will be painful though as the next ~20 patches\n> add more symbols mostly at the end.\n\n> , maybe I need\n> to restructure that such that the last symbol stays the same\n> throughout the series. Thanks for that thought.\n\nYes, this mostly-no-op one added in the first patch can be kept\nalways at the end.\n\n"},{"id":"322862","messageId":"CAGZ79kbfMumUjfUx4MzLWEXzXtv9GtaCohW4-QxJZ=5bAqcqng@mail.gmail.com","threadId":"46038","inReplyTo":"xmqqr2ydnlok.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 13/26] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-21T20:47:26Z","receivedAt":"2017-06-21T20:47:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jun 21, 2017 at 1:13 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> Signed-off-by: Stefan Beller \n>> ---\n>> diff.c | 21 ++++++++++++++++-----\n>> 1 file changed, 16 insertions(+), 5 deletions(-)\n>>\n>> diff --git a/diff.c b/diff.c\n>> index 95f11ea86f..e56962b844 100644\n>> --- a/diff.c\n>> +++ b/diff.c\n>> @@ -572,6 +572,7 @@ enum diff_symbol {\n>> DIFF_SYMBOL_WORDS,\n>> DIFF_SYMBOL_FILEPAIR,\n>> DIFF_SYMBOL_HEADER,\n>> + DIFF_SYMBOL_BINARY_FILES,\n>> };\n>> /*\n>> * Flags for content lines:\n>> @@ -689,6 +690,10 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n>> case DIFF_SYMBOL_HEADER:\n>> fprintf(o->file, \"%s\", line);\n>> break;\n>> + case DIFF_SYMBOL_BINARY_FILES:\n>> + fprintf(o->file, \"%sBinary files %s differ\\n\",\n>> + diff_line_prefix(o), line);\n>> + break;\n>\n> It is curious why the \"line\" is defined to be \"A and B\" here. It\n> could have been defined to be the whole string \"Binary files A and B\n> differ\" or even with the terminating LF.\n>\n> And with that it may have been able to share concepts with the\n> \"HEADER\" we see above. SYMBOL_HEADER becomes \"oneline without\n> prefix\", and this one (updated to make the caller prepare the whole\n> sentence) becomes \"oneline with prefix\".\n\nI thought about saving memory in the buffered operation,\nbut for simplicity it may be better to make them the same.\nThen they can even share the implementation.\nWith that thought, I'll check more different symbols to share\nthe same implementation.\n\n>\n>> @@ -2549,8 +2555,10 @@ static void builtin_diff(const char *name_a,\n>> }\n>> emit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n>> header.buf, header.len, 0);\n>> - fprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n>> - line_prefix, lbl[0], lbl[1]);\n>> + strbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n>> + emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n>> + sb.buf, sb.len, 0);\n>> + strbuf_release(&sb);\n>> goto free_ab_and_return;\n>> }\n>> if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)\n>> @@ -2567,9 +2575,12 @@ static void builtin_diff(const char *name_a,\n>> strbuf_reset(&header);\n>> if (DIFF_OPT_TST(o, BINARY))\n>> emit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n>> - else\n>> - fprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n>> - line_prefix, lbl[0], lbl[1]);\n>> + else {\n>> + strbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n>> + emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n>> + sb.buf, sb.len, 0);\n>> + strbuf_release(&sb);\n>> + }\n>> o->found_changes = 1;\n>> } else {\n>> /* Crazy xdl interfaces.. */\n"},{"id":"322866","messageId":"xmqq7f05niwt.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"CAGZ79kYUpJxX-wBU=ALPgJwVaA8h_iJRtAu3T7p4J7qmy=U4dg@mail.gmail.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T21:13:38Z","receivedAt":"2017-06-21T21:13:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n>>>\n>>> +enum diff_symbol {\n>>> + DIFF_SYMBOL_SEPARATOR,\n>>\n>> Drop the last comma from enum?\n>\n> I looked through out code base and for enums this is\n> actually strictly enforced, so I guess I have to play\n> by the rules here as I do not want to be the first\n> to deviate from an upheld standard.\n\nYou sound like you are trying to find an excuse \"not to play by the\nrules\". Don't.\n\n\n"},{"id":"322870","messageId":"CAGZ79kZnDdtB6O5jnt7ouG-qoOS90kZALqZB_-gOT0OQcJzvRg@mail.gmail.com","threadId":"46038","inReplyTo":"xmqq7f05niwt.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-21T21:23:14Z","receivedAt":"2017-06-21T21:23:35Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jun 21, 2017 at 2:13 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>>>>\n>>>> +enum diff_symbol {\n>>>> + DIFF_SYMBOL_SEPARATOR,\n>>>\n>>> Drop the last comma from enum?\n>>\n>> I looked through out code base and for enums this is\n>> actually strictly enforced, so I guess I have to play\n>> by the rules here as I do not want to be the first\n>> to deviate from an upheld standard.\n>\n> You sound like you are trying to find an excuse \"not to play by the\n> rules\". Don't.\n>\n\nWhen the rules are not yielding best results, adapting the rules\nshould be considered. And having no comma at the last element\nproduces review noise when working in a line oriented format, such\nas diffs. (Add another element at the end, and then you have to\nhave the diff that just changes the comma)\n\n C99 allows a trailing comma to follow the last enumeration\n constant initializer within an enumeration type declaration,\n similar to structure member initialization lists.\n C99: §6.7.2.2\n\n For struct member lists we already do that, e.g.\n builtin/help.c: struct common_guides[]\n builtin/repack.c: struct exts[]\n ...\n\nYet, I will construct this series in a resend to not\nhave this issue.\n\nThanks,\nStefan\n"},{"id":"322874","messageId":"20170621213949.GD53348@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-18-sbeller@google.com","subject":"Re: [PATCH 17/26] diff.c: convert show_stats to use emit_diff_symbol","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-06-21T21:39:49Z","receivedAt":"2017-06-21T21:40:03Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 06/19, Stefan Beller wrote:\n> We call print_stat_summary from builtin/apply, so we still\n> need the version with a file pointer, so introduce\n> print_stat_summary_0 that uses emit_string machinery and\n> keep print_stat_summary with the same arguments around.\n> \n> Signed-off-by: Stefan Beller \n> ---\n> diff.c | 118 +++++++++++++++++++++++++++++++++++++++++------------------------\n> diff.h | 4 +--\n> 2 files changed, 77 insertions(+), 45 deletions(-)\n> \n> diff --git a/diff.c b/diff.c\n> index e6ade5fde0..0a5ff2500f 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -586,7 +586,10 @@ enum diff_symbol {\n> \tDIFF_SYMBOL_SUBMODULE_HEADER,\n> \tDIFF_SYMBOL_SUBMODULE_ERROR,\n> \tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n> -\n> +\tDIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n> +\tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n> +\tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n> +\tDIFF_SYMBOL_STATS_LINE,\n> };\n> /*\n> * Flags for content lines:\n> @@ -628,6 +631,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n> {\n> \tstatic const char *nneof = \" No newline at end of file\\n\";\n> \tconst char *context, *reset, *set, *meta, *fraginfo;\n> +\tstruct strbuf sb = STRBUF_INIT;\n> \tswitch (s) {\n> \tcase DIFF_SYMBOL_SEPARATOR:\n> \t\tfprintf(o->file, \"%s%c\",\n> @@ -756,9 +760,22 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n> \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n> \t\temit_line(o, \"\", \"\", line, len);\n> \t\tbreak;\n> +\tcase DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:\n> +\t\tfprintf(o->file, \" 0 files changed\\n\");\n\nWhy is this one a fprintf call instead of an emit_line call? I'm sure\nthis is intended It just stands out to me.\n\n> +\t\tbreak;\n> +\tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n> +\t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n> +\t\tbreak;\n> +\tcase DIFF_SYMBOL_STATS_LINE:\n> +\t\temit_line(o, \"\", \"\", line, len);\n> +\t\tbreak;\n> +\tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n> +\t\temit_line(o, \"\", \"\", line, len);\n> +\t\tbreak;\n> \tdefault:\n> \t\tdie(\"BUG: unknown diff symbol\");\n> \t}\n> +\tstrbuf_release(&sb);\n> }\n> \n> void diff_emit_submodule_del(struct diff_options *o, const char *line)\n> @@ -1712,20 +1729,14 @@ static int scale_linear(int it, int width, int max_change)\n> \treturn 1 + (it * (width - 1) / max_change);\n> }\n> \n> -static void show_name(FILE *file,\n> -\t\t const char *prefix, const char *name, int len)\n> -{\n> -\tfprintf(file, \" %s%-*s |\", prefix, len, name);\n> -}\n> -\n> -static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)\n> +static void show_graph(struct strbuf *out, char ch, int cnt,\n> +\t\t const char *set, const char *reset)\n> {\n> \tif (cnt <= 0)\n> \t\treturn;\n> -\tfprintf(file, \"%s\", set);\n> -\twhile (cnt--)\n> -\t\tputc(ch, file);\n> -\tfprintf(file, \"%s\", reset);\n> +\tstrbuf_addstr(out, set);\n> +\tstrbuf_addchars(out, ch, cnt);\n> +\tstrbuf_addstr(out, reset);\n> }\n> \n> static void fill_print_name(struct diffstat_file *file)\n> @@ -1749,14 +1760,16 @@ static void fill_print_name(struct diffstat_file *file)\n> \tfile->print_name = pname;\n> }\n> \n> -int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n> +static void print_stat_summary_inserts_deletes(struct diff_options *options,\n> +\t\tint files, int insertions, int deletions)\n> {\n> \tstruct strbuf sb = STRBUF_INIT;\n> -\tint ret;\n> \n> \tif (!files) {\n> \t\tassert(insertions == 0 && deletions == 0);\n> -\t\treturn fprintf(fp, \"%s\\n\", \" 0 files changed\");\n> +\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n> +\t\t\t\t NULL, 0, 0);\n> +\t\treturn;\n> \t}\n> \n> \tstrbuf_addf(&sb,\n> @@ -1783,9 +1796,19 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n> \t\t\t deletions);\n> \t}\n> \tstrbuf_addch(&sb, '\\n');\n> -\tret = fputs(sb.buf, fp);\n> +\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n> +\t\t\t sb.buf, sb.len, 0);\n> \tstrbuf_release(&sb);\n> -\treturn ret;\n> +}\n> +\n> +void print_stat_summary(FILE *fp, int files,\n> +\t\t\tint insertions, int deletions)\n> +{\n> +\tstruct diff_options o;\n> +\tmemset(&o, 0, sizeof(o));\n> +\to.file = fp;\n> +\n> +\tprint_stat_summary_inserts_deletes(&o, files, insertions, deletions);\n> }\n> \n> static void show_stats(struct diffstat_t *data, struct diff_options *options)\n> @@ -1795,13 +1818,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n> \tint total_files = data->nr, count;\n> \tint width, name_width, graph_width, number_width = 0, bin_width = 0;\n> \tconst char *reset, *add_c, *del_c;\n> -\tconst char *line_prefix = \"\";\n> \tint extra_shown = 0;\n> +\tconst char *line_prefix = diff_line_prefix(options);\n> +\tstruct strbuf out = STRBUF_INIT;\n> \n> \tif (data->nr == 0)\n> \t\treturn;\n> \n> -\tline_prefix = diff_line_prefix(options);\n> \tcount = options->stat_count ? options->stat_count : data->nr;\n> \n> \treset = diff_get_color_opt(options, DIFF_RESET);\n> @@ -1955,26 +1978,32 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n> \t\t}\n> \n> \t\tif (file->is_binary) {\n> -\t\t\tfprintf(options->file, \"%s\", line_prefix);\n> -\t\t\tshow_name(options->file, prefix, name, len);\n> -\t\t\tfprintf(options->file, \" %*s\", number_width, \"Bin\");\n> +\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n> +\t\t\tstrbuf_addf(&out, \" %*s\", number_width, \"Bin\");\n> \t\t\tif (!added && !deleted) {\n> -\t\t\t\tputc('\\n', options->file);\n> +\t\t\t\tstrbuf_addch(&out, '\\n');\n> +\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n> +\t\t\t\t\t\t out.buf, out.len, 0);\n> +\t\t\t\tstrbuf_reset(&out);\n> \t\t\t\tcontinue;\n> \t\t\t}\n> -\t\t\tfprintf(options->file, \" %s%\"PRIuMAX\"%s\",\n> +\t\t\tstrbuf_addf(&out, \" %s%\"PRIuMAX\"%s\",\n> \t\t\t\tdel_c, deleted, reset);\n> -\t\t\tfprintf(options->file, \" -> \");\n> -\t\t\tfprintf(options->file, \"%s%\"PRIuMAX\"%s\",\n> +\t\t\tstrbuf_addstr(&out, \" -> \");\n> +\t\t\tstrbuf_addf(&out, \"%s%\"PRIuMAX\"%s\",\n> \t\t\t\tadd_c, added, reset);\n> -\t\t\tfprintf(options->file, \" bytes\");\n> -\t\t\tfprintf(options->file, \"\\n\");\n> +\t\t\tstrbuf_addstr(&out, \" bytes\\n\");\n> +\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n> +\t\t\t\t\t out.buf, out.len, 0);\n> +\t\t\tstrbuf_reset(&out);\n> \t\t\tcontinue;\n> \t\t}\n> \t\telse if (file->is_unmerged) {\n> -\t\t\tfprintf(options->file, \"%s\", line_prefix);\n> -\t\t\tshow_name(options->file, prefix, name, len);\n> -\t\t\tfprintf(options->file, \" Unmerged\\n\");\n> +\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n> +\t\t\tstrbuf_addstr(&out, \" Unmerged\\n\");\n> +\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n> +\t\t\t\t\t out.buf, out.len, 0);\n> +\t\t\tstrbuf_reset(&out);\n> \t\t\tcontinue;\n> \t\t}\n> \n> @@ -1997,14 +2026,16 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n> \t\t\t\tadd = total - del;\n> \t\t\t}\n> \t\t}\n> -\t\tfprintf(options->file, \"%s\", line_prefix);\n> -\t\tshow_name(options->file, prefix, name, len);\n> -\t\tfprintf(options->file, \" %*\"PRIuMAX\"%s\",\n> +\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n> +\t\tstrbuf_addf(&out, \" %*\"PRIuMAX\"%s\",\n> \t\t\tnumber_width, added + deleted,\n> \t\t\tadded + deleted ? \" \" : \"\");\n> -\t\tshow_graph(options->file, '+', add, add_c, reset);\n> -\t\tshow_graph(options->file, '-', del, del_c, reset);\n> -\t\tfprintf(options->file, \"\\n\");\n> +\t\tshow_graph(&out, '+', add, add_c, reset);\n> +\t\tshow_graph(&out, '-', del, del_c, reset);\n> +\t\tstrbuf_addch(&out, '\\n');\n> +\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n> +\t\t\t\t out.buf, out.len, 0);\n> +\t\tstrbuf_reset(&out);\n> \t}\n> \n> \tfor (i = 0; i < data->nr; i++) {\n> @@ -2025,11 +2056,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n> \t\tif (i < count)\n> \t\t\tcontinue;\n> \t\tif (!extra_shown)\n> -\t\t\tfprintf(options->file, \"%s ...\\n\", line_prefix);\n> +\t\t\temit_diff_symbol(options,\n> +\t\t\t\t\t DIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n> +\t\t\t\t\t NULL, 0, 0);\n> \t\textra_shown = 1;\n> \t}\n> -\tfprintf(options->file, \"%s\", line_prefix);\n> -\tprint_stat_summary(options->file, total_files, adds, dels);\n> +\n> +\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n> }\n> \n> static void show_shortstats(struct diffstat_t *data, struct diff_options *options)\n> @@ -2041,7 +2074,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n> \n> \tfor (i = 0; i < data->nr; i++) {\n> \t\tint added = data->files[i]->added;\n> -\t\tint deleted= data->files[i]->deleted;\n> +\t\tint deleted = data->files[i]->deleted;\n> \n> \t\tif (data->files[i]->is_unmerged ||\n> \t\t (!data->files[i]->is_interesting && (added + deleted == 0))) {\n> @@ -2051,8 +2084,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n> \t\t\tdels += deleted;\n> \t\t}\n> \t}\n> -\tfprintf(options->file, \"%s\", diff_line_prefix(options));\n> -\tprint_stat_summary(options->file, total_files, adds, dels);\n> +\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n> }\n> \n> static void show_numstat(struct diffstat_t *data, struct diff_options *options)\n> diff --git a/diff.h b/diff.h\n> index 2ee0ef3908..b73f7a1e75 100644\n> --- a/diff.h\n> +++ b/diff.h\n> @@ -398,8 +398,8 @@ extern int parse_rename_score(const char **cp_p);\n> \n> extern long parse_algorithm_value(const char *value);\n> \n> -extern int print_stat_summary(FILE *fp, int files,\n> -\t\t\t int insertions, int deletions);\n> +extern void print_stat_summary(FILE *fp, int files,\n> +\t\t\t int insertions, int deletions);\n> extern void setup_diff_pager(struct diff_options *);\n> \n> #endif /* DIFF_H */\n> -- \n> 2.12.2.575.gb14f27f917\n> \n\n-- \nBrandon Williams\n"},{"id":"322877","messageId":"xmqqtw39m2y6.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"CAGZ79kZnDdtB6O5jnt7ouG-qoOS90kZALqZB_-gOT0OQcJzvRg@mail.gmail.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T21:43:45Z","receivedAt":"2017-06-21T21:43:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n>>> I looked through out code base and for enums this is\n>>> actually strictly enforced, so I guess I have to play\n>>> by the rules here as I do not want to be the first\n>>> to deviate from an upheld standard.\n>>\n>> You sound like you are trying to find an excuse \"not to play by the\n>> rules\". Don't.\n>\n> When the rules are not yielding best results, adapting the rules\n> should be considered.\n\nI do not think anybody is saying that it is unreasonable to have a\nwish that it would be wonderful if we could use C99 features. \n\nTrailing comma at the end of enum is the least interesting addition\n(it is a fix for a previous editions' brown-paper-bag mistake). I'd\nbe delighted if we can drop support for older compilers and start\nusing designated initializers, for example.\n\nFinding other's violation of the current coding guideline is *not*\nthe right way to promote for that future, however.\n\n"},{"id":"322878","messageId":"20170621214551.GE53348@google.com","threadId":"46038","inReplyTo":"CAGZ79kYUpJxX-wBU=ALPgJwVaA8h_iJRtAu3T7p4J7qmy=U4dg@mail.gmail.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-06-21T21:45:51Z","receivedAt":"2017-06-21T21:45:58Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 06/21, Stefan Beller wrote:\n> On Wed, Jun 21, 2017 at 12:36 PM, Junio C Hamano wrote:\n> > Stefan Beller writes:\n> >\n> >> Signed-off-by: Stefan Beller \n> >> ---\n> >> diff.c | 22 +++++++++++++++++++---\n> >> 1 file changed, 19 insertions(+), 3 deletions(-)\n> >>\n> >> diff --git a/diff.c b/diff.c\n> >> index 2f9722b382..89466018e5 100644\n> >> --- a/diff.c\n> >> +++ b/diff.c\n> >> @@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n> >> emit_line_0(o, set, reset, line[0], line+1, len-1);\n> >> }\n> >>\n> >> +enum diff_symbol {\n> >> + DIFF_SYMBOL_SEPARATOR,\n> >\n> > Drop the last comma from enum?\n> \n> I looked through out code base and for enums this is\n> actually strictly enforced, so I guess I have to play\n> by the rules here as I do not want to be the first\n> to deviate from an upheld standard.\n> \n> This will be painful though as the next ~20 patches\n> add more symbols mostly at the end, maybe I need\n> to restructure that such that the last symbol stays the same\n> throughout the series. Thanks for that thought.\n\nI don't think this is strictly enforced. If you look at grep.h:197 the\nenum 'grep_source_type' has a trailing comma.\n\n> \n> >\n> >> +static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n> >> + const char *line, int len)\n> >> +{\n> >> + switch (s) {\n> >> + case DIFF_SYMBOL_SEPARATOR:\n> >> + fprintf(o->file, \"%s%c\",\n> >> + diff_line_prefix(o),\n> >> + o->line_termination);\n> >> + break;\n> >\n> > As the first patch in the \"diff-symbol\" subseries of this topic,\n> > this change must seriously be justified. Why is it so important\n> > that a printing of an empty line must be moved to a helper function,\n> > which later will gain ability to show other kind of lines?\n> \n> Ah yes. This got lost in comparison to the currently queued series with\n> diff_lines. The justification for the change was in the buffer patch,\n> but now we need to have the justification here.\n> \n> In the old series, I had copied the same text in all these\n> refactoring patches, but thought to delete them in this series. The first\n> refactoring patch makes sense though.\n> \n> Thanks,\n> Stefan\n\n-- \nBrandon Williams\n"},{"id":"322880","messageId":"20170621215104.GF53348@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-27-sbeller@google.com","subject":"Re: [showing-off RFC/PATCH 26/26] diff.c: have a \"machine parseable\" move coloring","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-06-21T21:51:04Z","receivedAt":"2017-06-21T21:51:12Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 06/19, Stefan Beller wrote:\n> Ævar asked for it, this is how you would do it.\n> (plus documentation, tests, CLI knobs, options)\n> \n> Signed-off-by: Stefan Beller \n> ---\n> diff.c | 15 +++++++++++----\n> diff.h | 2 ++\n> 2 files changed, 13 insertions(+), 4 deletions(-)\n> \n> diff --git a/diff.c b/diff.c\n> index 7756f7610c..61caa057ff 100644\n> --- a/diff.c\n> +++ b/diff.c\n> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n> \tstatic const char *nneof = \" No newline at end of file\\n\";\n> \tconst char *context, *reset, *set, *meta, *fraginfo;\n> \tstruct strbuf sb = STRBUF_INIT;\n> +\tint sign;\n\nshould this be a char instead of an int?\n\n> \n> \tenum diff_symbol s = eds->s;\n> \tconst char *line = eds->line;\n> @@ -1058,8 +1059,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n> \t\tdefault:\n> \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n> \t\t}\n> +\t\tsign = '+';\n> +\t\tif (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)\n> +\t\t\tsign = '*';\n> \t\treset = diff_get_color_opt(o, DIFF_RESET);\n> -\t\temit_line_ws_markup(o, set, reset, line, len, '+',\n> +\t\temit_line_ws_markup(o, set, reset, line, len, sign,\n> \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n> \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n> \t\tbreak;\n> @@ -1086,8 +1090,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n> \t\tdefault:\n> \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n> \t\t}\n> +\t\tsign = '-';\n> +\t\tif (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)\n> +\t\t\tsign = '_';\n> \t\treset = diff_get_color_opt(o, DIFF_RESET);\n> -\t\temit_line_ws_markup(o, set, reset, line, len, '-',\n> +\t\temit_line_ws_markup(o, set, reset, line, len, sign,\n> \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n> \t\tbreak;\n> \tcase DIFF_SYMBOL_WORDS_PORCELAIN:\n> @@ -5475,7 +5482,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n> \tstatic struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n> \tstruct diff_queue_struct *q = &diff_queued_diff;\n> \n> -\tif (o->color_moved)\n> +\tif (o->color_moved || o->machine_readable_moves)\n> \t\to->emitted_symbols = &esm;\n> \n> \tfor (i = 0; i < q->nr; i++) {\n> @@ -5485,7 +5492,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n> \t}\n> \n> \tif (o->emitted_symbols) {\n> -\t\tif (o->color_moved) {\n> +\t\tif (o->color_moved || o->machine_readable_moves) {\n> \t\t\tstruct hashmap add_lines, del_lines;\n> \t\t\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n> \n> diff --git a/diff.h b/diff.h\n> index 98abd75521..b6c4d7ab0f 100644\n> --- a/diff.h\n> +++ b/diff.h\n> @@ -194,6 +194,8 @@ struct diff_options {\n> \t\tCOLOR_MOVED_ZEBRA = 2,\n> \t\tCOLOR_MOVED_ZEBRA_DIM = 3,\n> \t} color_moved;\n> +\n> +\tint machine_readable_moves;\n> };\n> \n> void diff_emit_submodule_del(struct diff_options *o, const char *line);\n> -- \n> 2.12.2.575.gb14f27f917\n> \n\n-- \nBrandon Williams\n"},{"id":"322881","messageId":"CAGZ79kbvYBR2M1BRxUPx_e0m8h0TFLx+D44Ueh4XZ6bRMhr7qg@mail.gmail.com","threadId":"46038","inReplyTo":"xmqqtw39m2y6.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-21T21:51:00Z","receivedAt":"2017-06-21T21:51:26Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jun 21, 2017 at 2:43 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>>>> I looked through out code base and for enums this is\n>>>> actually strictly enforced, so I guess I have to play\n>>>> by the rules here as I do not want to be the first\n>>>> to deviate from an upheld standard.\n>>>\n>>> You sound like you are trying to find an excuse \"not to play by the\n>>> rules\". Don't.\n>>\n>> When the rules are not yielding best results, adapting the rules\n>> should be considered.\n>\n> I do not think anybody is saying that it is unreasonable to have a\n> wish that it would be wonderful if we could use C99 features.\n>\n> Trailing comma at the end of enum is the least interesting addition\n> (it is a fix for a previous editions' brown-paper-bag mistake). I'd\n> be delighted if we can drop support for older compilers and start\n> using designated initializers, for example.\n>\n> Finding other's violation of the current coding guideline is *not*\n> the right way to promote for that future, however.\n\nIn practice it is, though.\n\nDue to the nature of this project we cannot obtain a world view\nof our users (and compilers used), such that it is impossible to say\nwhen the last usage with a strict C89 happens.\n\nIf however there is code violating the C89 style for a long\nenough time and nobody speaks up, we can assume we can\nsafely upgrade to C99.\n\nTo do this in a controlled (ideal) way, we would have a\nsingle location in the code where we'd use a new desired\nfeature from C99, such that a \"rollback\" to strict C89 is\neasy either by reverting to the old style or even just reverting\na commit as the occurrence was in a dummy file.\n\nBy violating the style guide in more than just one place\nthis is cumbersome and should not be done, which is\nwhy I said I will not do it in this series.\n\nThanks,\nStefan\n"},{"id":"322882","messageId":"xmqqlgolm2jk.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170621214551.GE53348@google.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T21:52:31Z","receivedAt":"2017-06-21T21:52:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> I don't think this is strictly enforced. If you look at grep.h:197 the\n> enum 'grep_source_type' has a trailing comma.\n\nOK, so we now know that at least since e1327023 (\"grep: refactor the\nconcept of \"grep source\" into an object\", 2012-02-02) we haven't\nheard from anybody whose compiler fails to grok this.\n\nPerhaps we should start pushing for C99 features?\n"},{"id":"322883","messageId":"20170621215500.GG53348@google.com","threadId":"46038","inReplyTo":"xmqqlgolm2jk.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 04/26] diff.c: introduce emit_diff_symbol","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-06-21T21:55:00Z","receivedAt":"2017-06-21T21:55:26Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 06/21, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > I don't think this is strictly enforced. If you look at grep.h:197 the\n> > enum 'grep_source_type' has a trailing comma.\n> \n> OK, so we now know that at least since e1327023 (\"grep: refactor the\n> concept of \"grep source\" into an object\", 2012-02-02) we haven't\n> heard from anybody whose compiler fails to grok this.\n> \n> Perhaps we should start pushing for C99 features?\n\nYeah for a second I thought I introduced the trailing comma when i added\nrecursion to grep...but you're right, looking at blame shows that the\ntrailing comma has been there for much longer.\n\nAnd I'm all for moving to C99 :)\n\n-- \nBrandon Williams\n"},{"id":"322884","messageId":"xmqqh8z9m2ed.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170621215104.GF53348@google.com","subject":"Re: [showing-off RFC/PATCH 26/26] diff.c: have a \"machine parseable\" move coloring","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-21T21:55:38Z","receivedAt":"2017-06-21T21:55:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> On 06/19, Stefan Beller wrote:\n>> Ævar asked for it, this is how you would do it.\n>> (plus documentation, tests, CLI knobs, options)\n>> \n>> Signed-off-by: Stefan Beller \n>> ---\n>> diff.c | 15 +++++++++++----\n>> diff.h | 2 ++\n>> 2 files changed, 13 insertions(+), 4 deletions(-)\n>> \n>> diff --git a/diff.c b/diff.c\n>> index 7756f7610c..61caa057ff 100644\n>> --- a/diff.c\n>> +++ b/diff.c\n>> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n>> \tstatic const char *nneof = \" No newline at end of file\\n\";\n>> \tconst char *context, *reset, *set, *meta, *fraginfo;\n>> \tstruct strbuf sb = STRBUF_INIT;\n>> +\tint sign;\n>\n> should this be a char instead of an int?\n\nPerhaps. Once we start adding things other than + and -, I think\nthey should no longer be called \"sign\", though.\n\nIf Stefan chose '*' as a replacement for '+' because both makes\nthings larger (yes, I am a positive person and my numbers are all\npositive), probably the replacement for '-' should be '/' (or '%'),\nnot '_'. But that is just bikeshedding.\n\n"},{"id":"322886","messageId":"CAGZ79kbQRy_gv7ZYiFM6CbF-sDnBVpP=dP+qa7aqxPH+MiQNig@mail.gmail.com","threadId":"46038","inReplyTo":"20170621213949.GD53348@google.com","subject":"Re: [PATCH 17/26] diff.c: convert show_stats to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-21T22:16:04Z","receivedAt":"2017-06-21T22:16:09Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":">> case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n>> emit_line(o, \"\", \"\", line, len);\n>> break;\n>> + case DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:\n>> + fprintf(o->file, \" 0 files changed\\n\");\n>\n> Why is this one a fprintf call instead of an emit_line call? I'm sure\n> this is intended It just stands out to me.\n\nI am sure it is unintended to stand out in such a way.\nOriginally I wanted to make it as close 1:1 conversion,\nand as we did not emit the line prefix at the time but relied on\nthe line prefix being emitted prior to when this was printed.\nWill convert to emit_line as that seems to be a bug in this patch.\n"},{"id":"322890","messageId":"CAGZ79kYDBZG4AOueJjvQ61yyC-DQk-8cb5-y18fg2k4JUsQPKw@mail.gmail.com","threadId":"46038","inReplyTo":"xmqqh8z9m2ed.fsf@gitster.mtv.corp.google.com","subject":"Re: [showing-off RFC/PATCH 26/26] diff.c: have a \"machine parseable\" move coloring","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-21T22:40:31Z","receivedAt":"2017-06-21T22:40:37Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jun 21, 2017 at 2:55 PM, Junio C Hamano wrote:\n> Brandon Williams writes:\n>\n>> On 06/19, Stefan Beller wrote:\n>>> Ævar asked for it, this is how you would do it.\n>>> (plus documentation, tests, CLI knobs, options)\n>>>\n>>> Signed-off-by: Stefan Beller \n>>> ---\n>>> diff.c | 15 +++++++++++----\n>>> diff.h | 2 ++\n>>> 2 files changed, 13 insertions(+), 4 deletions(-)\n>>>\n>>> diff --git a/diff.c b/diff.c\n>>> index 7756f7610c..61caa057ff 100644\n>>> --- a/diff.c\n>>> +++ b/diff.c\n>>> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n>>> static const char *nneof = \" No newline at end of file\\n\";\n>>> const char *context, *reset, *set, *meta, *fraginfo;\n>>> struct strbuf sb = STRBUF_INIT;\n>>> + int sign;\n>>\n>> should this be a char instead of an int?\n>\n> Perhaps. Once we start adding things other than + and -, I think\n> they should no longer be called \"sign\", though.\n\nWell 'sign' is currently meant as a mathematical operation,\nbut it could also be meant as a street sign?\n\nThe empty sign ' ' tells you it is context, and not subject\nto bikeshedding whilst reviewing a patch. :)\n\n>\n> If Stefan chose '*' as a replacement for '+' because both makes\n> things larger (yes, I am a positive person and my numbers are all\n> positive), probably the replacement for '-' should be '/' (or '%'),\n> not '_'. But that is just bikeshedding.\n\nI was trying to choose characters that sort of resembled the shape,\nbut slightly differently. But a machine would not care any ways.\n\nThere was also temptation to use bogus signs like 'M' and 'm' for\nmoved (capital M for + and lower m for -, obviously).\n\nBut this is a proof of concept on top of the series showing that\nthis way with bits set in the flags field we do not need to have\ncolors. The older series was operating on the colors and by that\nfiguring out if it was say at a boundary, but here we would just\nuse the flagbits to come up with a reasonable first character.\n(for example: Would the machine parse-able format also care\nabout dimming? I defer such a discussion in the future.)\n"},{"id":"323020","messageId":"CAGZ79kY46UGwBt2A20bwX_CFNuVP8hy_inCqh6hqiA5VfcfSxA@mail.gmail.com","threadId":"46038","inReplyTo":"xmqq37atp0ng.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-22T23:30:01Z","receivedAt":"2017-06-22T23:30:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jun 21, 2017 at 1:05 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> static void emit_add_line(const char *reset,\n>> struct emit_callback *ecbdata,\n>> const char *line, int len)\n>> {\n>> - emit_line_checked(reset, ecbdata, line, len,\n>> - DIFF_FILE_NEW, WSEH_NEW, '+');\n>> + unsigned flags = WSEH_NEW | ecbdata->ws_rule;\n>> + if (new_blank_line_at_eof(ecbdata, line, len))\n>> + flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;\n>> +\n>> + emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);\n>> }\n>\n> This is a bit unsatisfactory that the original code didn't even have\n> to make a call to new_blank_line_at_eof() at all when we know we are\n> not checking for / coloring whitespace errors, but the function is\n> called unconditionally in the new code.\n\nWe could check if we do colored output here, I think'll just do that for now,\nbut on the other hand this becomes a maintenance nightmare when we\nrely on these flags in the future e.g. for \"machine parse-able coloring\"\nand would markup according to the flags strictly. I guess we can fix it then.\n\n>\n>> diff --git a/diff.h b/diff.h\n>> index 5be1ee77a7..8483ca0991 100644\n>> --- a/diff.h\n>> +++ b/diff.h\n>> @@ -148,9 +148,9 @@ struct diff_options {\n>> int abbrev;\n>> int ita_invisible_in_index;\n>> /* white-space error highlighting */\n>> -#define WSEH_NEW 1\n>> -#define WSEH_CONTEXT 2\n>> -#define WSEH_OLD 4\n>> +#define WSEH_NEW (1<<12)\n>> +#define WSEH_CONTEXT (1<<13)\n>> +#define WSEH_OLD (1<<14)\n>\n> Hopefully this is a safe conversion, as everything should come from\n> parse_ws_error_highlight() that uses these constants.\n\nI reviewed the conversion again and I think it is safe, still.\n\nWe could do the up and downshifting for the flags to keep these\nas-is, but I am not sure if that is a good idea, the up and downshifting\nis not just run time complexity but also a readability issue.\n\n>\n> But this is brittle; there must be some hint to future readers of\n> this code why these bits begin at 12th (it is linked to 07777 we saw\n> earlier); otherwise when they change something that needs to widen\n> that 07777, they will forget to shift these up, which would screw\n> up everything that is carefully laid out here.\n\nI added a note there. and a BUG check at the beginning of the output.\nSmart compilers will drop that and for dumb compilers we'll have an\nextra check per commit, which is not too bad\n"},{"id":"323021","messageId":"CAGZ79kben16T9nUpOjsq5-Sv8q7z+C1Nrq++jn2aO17CzqToHw@mail.gmail.com","threadId":"46038","inReplyTo":"CAGZ79kY46UGwBt2A20bwX_CFNuVP8hy_inCqh6hqiA5VfcfSxA@mail.gmail.com","subject":"Re: [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-22T23:37:33Z","receivedAt":"2017-06-22T23:37:39Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Jun 22, 2017 at 4:30 PM, Stefan Beller wrote:\n> On Wed, Jun 21, 2017 at 1:05 PM, Junio C Hamano wrote:\n>> Stefan Beller writes:\n>>\n>>> static void emit_add_line(const char *reset,\n>>> struct emit_callback *ecbdata,\n>>> const char *line, int len)\n>>> {\n>>> - emit_line_checked(reset, ecbdata, line, len,\n>>> - DIFF_FILE_NEW, WSEH_NEW, '+');\n>>> + unsigned flags = WSEH_NEW | ecbdata->ws_rule;\n>>> + if (new_blank_line_at_eof(ecbdata, line, len))\n>>> + flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;\n>>> +\n>>> + emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);\n>>> }\n>>\n>> This is a bit unsatisfactory that the original code didn't even have\n>> to make a call to new_blank_line_at_eof() at all when we know we are\n>> not checking for / coloring whitespace errors, but the function is\n>> called unconditionally in the new code.\n>\n> We could check if we do colored output here, I think'll just do that for now,\n> but on the other hand this becomes a maintenance nightmare when we\n> rely on these flags in the future e.g. for \"machine parse-able coloring\"\n> and would markup according to the flags strictly. I guess we can fix it then.\n\nActually that function already has some quick return:\n\nstatic int new_blank_line_at_eof(struct emit_callback *ecbdata, const\nchar *line, int len)\n{\n if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n ecbdata->blank_at_eof_in_preimage &&\n ecbdata->blank_at_eof_in_postimage &&\n ecbdata->blank_at_eof_in_preimage <= ecbdata->lno_in_preimage &&\n ecbdata->blank_at_eof_in_postimage <= ecbdata->lno_in_postimage))\n return 0;\n return ws_blank_line(line, len, ecbdata->ws_rule);\n}\n\nso maybe we could 'inline' it, as there is no reasonable faster check\nthan what we\nhave in there.\n\nI'll keep it 'a bit unsatisfactory' here and if it actually is a\nperformance issue, we\ncan fix it.\n"},{"id":"323022","messageId":"CAGZ79kZPcQd8oy2Q23v3vN9h667fu+SYjQ8ttDxgoRmH80QQzg@mail.gmail.com","threadId":"46038","inReplyTo":"xmqqvanpnlva.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 11/26] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-22T23:59:33Z","receivedAt":"2017-06-22T23:59:39Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Jun 21, 2017 at 1:09 PM, Junio C Hamano wrote:\n> Jonathan Tan writes:\n>\n>> On Mon, 19 Jun 2017 19:48:01 -0700\n>> Stefan Beller wrote:\n>>\n>>> @@ -676,6 +677,14 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n>>> }\n>>> emit_line(o, context, reset, line, len);\n>>> break;\n>>> + case DIFF_SYMBOL_FILEPAIR:\n>>> + meta = diff_get_color_opt(o, DIFF_METAINFO);\n>>> + reset = diff_get_color_opt(o, DIFF_RESET);\n>>> + fprintf(o->file, \"%s%s%s%s%s%s\\n\", diff_line_prefix(o), meta,\n>>> + flags ? \"+++ \" : \"--- \",\n>>\n>> I think that a constant should be defined for this flag, just like for\n>> content lines. If not it is not immediately obvious that a flag should\n>> be set for the +++ lines.\n>\n> True.\n>\n>>> + line, reset,\n>>> + strchr(line, ' ') ? \"\\t\" : \"\");\n>>> + break;\n>\n> Also this is somewhat misnamed in that a call to it only deals with\n> one half of a filepair, iow, the caller must call it twice to show a\n> single filepair.\n\nSplit into two and renamed to DIFF_SYMBOL_FILEPAIR_PLUS\nand DIFF_SYMBOL_FILEPAIR_MINUS.\n\nThanks,\nStefan\n"},{"id":"323023","messageId":"20170623012919.28693-1-sbeller@google.com","threadId":"46038","inReplyTo":"20170620024816.20021-1-sbeller@google.com","subject":"[PATCHv2 00/25] reroll of sb/diff-color-moved","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:28:54Z","receivedAt":"2017-06-23T01:29:36Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"v2:\n* addressed all issues raised\n* last patch dropped (WIP/RFC: diff.c: have a \"machine parseable\" move coloring)\n* interdiff below\n\nv1:\nThis is a complete rewrite of the series. Highlights:\n* instead of buffering partial lines, we'll pretend all diff output\n follows a well defined grammar, and we emit symbols thereof.\n (The difference is mostly mental, though by this trick we reduce\n the memory footprint for storing one of these symbols from 7 variables\n (3 pointers, 3 ints, one state (also int) down to 4 variables\n (one pointer, 2 ints, one state).\n* The algorithm for color painting was detangled:\n -> different functions for block detection and dimming\n -> The last patch (not to be applied) is an RFC that shows\n how we would approach non-colored, but machine parseable highlighting\n of moved lines.\n\nThanks,\nStefan\n\nStefan Beller (25):\n diff.c: readability fix\n diff.c: move line ending check into emit_hunk_header\n diff.c: factor out diff_flush_patch_all_file_pairs\n diff.c: introduce emit_diff_symbol\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF\n diff.c: migrate emit_line_checked to use emit_diff_symbol\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF\n submodule.c: migrate diff output to use emit_diff_symbol\n diff.c: convert emit_binary_diff_body to use emit_diff_symbol\n diff.c: convert show_stats to use emit_diff_symbol\n diff.c: convert word diffing to use emit_diff_symbol\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY\n diff.c: buffer all output if asked to\n diff.c: color moved lines differently\n diff.c: color moved lines differently, plain mode\n diff.c: add dimming to moved line detection\n diff: document the new --color-moved setting\n\n Documentation/config.txt | 12 +-\n Documentation/diff-options.txt | 27 +\n cache.h | 2 +\n color.h | 2 +\n diff.c | 1270 ++++++++++++++++++++++++++++++++--------\n diff.h | 37 +-\n submodule.c | 85 ++-\n submodule.h | 13 +-\n t/t4015-diff-whitespace.sh | 369 ++++++++++++\n 9 files changed, 1501 insertions(+), 316 deletions(-)\n\ndiff to what is currently queued:\ndiff --git a/cache.h b/cache.h\nindex 4d63c44f07..d2204bf6d1 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -2168,6 +2168,7 @@ void shift_tree_by(const struct object_id *, const struct object_id *, struct ob\n #define WS_TRAILING_SPACE (WS_BLANK_AT_EOL|WS_BLANK_AT_EOF)\n #define WS_DEFAULT_RULE (WS_TRAILING_SPACE|WS_SPACE_BEFORE_TAB|8)\n #define WS_TAB_WIDTH_MASK 077\n+/* All WS_* -- when extended, adapt diff.c emit_symbol */\n #define WS_RULE_MASK 07777\n extern unsigned whitespace_rule_cfg;\n extern unsigned whitespace_rule(const char *);\ndiff --git a/color.h b/color.h\nindex 0e091b0cf5..fd2b688dfb 100644\n--- a/color.h\n+++ b/color.h\n@@ -42,8 +42,8 @@ struct strbuf;\n #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n-#define GIT_COLOR_DI\t\t\"\\033[2m\"\n-#define GIT_COLOR_DI_IT\t\t\"\\033[2;3m\"\n+#define GIT_COLOR_FAINT\t\t\"\\033[2m\"\n+#define GIT_COLOR_FAINT_ITALIC\t\"\\033[2;3m\"\n \n /* A special value meaning \"no color selected\" */\n #define GIT_COLOR_NIL \"NIL\"\ndiff --git a/diff.c b/diff.c\nindex 7756f7610c..82ace48c38 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -59,12 +59,12 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n \tGIT_COLOR_BOLD_MAGENTA,\t/* OLD_MOVED */\n \tGIT_COLOR_BOLD_BLUE,\t/* OLD_MOVED ALTERNATIVE */\n-\tGIT_COLOR_DI,\t\t/* OLD_MOVED_DIM */\n-\tGIT_COLOR_DI_IT,\t/* OLD_MOVED_ALTERNATIVE_DIM */\n+\tGIT_COLOR_FAINT,\t/* OLD_MOVED_DIM */\n+\tGIT_COLOR_FAINT_ITALIC,\t/* OLD_MOVED_ALTERNATIVE_DIM */\n \tGIT_COLOR_BOLD_CYAN,\t/* NEW_MOVED */\n \tGIT_COLOR_BOLD_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n-\tGIT_COLOR_DI,\t\t/* NEW_MOVED_DIM */\n-\tGIT_COLOR_DI_IT,\t/* NEW_MOVED_ALTERNATIVE_DIM */\n+\tGIT_COLOR_FAINT,\t/* NEW_MOVED_DIM */\n+\tGIT_COLOR_FAINT_ITALIC,\t/* NEW_MOVED_ALTERNATIVE_DIM */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -607,32 +607,11 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n-\tDIFF_SYMBOL_SEPARATOR,\n-\tDIFF_SYMBOL_CONTEXT_MARKER,\n-\tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n-\tDIFF_SYMBOL_NO_LF_EOF,\n-\tDIFF_SYMBOL_CONTEXT,\n-\tDIFF_SYMBOL_CONTEXT_INCOMPLETE,\n-\tDIFF_SYMBOL_PLUS,\n-\tDIFF_SYMBOL_MINUS,\n-\tDIFF_SYMBOL_WORDS_PORCELAIN,\n-\tDIFF_SYMBOL_WORDS,\n-\tDIFF_SYMBOL_FILEPAIR,\n-\tDIFF_SYMBOL_HEADER,\n-\tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_BINARY_DIFF_HEADER,\n \tDIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n \tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n \tDIFF_SYMBOL_BINARY_DIFF_BODY,\n \tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n-\tDIFF_SYMBOL_REWRITE_DIFF,\n-\tDIFF_SYMBOL_SUBMODULE_ADD,\n-\tDIFF_SYMBOL_SUBMODULE_DEL,\n-\tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n-\tDIFF_SYMBOL_SUBMODULE_MODIFIED,\n-\tDIFF_SYMBOL_SUBMODULE_HEADER,\n-\tDIFF_SYMBOL_SUBMODULE_ERROR,\n-\tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n \tDIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n \tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n@@ -640,6 +619,28 @@ enum diff_symbol {\n \tDIFF_SYMBOL_WORD_DIFF,\n \tDIFF_SYMBOL_STAT_SEP,\n \tDIFF_SYMBOL_SUMMARY,\n+\tDIFF_SYMBOL_SUBMODULE_ADD,\n+\tDIFF_SYMBOL_SUBMODULE_DEL,\n+\tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\tDIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\tDIFF_SYMBOL_SUBMODULE_HEADER,\n+\tDIFF_SYMBOL_SUBMODULE_ERROR,\n+\tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n+\tDIFF_SYMBOL_REWRITE_DIFF,\n+\tDIFF_SYMBOL_BINARY_FILES,\n+\tDIFF_SYMBOL_HEADER,\n+\tDIFF_SYMBOL_FILEPAIR_PLUS,\n+\tDIFF_SYMBOL_FILEPAIR_MINUS,\n+\tDIFF_SYMBOL_WORDS_PORCELAIN,\n+\tDIFF_SYMBOL_WORDS,\n+\tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_CONTEXT_INCOMPLETE,\n+\tDIFF_SYMBOL_PLUS,\n+\tDIFF_SYMBOL_MINUS,\n+\tDIFF_SYMBOL_NO_LF_EOF,\n+\tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n+\tDIFF_SYMBOL_CONTEXT_MARKER,\n+\tDIFF_SYMBOL_SEPARATOR\n };\n /*\n * Flags for content lines:\n@@ -649,7 +650,7 @@ enum diff_symbol {\n */\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n #define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n-#define DIFF_SYMBOL_MOVED_LINE_ZEBRA\t\t(1<<18)\n+#define DIFF_SYMBOL_MOVED_LINE_ALT\t\t(1<<18)\n #define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING\t(1<<19)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n@@ -898,14 +899,14 @@ static void mark_color_as_moved(struct diff_options *o,\n \t\t}\n \n \t\tif (flipped_block)\n-\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_ZEBRA;\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;\n \t}\n \n \tfree(pmb);\n }\n \n #define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \\\n- (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ZEBRA)\n+ (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ALT)\n static void dim_moved_lines(struct diff_options *o)\n {\n \tint n;\n@@ -949,12 +950,12 @@ static void dim_moved_lines(struct diff_options *o)\n \n \t\t/* Check if we are at an interesting bound: */\n \t\tif (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&\n-\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA) !=\n-\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA))\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))\n \t\t\tcontinue;\n \t\tif (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&\n-\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA) !=\n-\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA))\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))\n \t\t\tcontinue;\n \n \t\t/*\n@@ -1004,25 +1005,33 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \tunsigned flags = eds->flags;\n \n \tswitch (s) {\n-\tcase DIFF_SYMBOL_SEPARATOR:\n-\t\tfprintf(o->file, \"%s%c\",\n-\t\t\tdiff_line_prefix(o),\n-\t\t\to->line_termination);\n-\t\tbreak;\n-\tcase DIFF_SYMBOL_CONTEXT_MARKER:\n+\tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n-\t\temit_line(o, context, reset, line, len);\n+\t\tputc('\\n', o->file);\n+\t\temit_line_0(o, context, reset, '\\\\',\n+\t\t\t nneof, strlen(nneof));\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n+\tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n+\tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n+\tcase DIFF_SYMBOL_SUMMARY:\n+\tcase DIFF_SYMBOL_STATS_LINE:\n+\tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n-\tcase DIFF_SYMBOL_NO_LF_EOF:\n+\tcase DIFF_SYMBOL_CONTEXT_INCOMPLETE:\n+\tcase DIFF_SYMBOL_CONTEXT_MARKER:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n-\t\tputc('\\n', o->file);\n-\t\temit_line_0(o, context, reset, '\\\\',\n-\t\t\t nneof, strlen(nneof));\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SEPARATOR:\n+\t\tfprintf(o->file, \"%s%c\",\n+\t\t\tdiff_line_prefix(o),\n+\t\t\to->line_termination);\n \t\tbreak;\n \tcase DIFF_SYMBOL_CONTEXT:\n \t\tset = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -1030,22 +1039,17 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\temit_line_ws_markup(o, set, reset, line, len, ' ',\n \t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n \t\tbreak;\n-\tcase DIFF_SYMBOL_CONTEXT_INCOMPLETE:\n-\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n-\t\treset = diff_get_color_opt(o, DIFF_RESET);\n-\t\temit_line(o, context, reset, line, len);\n-\t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n \t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n-\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n \t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n \t\tcase DIFF_SYMBOL_MOVED_LINE |\n-\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n \t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);\n \t\t\tbreak;\n \t\tcase DIFF_SYMBOL_MOVED_LINE |\n-\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA:\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n \t\t\tbreak;\n \t\tcase DIFF_SYMBOL_MOVED_LINE |\n@@ -1065,15 +1069,15 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n \t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n-\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n \t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n \t\tcase DIFF_SYMBOL_MOVED_LINE |\n-\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n \t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);\n \t\t\tbreak;\n \t\tcase DIFF_SYMBOL_MOVED_LINE |\n-\t\t DIFF_SYMBOL_MOVED_LINE_ZEBRA:\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n \t\t\tbreak;\n \t\tcase DIFF_SYMBOL_MOVED_LINE |\n@@ -1110,20 +1114,23 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\t}\n \t\temit_line(o, context, reset, line, len);\n \t\tbreak;\n-\tcase DIFF_SYMBOL_FILEPAIR:\n+\tcase DIFF_SYMBOL_FILEPAIR_PLUS:\n \t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n-\t\tfprintf(o->file, \"%s%s%s%s%s%s\\n\", diff_line_prefix(o), meta,\n-\t\t\tflags ? \"+++ \" : \"--- \",\n+\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\", diff_line_prefix(o), meta,\n \t\t\tline, reset,\n \t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n-\tcase DIFF_SYMBOL_HEADER:\n-\t\tfprintf(o->file, \"%s\", line);\n+\tcase DIFF_SYMBOL_FILEPAIR_MINUS:\n+\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\", diff_line_prefix(o), meta,\n+\t\t\tline, reset,\n+\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n \tcase DIFF_SYMBOL_BINARY_FILES:\n-\t\tfprintf(o->file, \"%sBinary files %s differ\\n\",\n-\t\t\tdiff_line_prefix(o), line);\n+\tcase DIFF_SYMBOL_HEADER:\n+\t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n \tcase DIFF_SYMBOL_BINARY_DIFF_HEADER:\n \t\tfprintf(o->file, \"%sGIT binary patch\\n\", diff_line_prefix(o));\n@@ -1134,9 +1141,6 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:\n \t\tfprintf(o->file, \"%sliteral %s\\n\", diff_line_prefix(o), line);\n \t\tbreak;\n-\tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n-\t\temit_line(o, \"\", \"\", line, len);\n-\t\tbreak;\n \tcase DIFF_SYMBOL_BINARY_DIFF_FOOTER:\n \t\tfputs(diff_line_prefix(o), o->file);\n \t\tfputc('\\n', o->file);\n@@ -1164,36 +1168,19 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\tfprintf(o->file, \"%sSubmodule %s contains modified content\\n\",\n \t\t\tdiff_line_prefix(o), line);\n \t\tbreak;\n-\tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n-\t\tfprintf(o->file, \"%s%s\", diff_line_prefix(o), line);\n-\t\tbreak;\n-\tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n-\t\temit_line(o, \"\", \"\", line, len);\n-\t\tbreak;\n-\tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n-\t\temit_line(o, \"\", \"\", line, len);\n-\t\tbreak;\n \tcase DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:\n-\t\tfprintf(o->file, \" 0 files changed\\n\");\n+\t\temit_line(o, \"\", \"\", \" 0 files changed\\n\",\n+\t\t\t strlen(\" 0 files changed\\n\"));\n \t\tbreak;\n \tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n \t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n \t\tbreak;\n-\tcase DIFF_SYMBOL_STATS_LINE:\n-\t\temit_line(o, \"\", \"\", line, len);\n-\t\tbreak;\n-\tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n-\t\temit_line(o, \"\", \"\", line, len);\n-\t\tbreak;\n \tcase DIFF_SYMBOL_WORD_DIFF:\n \t\tfprintf(o->file, \"%.*s\", len, line);\n \t\tbreak;\n \tcase DIFF_SYMBOL_STAT_SEP:\n \t\tfputs(o->stat_sep, o->file);\n \t\tbreak;\n-\tcase DIFF_SYMBOL_SUMMARY:\n-\t\temit_line(o, \"\", \"\", line, len);\n-\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1460,8 +1447,10 @@ static void emit_rewrite_diff(const char *name_a,\n \tlc_a = count_lines(data_one, size_one);\n \tlc_b = count_lines(data_two, size_two);\n \n-\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, a_name.buf, a_name.len, 0);\n-\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR, b_name.buf, b_name.len, 1);\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,\n+\t\t\t a_name.buf, a_name.len, 0);\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n+\t\t\t b_name.buf, b_name.len, 0);\n \n \tstrbuf_addstr(&out, \"@@ -\");\n \tif (!o->irreversible_delete)\n@@ -1983,12 +1972,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->label_path[0]) {\n-\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR,\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,\n \t\t\t\t ecbdata->label_path[0],\n \t\t\t\t strlen(ecbdata->label_path[0]), 0);\n-\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR,\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n \t\t\t\t ecbdata->label_path[1],\n-\t\t\t\t strlen(ecbdata->label_path[1]), 1);\n+\t\t\t\t strlen(ecbdata->label_path[1]), 0);\n \t\tecbdata->label_path[0] = ecbdata->label_path[1] = NULL;\n \t}\n \n@@ -3159,7 +3148,8 @@ static void builtin_diff(const char *name_a,\n \t\t\t}\n \t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n \t\t\t\t\t header.buf, header.len, 0);\n-\t\t\tstrbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n+\t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n+\t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n \t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n \t\t\t\t\t sb.buf, sb.len, 0);\n \t\t\tstrbuf_release(&sb);\n@@ -3180,7 +3170,8 @@ static void builtin_diff(const char *name_a,\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o, &mf1, &mf2);\n \t\telse {\n-\t\t\tstrbuf_addf(&sb, \"%s and %s\", lbl[0], lbl[1]);\n+\t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n+\t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n \t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n \t\t\t\t\t sb.buf, sb.len, 0);\n \t\t\tstrbuf_release(&sb);\n@@ -5475,6 +5466,9 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \tstatic struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n \n+\tif (WSEH_NEW & WS_RULE_MASK)\n+\t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n+\n \tif (o->color_moved)\n \t\to->emitted_symbols = &esm;\n \n\n\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323024","messageId":"20170623012919.28693-3-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 02/25] diff.c: move line ending check into emit_hunk_header","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:28:56Z","receivedAt":"2017-06-23T01:29:38Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The emit_hunk_header() function is responsible for assembling a\nhunk header and calling emit_line() to send the hunk header\nto the output file. Its only caller fn_out_consume() needs\nto prepare for a case where the function emits an incomplete\nline and add the terminating LF.\n\nInstead make sure emit_hunk_header() to always send a\ncompleted line to emit_line().\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 3f5bf8b5a4..c2ed605cd0 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -677,6 +677,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t}\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n+\tstrbuf_complete_line(&msgbuf);\n+\n \temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n@@ -1315,8 +1317,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\tlen = sane_truncate_line(ecbdata, line, len);\n \t\tfind_lno(line, ecbdata);\n \t\temit_hunk_header(ecbdata, line, len);\n-\t\tif (line[len-1] != '\\n')\n-\t\t\tputc('\\n', o->file);\n \t\treturn;\n \t}\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323025","messageId":"20170623012919.28693-5-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 04/25] diff.c: introduce emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:28:58Z","receivedAt":"2017-06-23T01:29:40Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we want to buffer all output before emitting it as a\nnew feature (\"markup moved lines\") conceptually cannot be implemented\nin a single pass over the output.\n\nThere are different approaches to buffer all output such as:\n* Buffering on the char level, i.e. we'd have a char[] which would\n grow at approximately 80 characters a line. This would keep the\n output completely unstructured, but might be very easy to implement,\n such as redirecting all output to a temporary file and working off\n that. The later passes over the buffer are quite complicated though,\n because we have to parse back any output and then decide if it should\n be modified.\n\n* Buffer on a line level. As the output is mostly line oriented already,\n this would make sense, but it still is a bit awkward as we'd have to\n make sense of it again by looking at the first characters of a line\n to decide what part of a diff a line is.\n\n* Buffer semantically. Imagine there is a formal grammar for the diff\n output and we'd keep the symbols of this grammar around. This keeps\n the highest level of structure in the buffered data, such that the\n actual memory requirements are less than say the first option. Instead\n of buffering the characters of the line, we'll buffer what we intend\n to do plus additional information for the specifics. An output of\n\n diff --git a/new.txt b/new.txt\n index fa69b07..412428c 100644\n Binary files a/new.txt and b/new.txt differ\n\n could be buffered as\n DIFF_SYMBOL_DIFF_START + new.txt\n DIFF_SYMBOL_INDEX_MODE + fa69b07 412428c \"non-executable\" flag\n DIFF_SYMBOL_BINARY_FILES + new.txt\n\nThis and the following patches introduce the third option of buffering\nby first moving any output to emit_diff_symbol, and then introducing the\nbuffering in this function.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 22 +++++++++++++++++++---\n 1 file changed, 19 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2f9722b382..2257d44e2c 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -559,6 +559,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n \temit_line_0(o, set, reset, line[0], line+1, len-1);\n }\n \n+enum diff_symbol {\n+\tDIFF_SYMBOL_SEPARATOR\n+};\n+\n+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n+\t\t\t const char *line, int len)\n+{\n+\tswitch (s) {\n+\tcase DIFF_SYMBOL_SEPARATOR:\n+\t\tfprintf(o->file, \"%s%c\",\n+\t\t\tdiff_line_prefix(o),\n+\t\t\to->line_termination);\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: unknown diff symbol\");\n+\t}\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -4833,9 +4851,7 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\tfprintf(options->file, \"%s%c\",\n-\t\t\t\tdiff_line_prefix(options),\n-\t\t\t\toptions->line_termination);\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323026","messageId":"20170623012919.28693-7-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 06/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:00Z","receivedAt":"2017-06-23T01:29:41Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex c550f75195..16818fa571 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n@@ -569,6 +570,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n {\n \tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_CONTEXT_MARKER:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -704,8 +708,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n-\n-\temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n+\temit_diff_symbol(ecbdata->opt,\n+\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323027","messageId":"20170623012919.28693-4-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 03/25] diff.c: factor out diff_flush_patch_all_file_pairs","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:28:57Z","receivedAt":"2017-06-23T01:29:45Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we want to do more things before and after all filepairs\nare flushed. So factor flushing out all file pairs into its own function\nthat the new code can be plugged in easily.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 17 ++++++++++++-----\n 1 file changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex c2ed605cd0..2f9722b382 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4737,6 +4737,17 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n \t\twarning(_(rename_limit_advice), varname, needed);\n }\n \n+static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n+{\n+\tint i;\n+\tstruct diff_queue_struct *q = &diff_queued_diff;\n+\tfor (i = 0; i < q->nr; i++) {\n+\t\tstruct diff_filepair *p = q->queue[i];\n+\t\tif (check_pair_status(p))\n+\t\t\tdiff_flush_patch(p, o);\n+\t}\n+}\n+\n void diff_flush(struct diff_options *options)\n {\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n@@ -4831,11 +4842,7 @@ void diff_flush(struct diff_options *options)\n \t\t\t}\n \t\t}\n \n-\t\tfor (i = 0; i < q->nr; i++) {\n-\t\t\tstruct diff_filepair *p = q->queue[i];\n-\t\t\tif (check_pair_status(p))\n-\t\t\t\tdiff_flush_patch(p, options);\n-\t\t}\n+\t\tdiff_flush_patch_all_file_pairs(options);\n \t}\n \n \tif (output_format & DIFF_FORMAT_CALLBACK)\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323028","messageId":"20170623012919.28693-6-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 05/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:28:59Z","receivedAt":"2017-06-23T01:29:46Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 10 +++++++++-\n 1 file changed, 9 insertions(+), 1 deletion(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2257d44e2c..c550f75195 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,13 +560,20 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len)\n {\n+\tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_CONTEXT_MARKER:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n \t\t\tdiff_line_prefix(o),\n@@ -661,7 +668,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tif (len < 10 ||\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n-\t\temit_line(ecbdata->opt, context, reset, line, len);\n+\t\temit_diff_symbol(ecbdata->opt,\n+\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323029","messageId":"20170623012919.28693-10-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 09/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:03Z","receivedAt":"2017-06-23T01:29:48Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 42 ++++++++++++++++++++++++++----------------\n 1 file changed, 26 insertions(+), 16 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex e7583efca3..3d081edd12 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,8 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_WORDS_PORCELAIN,\n+\tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n \tDIFF_SYMBOL_PLUS,\n \tDIFF_SYMBOL_MINUS,\n@@ -648,6 +650,26 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_WORDS_PORCELAIN:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tfputs(\"~\\n\", o->file);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_WORDS:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\t/*\n+\t\t * Skip the prefix character, if any. With\n+\t\t * diff_suppress_blank_empty, there may be\n+\t\t * none.\n+\t\t */\n+\t\tif (line[0] != '\\n') {\n+\t\t\tline++;\n+\t\t\tlen--;\n+\t\t}\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1342,7 +1364,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n {\n \tstruct emit_callback *ecbdata = priv;\n \tconst char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);\n-\tconst char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n \tconst char *line_prefix = diff_line_prefix(o);\n@@ -1384,6 +1405,9 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->diff_words) {\n+\t\tenum diff_symbol s =\n+\t\t\tecbdata->diff_words->type == DIFF_WORDS_PORCELAIN ?\n+\t\t\tDIFF_SYMBOL_WORDS_PORCELAIN : DIFF_SYMBOL_WORDS;\n \t\tif (line[0] == '-') {\n \t\t\tdiff_words_append(line, len,\n \t\t\t\t\t &ecbdata->diff_words->minus);\n@@ -1403,21 +1427,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t\treturn;\n \t\t}\n \t\tdiff_words_flush(ecbdata);\n-\t\tif (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {\n-\t\t\temit_line(o, context, reset, line, len);\n-\t\t\tfputs(\"~\\n\", o->file);\n-\t\t} else {\n-\t\t\t/*\n-\t\t\t * Skip the prefix character, if any. With\n-\t\t\t * diff_suppress_blank_empty, there may be\n-\t\t\t * none.\n-\t\t\t */\n-\t\t\tif (line[0] != '\\n') {\n-\t\t\t line++;\n-\t\t\t len--;\n-\t\t\t}\n-\t\t\temit_line(o, context, reset, line, len);\n-\t\t}\n+\t\temit_diff_symbol(o, s, line, len, 0);\n \t\treturn;\n \t}\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323030","messageId":"20170623012919.28693-8-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 07/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:01Z","receivedAt":"2017-06-23T01:29:49Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 19 +++++++++++--------\n 1 file changed, 11 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 16818fa571..b78f698cad 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_NO_LF_EOF,\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n@@ -568,8 +569,16 @@ enum diff_symbol {\n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len)\n {\n+\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_NO_LF_EOF:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tputc('\\n', o->file);\n+\t\temit_line_0(o, context, reset, '\\\\',\n+\t\t\t nneof, strlen(nneof));\n+\t\tbreak;\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -750,7 +759,6 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\t\t int prefix, const char *data, int size)\n {\n \tconst char *endp = NULL;\n-\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);\n \n \twhile (0 < size) {\n@@ -768,13 +776,8 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tsize -= len;\n \t\tdata += len;\n \t}\n-\tif (!endp) {\n-\t\tconst char *context = diff_get_color(ecb->color_diff,\n-\t\t\t\t\t\t DIFF_CONTEXT);\n-\t\tputc('\\n', ecb->opt->file);\n-\t\temit_line_0(ecb->opt, context, reset, '\\\\',\n-\t\t\t nneof, strlen(nneof));\n-\t}\n+\tif (!endp)\n+\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);\n }\n \n static void emit_rewrite_diff(const char *name_a,\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323031","messageId":"20170623012919.28693-14-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 13/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:07Z","receivedAt":"2017-06-23T01:29:53Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"we could save a little bit of memory when buffering in a later mode\nby just passing the inner part (\"%s and %s\", file1, file 2), but\nthose a just a few bytes, so instead let's reuse the implementation from\nDIFF_SYMBOL_HEADER and keep the whole line around.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 20 +++++++++++++++-----\n 1 file changed, 15 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 34314455b5..78cf5ad691 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n \tDIFF_SYMBOL_FILEPAIR_MINUS,\n@@ -689,6 +690,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tline, reset,\n \t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_FILES:\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n@@ -2542,6 +2544,7 @@ static void builtin_diff(const char *name_a,\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n \t ( (!textconv_one && diff_filespec_is_binary(one)) ||\n \t (!textconv_two && diff_filespec_is_binary(two)) )) {\n+\t\tstruct strbuf sb = STRBUF_INIT;\n \t\tif (!one->data && !two->data &&\n \t\t S_ISREG(one->mode) && S_ISREG(two->mode) &&\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n@@ -2554,8 +2557,11 @@ static void builtin_diff(const char *name_a,\n \t\t\t}\n \t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n \t\t\t\t\t header.buf, header.len, 0);\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n+\t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_release(&sb);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n \t\tif (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)\n@@ -2572,9 +2578,13 @@ static void builtin_diff(const char *name_a,\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n-\t\telse\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\telse {\n+\t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n+\t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_release(&sb);\n+\t\t}\n \t\to->found_changes = 1;\n \t} else {\n \t\t/* Crazy xdl interfaces.. */\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323032","messageId":"20170623012919.28693-13-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 12/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:06Z","receivedAt":"2017-06-23T01:30:04Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The header is constructed lazily including line breaks, so just emit\nthe raw string as is.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 28 ++++++++++++++++++++--------\n 1 file changed, 20 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 761ee581ad..34314455b5 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n \tDIFF_SYMBOL_FILEPAIR_MINUS,\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n@@ -688,6 +689,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tline, reset,\n \t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n+\tcase DIFF_SYMBOL_HEADER:\n+\t\tfprintf(o->file, \"%s\", line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1385,7 +1389,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \to->found_changes = 1;\n \n \tif (ecbdata->header) {\n-\t\tfprintf(o->file, \"%s\", ecbdata->header->buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t ecbdata->header->buf, ecbdata->header->len, 0);\n \t\tstrbuf_reset(ecbdata->header);\n \t\tecbdata->header = NULL;\n \t}\n@@ -2519,7 +2524,8 @@ static void builtin_diff(const char *name_a,\n \t\tif (complete_rewrite &&\n \t\t (textconv_one || !diff_filespec_is_binary(one)) &&\n \t\t (textconv_two || !diff_filespec_is_binary(two))) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tstrbuf_reset(&header);\n \t\t\temit_rewrite_diff(name_a, name_b, one, two,\n \t\t\t\t\t\ttextconv_one, textconv_two, o);\n@@ -2529,7 +2535,8 @@ static void builtin_diff(const char *name_a,\n \t}\n \n \tif (o->irreversible_delete && lbl[1][0] == '/') {\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf,\n+\t\t\t\t header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tgoto free_ab_and_return;\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n@@ -2540,10 +2547,13 @@ static void builtin_diff(const char *name_a,\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n \t\t\tif (!oidcmp(&one->oid, &two->oid)) {\n \t\t\t\tif (must_show_header)\n-\t\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t\t\t header.buf, header.len,\n+\t\t\t\t\t\t\t 0);\n \t\t\t\tgoto free_ab_and_return;\n \t\t\t}\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n \t\t\t\tline_prefix, lbl[0], lbl[1]);\n \t\t\tgoto free_ab_and_return;\n@@ -2554,10 +2564,11 @@ static void builtin_diff(const char *name_a,\n \t\tif (mf1.size == mf2.size &&\n \t\t !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {\n \t\t\tif (must_show_header)\n-\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n@@ -2575,7 +2586,8 @@ static void builtin_diff(const char *name_a,\n \t\tconst struct userdiff_funcname *pe;\n \n \t\tif (must_show_header) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tstrbuf_reset(&header);\n \t\t}\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323033","messageId":"20170623012919.28693-15-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 14/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:08Z","receivedAt":"2017-06-23T01:30:09Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 35 +++++++++++++++++++++--------------\n 1 file changed, 21 insertions(+), 14 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 78cf5ad691..0314d57647 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_REWRITE_DIFF,\n \tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n@@ -614,7 +615,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset, *set, *meta;\n+\tconst char *context, *reset, *set, *meta, *fraginfo;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -694,6 +695,11 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_REWRITE_DIFF:\n+\t\tfraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, fraginfo, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -816,17 +822,17 @@ static void remove_tempfile(void)\n \t}\n }\n \n-static void print_line_count(FILE *file, int count)\n+static void add_line_count(struct strbuf *out, int count)\n {\n \tswitch (count) {\n \tcase 0:\n-\t\tfprintf(file, \"0,0\");\n+\t\tstrbuf_addstr(out, \"0,0\");\n \t\tbreak;\n \tcase 1:\n-\t\tfprintf(file, \"1\");\n+\t\tstrbuf_addstr(out, \"1\");\n \t\tbreak;\n \tdefault:\n-\t\tfprintf(file, \"1,%d\", count);\n+\t\tstrbuf_addf(out, \"1,%d\", count);\n \t\tbreak;\n \t}\n }\n@@ -865,14 +871,12 @@ static void emit_rewrite_diff(const char *name_a,\n \t\t\t struct diff_options *o)\n {\n \tint lc_a, lc_b;\n-\tconst char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n-\tconst char *reset = diff_get_color(o->use_color, DIFF_RESET);\n \tstatic struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;\n \tconst char *a_prefix, *b_prefix;\n \tchar *data_one, *data_two;\n \tsize_t size_one, size_two;\n \tstruct emit_callback ecbdata;\n-\tconst char *line_prefix = diff_line_prefix(o);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\ta_prefix = o->b_prefix;\n@@ -916,14 +920,17 @@ static void emit_rewrite_diff(const char *name_a,\n \temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n \t\t\t b_name.buf, b_name.len, 0);\n \n-\tfprintf(o->file, \"%s%s@@ -\", line_prefix, fraginfo);\n+\tstrbuf_addstr(&out, \"@@ -\");\n \tif (!o->irreversible_delete)\n-\t\tprint_line_count(o->file, lc_a);\n+\t\tadd_line_count(&out, lc_a);\n \telse\n-\t\tfprintf(o->file, \"?,?\");\n-\tfprintf(o->file, \" +\");\n-\tprint_line_count(o->file, lc_b);\n-\tfprintf(o->file, \" @@%s\\n\", reset);\n+\t\tstrbuf_addstr(&out, \"?,?\");\n+\tstrbuf_addstr(&out, \" +\");\n+\tadd_line_count(&out, lc_b);\n+\tstrbuf_addstr(&out, \" @@\\n\");\n+\temit_diff_symbol(o, DIFF_SYMBOL_REWRITE_DIFF, out.buf, out.len, 0);\n+\tstrbuf_release(&out);\n+\n \tif (lc_a && !o->irreversible_delete)\n \t\temit_rewrite_lines(&ecbdata, '-', data_one, size_one);\n \tif (lc_b)\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323034","messageId":"20170623012919.28693-24-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 23/25] diff.c: color moved lines differently, plain mode","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:17Z","receivedAt":"2017-06-23T01:30:42Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add the 'plain' mode for move detection of code.\nThis omits the checking for adjacent blocks, so it is not as useful.\nIf you have a lot of the same blocks moved in the same patch, the 'Zebra'\nwould end up slow as it is O(n^2) (n is number of same blocks).\nSo this may be useful there and is generally easy to add\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 5 +++++\n diff.h | 1 +\n t/t4015-diff-whitespace.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++-\n 3 files changed, 56 insertions(+), 1 deletion(-)\n\ndiff --git a/diff.c b/diff.c\nindex e6f547804f..0eb744514f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -246,6 +246,8 @@ static int parse_color_moved(const char *arg)\n {\n \tif (!strcmp(arg, \"no\"))\n \t\treturn COLOR_MOVED_NO;\n+\telse if (!strcmp(arg, \"plain\"))\n+\t\treturn COLOR_MOVED_PLAIN;\n \telse if (!strcmp(arg, \"zebra\"))\n \t\treturn COLOR_MOVED_ZEBRA;\n \telse\n@@ -831,6 +833,9 @@ static void mark_color_as_moved(struct diff_options *o,\n \n \t\tl->flags |= DIFF_SYMBOL_MOVED_LINE;\n \n+\t\tif (o->color_moved == COLOR_MOVED_PLAIN)\n+\t\t\tcontinue;\n+\n \t\t/* Check any potential block runs, advance each or nullify */\n \t\tfor (i = 0; i < pmb_nr; i++) {\n \t\t\tstruct moved_entry *p = pmb[i];\ndiff --git a/diff.h b/diff.h\nindex 7726ad255c..1aae8738ca 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -190,6 +190,7 @@ struct diff_options {\n \tstruct emitted_diff_symbols *emitted_symbols;\n \tenum {\n \t\tCOLOR_MOVED_NO = 0,\n+\t\tCOLOR_MOVED_PLAIN = 1,\n \t\tCOLOR_MOVED_ZEBRA = 2,\n \t} color_moved;\n };\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 4a03766f1f..1ca16435d6 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -986,7 +986,7 @@ test_expect_success 'detect moved code, complete file' '\n \tgit mv test.c main.c &&\n \ttest_config color.diff.oldMoved \"normal red\" &&\n \ttest_config color.diff.newMoved \"normal green\" &&\n-\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tgit diff HEAD --color-moved=zebra --no-renames | test_decode_color >actual &&\n \tcat >expected <<-\\EOF &&\n \tdiff --git a/main.c b/main.c\n \tnew file mode 100644\n@@ -1130,6 +1130,55 @@ test_expect_success 'detect malicious moved code, inside file' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'plain moved code, inside file' '\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\t# needs previous test as setup\n+\tgit diff HEAD --no-renames --color-moved=plain| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'no effect from --color-moved with --word-diff' '\n \tcat <<-\\EOF >text.txt &&\n \tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323035","messageId":"20170623012919.28693-23-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 22/25] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:16Z","receivedAt":"2017-06-23T01:30:44Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a patch consists mostly of moving blocks of code around, it can\nbe quite tedious to ensure that the blocks are moved verbatim, and not\nundesirably modified in the move. To that end, color blocks that are\nmoved within the same patch differently. For example (OM, del, add,\nand NM are different colors):\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OM] - if (!is_authorized_user())\n [OM] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OM] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NM] + sensitive_stuff(spanning,\n [NM] + multiple,\n [NM] + lines);\n [NM] +}\n\nHowever adjacent blocks may be problematic. For example, in this\npotentially malicious patch, the swapping of blocks can be spotted:\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OMA] - if (!is_authorized_user())\n [OMA] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OMA] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NMA] + sensitive_stuff(spanning,\n [NMA] + multiple,\n [NMA] + lines);\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NMA] +}\n\nIf the moved code is larger, it is easier to hide some permutation in the\ncode, which is why some alternative coloring is needed.\n\nThis patch implements the first mode:\n* basic alternating 'Zebra' mode\n This conveys all information needed to the user. Defer customization to\n later patches.\n\nFirst I implemented an alternative design, which would try to fingerprint\na line by its neighbors to detect if we are in a block or at the boundary.\nThis idea iss error prone as it inspected each line and its neighboring\nlines to determine if the line was (a) moved and (b) if was deep inside\na hunk by having matching neighboring lines. This is unreliable as the\nwe can construct hunks which have equal neighbors that just exceed the\nnumber of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter\nas a line, that is permutated to AXYZCXYZBXYZD..').\n\nInstead this provides a dynamic programming greedy algorithm that finds\nthe largest moved hunk and then has several modes on highlighting bounds.\n\nA note on the options '--submodule=diff' and '--color-words/--word-diff':\nIn the conversion to use emit_line in the prior patches both submodules\nas well as word diff output carefully chose to call emit_line with sign=0.\nAll output with sign=0 is ignored for move detection purposes in this\npatch, such that no weird looking output will be generated for these\ncases. This leads to another thought: We could pass on '--color-moved' to\nsubmodules such that they color up moved lines for themselves. If we'd do\nso only line moves within a repository boundary are marked up.\n\nHelped-by: Jonathan Tan \nSigned-off-by: Stefan Beller \n---\n diff.c | 303 ++++++++++++++++++++++++++++++++++++++++++---\n diff.h | 10 +-\n t/t4015-diff-whitespace.sh | 196 +++++++++++++++++++++++++++++\n 3 files changed, 494 insertions(+), 15 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex b344808c20..e6f547804f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -15,6 +15,7 @@\n #include \"userdiff.h\"\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n+#include \"hashmap.h\"\n #include \"ll-merge.h\"\n #include \"string-list.h\"\n #include \"argv-array.h\"\n@@ -31,6 +32,7 @@ static int diff_indent_heuristic; /* experimental */\n static int diff_rename_limit_default = 400;\n static int diff_suppress_blank_empty;\n static int diff_use_color_default = -1;\n+static int diff_color_moved_default;\n static int diff_context_default = 3;\n static int diff_interhunk_context_default;\n static const char *diff_word_regex_cfg;\n@@ -55,6 +57,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n+\tGIT_COLOR_MAGENTA,\t/* OLD_MOVED */\n+\tGIT_COLOR_BLUE,\t\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_CYAN,\t\t/* NEW_MOVED */\n+\tGIT_COLOR_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -80,6 +86,14 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_WHITESPACE;\n \tif (!strcasecmp(var, \"func\"))\n \t\treturn DIFF_FUNCINFO;\n+\tif (!strcasecmp(var, \"oldmoved\"))\n+\t\treturn DIFF_FILE_OLD_MOVED;\n+\tif (!strcasecmp(var, \"oldmovedalternative\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoved\"))\n+\t\treturn DIFF_FILE_NEW_MOVED;\n+\tif (!strcasecmp(var, \"newmovedalternative\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT;\n \treturn -1;\n }\n \n@@ -228,12 +242,29 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)\n \treturn 0;\n }\n \n+static int parse_color_moved(const char *arg)\n+{\n+\tif (!strcmp(arg, \"no\"))\n+\t\treturn COLOR_MOVED_NO;\n+\telse if (!strcmp(arg, \"zebra\"))\n+\t\treturn COLOR_MOVED_ZEBRA;\n+\telse\n+\t\treturn -1;\n+}\n+\n int git_diff_ui_config(const char *var, const char *value, void *cb)\n {\n \tif (!strcmp(var, \"diff.color\") || !strcmp(var, \"color.diff\")) {\n \t\tdiff_use_color_default = git_config_colorbool(var, value);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"diff.colormoved\")) {\n+\t\tint cm = parse_color_moved(value);\n+\t\tif (cm < 0)\n+\t\t\treturn -1;\n+\t\tdiff_color_moved_default = cm;\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"diff.context\")) {\n \t\tdiff_context_default = git_config_int(var, value);\n \t\tif (diff_context_default < 0)\n@@ -601,7 +632,9 @@ enum diff_symbol {\n * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT\n * 16 is marking if the line is blank at EOF\n */\n-#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n+#define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n+#define DIFF_SYMBOL_MOVED_LINE_ALT\t\t(1<<18)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n /*\n@@ -644,6 +677,213 @@ static void append_emitted_diff_symbol(struct diff_options *o,\n \tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n }\n \n+struct moved_entry {\n+\tstruct hashmap_entry ent;\n+\tconst struct emitted_diff_symbol *es;\n+\tstruct moved_entry *next_line;\n+};\n+\n+static void get_ws_cleaned_string(const struct emitted_diff_symbol *l,\n+\t\t\t\t struct strbuf *out)\n+{\n+\tint i;\n+\tfor (i = 0; i < l->len; i++) {\n+\t\tif (isspace(l->line[i]))\n+\t\t\tcontinue;\n+\t\tstrbuf_addch(out, l->line[i]);\n+\t}\n+}\n+\n+static int emitted_symbol_cmp_no_ws(const struct emitted_diff_symbol *a,\n+\t\t\t\t const struct emitted_diff_symbol *b,\n+\t\t\t\t const void *keydata)\n+{\n+\tint ret;\n+\tstruct strbuf sba = STRBUF_INIT;\n+\tstruct strbuf sbb = STRBUF_INIT;\n+\n+\tget_ws_cleaned_string(a, &sba);\n+\tget_ws_cleaned_string(b, &sbb);\n+\tret = sba.len != sbb.len || strncmp(sba.buf, sbb.buf, sba.len);\n+\n+\tstrbuf_release(&sba);\n+\tstrbuf_release(&sbb);\n+\treturn ret;\n+}\n+\n+static int emitted_symbol_cmp(const struct emitted_diff_symbol *a,\n+\t\t\t const struct emitted_diff_symbol *b,\n+\t\t\t const void *keydata)\n+{\n+\treturn a->len != b->len || strncmp(a->line, b->line, a->len);\n+}\n+\n+static int moved_entry_cmp(const struct moved_entry *a,\n+\t\t\t const struct moved_entry *b,\n+\t\t\t const void *keydata)\n+{\n+\treturn emitted_symbol_cmp(a->es, b->es, keydata);\n+}\n+\n+static int moved_entry_cmp_no_ws(const struct moved_entry *a,\n+\t\t\t\t const struct moved_entry *b,\n+\t\t\t\t const void *keydata)\n+{\n+\treturn emitted_symbol_cmp_no_ws(a->es, b->es, keydata);\n+}\n+\n+static unsigned get_string_hash(struct emitted_diff_symbol *es, unsigned ignore_ws)\n+{\n+\tstatic struct strbuf sb = STRBUF_INIT;\n+\n+\tif (ignore_ws) {\n+\t\tstrbuf_reset(&sb);\n+\t\tget_ws_cleaned_string(es, &sb);\n+\t\treturn memhash(sb.buf, sb.len);\n+\t} else {\n+\t\treturn memhash(es->line, es->len);\n+\t}\n+}\n+\n+static struct moved_entry *prepare_entry(struct diff_options *o,\n+\t\t\t\t\t int line_no)\n+{\n+\tstruct moved_entry *ret = xmalloc(sizeof(*ret));\n+\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];\n+\n+\tret->ent.hash = get_string_hash(l, ignore_ws);\n+\tret->es = l;\n+\tret->next_line = NULL;\n+\n+\treturn ret;\n+}\n+\n+static void add_lines_to_move_detection(struct diff_options *o,\n+\t\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry *prev_line = NULL;\n+\n+\tint n;\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct hashmap *hm;\n+\t\tstruct moved_entry *key;\n+\n+\t\tswitch (o->emitted_symbols->buf[n].s) {\n+\t\tcase DIFF_SYMBOL_PLUS:\n+\t\t\thm = add_lines;\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MINUS:\n+\t\t\thm = del_lines;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tkey = prepare_entry(o, n);\n+\t\tif (prev_line && prev_line->es->s == o->emitted_symbols->buf[n].s)\n+\t\t\tprev_line->next_line = key;\n+\n+\t\thashmap_add(hm, key);\n+\t\tprev_line = key;\n+\t}\n+}\n+\n+/* Find blocks of moved code, delegate actual coloring decision to helper */\n+static void mark_color_as_moved(struct diff_options *o,\n+\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry **pmb = NULL; /* potentially moved blocks */\n+\tint pmb_nr = 0, pmb_alloc = 0;\n+\tint n, flipped_block = 1;\n+\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct hashmap *hm = NULL;\n+\t\tstruct moved_entry *key;\n+\t\tstruct moved_entry *match = NULL;\n+\t\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];\n+\t\tint i, lp, rp;\n+\n+\t\tswitch (l->s) {\n+\t\tcase DIFF_SYMBOL_PLUS:\n+\t\t\thm = del_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MINUS:\n+\t\t\thm = add_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tflipped_block = 1;\n+\t\t}\n+\n+\t\tif (!match) {\n+\t\t\tpmb_nr = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE;\n+\n+\t\t/* Check any potential block runs, advance each or nullify */\n+\t\tfor (i = 0; i < pmb_nr; i++) {\n+\t\t\tstruct moved_entry *p = pmb[i];\n+\t\t\tstruct moved_entry *pnext = (p && p->next_line) ?\n+\t\t\t\t\tp->next_line : NULL;\n+\t\t\tif (pnext &&\n+\t\t\t !emitted_symbol_cmp(pnext->es, l, o)) {\n+\t\t\t\tpmb[i] = p->next_line;\n+\t\t\t} else {\n+\t\t\t\tpmb[i] = NULL;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Shrink the set of potential block to the remaining running */\n+\t\tfor (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n+\t\t\twhile (lp < pmb_nr && pmb[lp])\n+\t\t\t\tlp++;\n+\t\t\t/* lp points at the first NULL now */\n+\n+\t\t\twhile (rp > -1 && !pmb[rp])\n+\t\t\t\trp--;\n+\t\t\t/* rp points at the last non-NULL */\n+\n+\t\t\tif (lp < pmb_nr && rp > -1 && lp < rp) {\n+\t\t\t\tpmb[lp] = pmb[rp];\n+\t\t\t\tpmb[rp] = NULL;\n+\t\t\t\trp--;\n+\t\t\t\tlp++;\n+\t\t\t}\n+\t\t}\n+\n+\t\t/* Remember the number of running sets */\n+\t\tpmb_nr = rp + 1;\n+\n+\t\tif (pmb_nr == 0) {\n+\t\t\t/*\n+\t\t\t * The current line is the start of a new block.\n+\t\t\t * Setup the set of potential blocks.\n+\t\t\t */\n+\t\t\tfor (; match; match = hashmap_get_next(hm, match)) {\n+\t\t\t\tALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n+\t\t\t\tpmb[pmb_nr++] = match;\n+\t\t\t}\n+\n+\t\t\tflipped_block = (flipped_block + 1) % 2;\n+\t\t}\n+\n+\t\tif (flipped_block)\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;\n+\t}\n+\n+\tfree(pmb);\n+}\n \n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n@@ -719,14 +959,24 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n \t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n-\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n+\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);\n+\t\telse\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '+',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n-\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n+\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);\n+\t\telse\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n@@ -3741,6 +3991,8 @@ void diff_setup(struct diff_options *options)\n \t\toptions->a_prefix = \"a/\";\n \t\toptions->b_prefix = \"b/\";\n \t}\n+\n+\toptions->color_moved = diff_color_moved_default;\n }\n \n void diff_setup_done(struct diff_options *options)\n@@ -3850,6 +4102,9 @@ void diff_setup_done(struct diff_options *options)\n \n \tif (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)\n \t\tdie(_(\"--follow requires exactly one pathspec\"));\n+\n+\tif (!options->use_color || external_diff())\n+\t\toptions->color_moved = 0;\n }\n \n static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)\n@@ -4274,7 +4529,16 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n-\telse if (!strcmp(arg, \"--color-words\")) {\n+\telse if (!strcmp(arg, \"--color-moved\"))\n+\t\toptions->color_moved = COLOR_MOVED_ZEBRA;\n+\telse if (!strcmp(arg, \"--no-color-moved\"))\n+\t\toptions->color_moved = COLOR_MOVED_NO;\n+\telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n+\t\tint cm = parse_color_moved(arg);\n+\t\tif (cm < 0)\n+\t\t\tdie(\"bad --color-moved argument: %s\", arg);\n+\t\toptions->color_moved = cm;\n+\t} else if (!strcmp(arg, \"--color-words\")) {\n \t\toptions->use_color = 1;\n \t\toptions->word_diff = DIFF_WORDS_COLOR;\n \t}\n@@ -5089,16 +5353,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \tif (WSEH_NEW & WS_RULE_MASK)\n \t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n \n-\t/*\n-\t * For testing purposes we want to make sure the diff machinery\n-\t * works completely with the buffer. If there is anything emitted\n-\t * outside the emit_string, then the order is screwed\n-\t * up and the tests will fail.\n-\t *\n-\t * TODO (later in this series):\n-\t * We'll unset this pointer in a later patch.\n-\t */\n-\to->emitted_symbols = &esm;\n+\tif (o->color_moved)\n+\t\to->emitted_symbols = &esm;\n \n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n@@ -5107,6 +5363,24 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \t}\n \n \tif (o->emitted_symbols) {\n+\t\tif (o->color_moved) {\n+\t\t\tstruct hashmap add_lines, del_lines;\n+\t\t\tunsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n+\n+\t\t\thashmap_init(&del_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\t\t\thashmap_init(&add_lines, ignore_ws ?\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp_no_ws :\n+\t\t\t\t(hashmap_cmp_fn)moved_entry_cmp, 0);\n+\n+\t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n+\t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\n+\t\t\thashmap_free(&add_lines, 0);\n+\t\t\thashmap_free(&del_lines, 0);\n+\t\t}\n+\n \t\tfor (i = 0; i < esm.nr; i++)\n \t\t\temit_diff_symbol_from_struct(o, &esm.buf[i]);\n \n@@ -5190,6 +5464,7 @@ void diff_flush(struct diff_options *options)\n \t\tif (!options->file)\n \t\t\tdie_errno(\"Could not open /dev/null\");\n \t\toptions->close_file = 1;\n+\t\toptions->color_moved = 0;\n \t\tfor (i = 0; i < q->nr; i++) {\n \t\t\tstruct diff_filepair *p = q->queue[i];\n \t\t\tif (check_pair_status(p))\ndiff --git a/diff.h b/diff.h\nindex 65fc9dbb4b..7726ad255c 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,10 @@ struct diff_options {\n \tint diff_path_counter;\n \n \tstruct emitted_diff_symbols *emitted_symbols;\n+\tenum {\n+\t\tCOLOR_MOVED_NO = 0,\n+\t\tCOLOR_MOVED_ZEBRA = 2,\n+\t} color_moved;\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n@@ -208,7 +212,11 @@ enum color_diff {\n \tDIFF_FILE_NEW = 5,\n \tDIFF_COMMIT = 6,\n \tDIFF_WHITESPACE = 7,\n-\tDIFF_FUNCINFO = 8\n+\tDIFF_FUNCINFO = 8,\n+\tDIFF_FILE_OLD_MOVED = 9,\n+\tDIFF_FILE_OLD_MOVED_ALT = 10,\n+\tDIFF_FILE_NEW_MOVED = 11,\n+\tDIFF_FILE_NEW_MOVED_ALT = 12\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 289806d0c7..4a03766f1f 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -972,4 +972,200 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '\n \n '\n \n+test_expect_success 'detect moved code, complete file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >test.c &&\n+\t#include\n+\tmain()\n+\t{\n+\tprintf(\"Hello World\");\n+\t}\n+\tEOF\n+\tgit add test.c &&\n+\tgit commit -m \"add main function\" &&\n+\tgit mv test.c main.c &&\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tdiff --git a/main.c b/main.c\n+\tnew file mode 100644\n+\tindex 0000000..a986c57\n+\t--- /dev/null\n+\t+++ b/main.c\n+\t@@ -0,0 +1,5 @@\n+\t+#include\n+\t+main()\n+\t+{\n+\t+printf(\"Hello World\");\n+\t+}\n+\tdiff --git a/test.c b/test.c\n+\tdeleted file mode 100644\n+\tindex a986c57..0000000\n+\t--- a/test.c\n+\t+++ /dev/null\n+\t@@ -1,5 +0,0 @@\n+\t-#include\n+\t-main()\n+\t-{\n+\t-printf(\"Hello World\");\n+\t-}\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect malicious moved code, inside file' '\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit add main.c test.c &&\n+\tgit commit -m \"add main and test file\" &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tfoo(u);\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit diff HEAD --no-renames --color-moved=zebra| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'no effect from --color-moved with --word-diff' '\n+\tcat <<-\\EOF >text.txt &&\n+\tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n+\tEOF\n+\tgit add text.txt &&\n+\tgit commit -a -m \"clean state\" &&\n+\tcat <<-\\EOF >text.txt &&\n+\tsimply Lorem Ipsum dummy is text of the typesetting and printing industry.\n+\tEOF\n+\tgit diff --color-moved --word-diff >actual &&\n+\tgit diff --word-diff >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'move detection with submodules' '\n+\ttest_create_repo bananas &&\n+\techo ripe >bananas/recipe &&\n+\tgit -C bananas add recipe &&\n+\ttest_commit fruit &&\n+\ttest_commit -C bananas recipe &&\n+\tgit submodule add ./bananas &&\n+\tgit add bananas &&\n+\tgit commit -a -m \"bananas are like a heavy library?\" &&\n+\techo foul >bananas/recipe &&\n+\techo ripe >fruit.t &&\n+\n+\tgit diff --submodule=diff --color-moved >actual &&\n+\n+\t# no move detection as the moved line is across repository boundaries.\n+\ttest_decode_color decoded_actual &&\n+\t! grep BGREEN decoded_actual &&\n+\t! grep BRED decoded_actual &&\n+\n+\t# nor did we mess with it another way\n+\tgit diff --submodule=diff | test_decode_color >expect &&\n+\ttest_cmp expect decoded_actual\n+'\n+\n test_done\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323036","messageId":"20170623012919.28693-16-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 15/25] submodule.c: migrate diff output to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:09Z","receivedAt":"2017-06-23T01:30:46Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"As the submodule process is no longer attached to the same file pointer\n'o->file' as the superprojects process, there is a different result in\ncolor.c::check_auto_color. That is why we need to pass coloring explicitly,\nsuch that the submodule coloring decision will be made by the child process\nprocessing the submodule. Only DIFF_SYMBOL_SUBMODULE_PIPETHROUGH contains\ncolor, the other symbols are for embedding the submodule output into the\nsuperprojects output.\n\nRemove the colors from the function signatures, as all the coloring\ndecisions will be made either inside the child process or the final\nemit_diff_symbol, but not in the functions driving the submodule diff.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++---------\n diff.h | 9 +++++++\n submodule.c | 85 +++++++++++++++++++++++++++----------------------------------\n submodule.h | 13 +++-------\n 4 files changed, 122 insertions(+), 68 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 0314d57647..bc78a216ab 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,13 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_SUBMODULE_ADD,\n+\tDIFF_SYMBOL_SUBMODULE_DEL,\n+\tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\tDIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\tDIFF_SYMBOL_SUBMODULE_HEADER,\n+\tDIFF_SYMBOL_SUBMODULE_ERROR,\n+\tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n \tDIFF_SYMBOL_REWRITE_DIFF,\n \tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n@@ -624,6 +631,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_0(o, context, reset, '\\\\',\n \t\t\t nneof, strlen(nneof));\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n+\tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n+\tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -700,11 +710,68 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line(o, fraginfo, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_ADD:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, set, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_DEL:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, set, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_UNTRACKED:\n+\t\tfprintf(o->file, \"%sSubmodule %s contains untracked content\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_MODIFIED:\n+\t\tfprintf(o->file, \"%sSubmodule %s contains modified content\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n }\n \n+void diff_emit_submodule_del(struct diff_options *o, const char *line)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);\n+}\n+\n+void diff_emit_submodule_add(struct diff_options *o, const char *line)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ADD, line, strlen(line), 0);\n+}\n+\n+void diff_emit_submodule_untracked(struct diff_options *o, const char *path)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\t\t\t path, strlen(path), 0);\n+}\n+\n+void diff_emit_submodule_modified(struct diff_options *o, const char *path)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\t\t\t path, strlen(path), 0);\n+}\n+\n+void diff_emit_submodule_header(struct diff_options *o, const char *header)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_HEADER,\n+\t\t\t header, strlen(header), 0);\n+}\n+\n+void diff_emit_submodule_error(struct diff_options *o, const char *err)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ERROR, err, strlen(err), 0);\n+}\n+\n+void diff_emit_submodule_pipethrough(struct diff_options *o,\n+\t\t\t\t const char *line, int len)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_PIPETHROUGH, line, len, 0);\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -2467,24 +2534,16 @@ static void builtin_diff(const char *name_a,\n \tif (o->submodule_format == DIFF_SUBMODULE_LOG &&\n \t (!one->mode || S_ISGITLINK(one->mode)) &&\n \t (!two->mode || S_ISGITLINK(two->mode))) {\n-\t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n-\t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_summary(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_summary(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n-\t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset);\n+\t\t\t\ttwo->dirty_submodule);\n \t\treturn;\n \t} else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&\n \t\t (!one->mode || S_ISGITLINK(one->mode)) &&\n \t\t (!two->mode || S_ISGITLINK(two->mode))) {\n-\t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n-\t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_inline_diff(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_inline_diff(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n-\t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset, o);\n+\t\t\t\ttwo->dirty_submodule);\n \t\treturn;\n \t}\n \ndiff --git a/diff.h b/diff.h\nindex 8483ca0991..2ee0ef3908 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,15 @@ struct diff_options {\n \tint diff_path_counter;\n };\n \n+void diff_emit_submodule_del(struct diff_options *o, const char *line);\n+void diff_emit_submodule_add(struct diff_options *o, const char *line);\n+void diff_emit_submodule_untracked(struct diff_options *o, const char *path);\n+void diff_emit_submodule_modified(struct diff_options *o, const char *path);\n+void diff_emit_submodule_header(struct diff_options *o, const char *header);\n+void diff_emit_submodule_error(struct diff_options *o, const char *err);\n+void diff_emit_submodule_pipethrough(struct diff_options *o,\n+\t\t\t\t const char *line, int len);\n+\n enum color_diff {\n \tDIFF_RESET = 0,\n \tDIFF_CONTEXT = 1,\ndiff --git a/submodule.c b/submodule.c\nindex d3299e29c0..71471f3241 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -362,9 +362,7 @@ static int prepare_submodule_summary(struct rev_info *rev, const char *path,\n \treturn prepare_revision_walk(rev);\n }\n \n-static void print_submodule_summary(struct rev_info *rev, FILE *f,\n-\t\tconst char *line_prefix,\n-\t\tconst char *del, const char *add, const char *reset)\n+static void print_submodule_summary(struct rev_info *rev, struct diff_options *o)\n {\n \tstatic const char format[] = \" %m %s\";\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -375,18 +373,12 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \t\tctx.date_mode = rev->date_mode;\n \t\tctx.output_encoding = get_log_output_encoding();\n \t\tstrbuf_setlen(&sb, 0);\n-\t\tstrbuf_addstr(&sb, line_prefix);\n-\t\tif (commit->object.flags & SYMMETRIC_LEFT) {\n-\t\t\tif (del)\n-\t\t\t\tstrbuf_addstr(&sb, del);\n-\t\t}\n-\t\telse if (add)\n-\t\t\tstrbuf_addstr(&sb, add);\n \t\tformat_commit_message(commit, format, &sb, &ctx);\n-\t\tif (reset)\n-\t\t\tstrbuf_addstr(&sb, reset);\n \t\tstrbuf_addch(&sb, '\\n');\n-\t\tfprintf(f, \"%s\", sb.buf);\n+\t\tif (commit->object.flags & SYMMETRIC_LEFT)\n+\t\t\tdiff_emit_submodule_del(o, sb.buf);\n+\t\telse\n+\t\t\tdiff_emit_submodule_add(o, sb.buf);\n \t}\n \tstrbuf_release(&sb);\n }\n@@ -413,11 +405,9 @@ void prepare_submodule_repo_env(struct argv_array *out)\n * attempt to lookup both the left and right commits and put them into the\n * left and right pointers.\n */\n-static void show_submodule_header(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+static void show_submodule_header(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *reset,\n+\t\tunsigned dirty_submodule,\n \t\tstruct commit **left, struct commit **right,\n \t\tstruct commit_list **merge_bases)\n {\n@@ -426,11 +416,10 @@ static void show_submodule_header(FILE *f, const char *path,\n \tint fast_forward = 0, fast_backward = 0;\n \n \tif (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED)\n-\t\tfprintf(f, \"%sSubmodule %s contains untracked content\\n\",\n-\t\t\tline_prefix, path);\n+\t\tdiff_emit_submodule_untracked(o, path);\n+\n \tif (dirty_submodule & DIRTY_SUBMODULE_MODIFIED)\n-\t\tfprintf(f, \"%sSubmodule %s contains modified content\\n\",\n-\t\t\tline_prefix, path);\n+\t\tdiff_emit_submodule_modified(o, path);\n \n \tif (is_null_oid(one))\n \t\tmessage = \"(new submodule)\";\n@@ -472,31 +461,29 @@ static void show_submodule_header(FILE *f, const char *path,\n \t}\n \n output_header:\n-\tstrbuf_addf(&sb, \"%s%sSubmodule %s \", line_prefix, meta, path);\n+\tstrbuf_addf(&sb, \"Submodule %s \", path);\n \tstrbuf_add_unique_abbrev(&sb, one->hash, DEFAULT_ABBREV);\n \tstrbuf_addstr(&sb, (fast_backward || fast_forward) ? \"..\" : \"...\");\n \tstrbuf_add_unique_abbrev(&sb, two->hash, DEFAULT_ABBREV);\n \tif (message)\n-\t\tstrbuf_addf(&sb, \" %s%s\\n\", message, reset);\n+\t\tstrbuf_addf(&sb, \" %s\\n\", message);\n \telse\n-\t\tstrbuf_addf(&sb, \"%s:%s\\n\", fast_backward ? \" (rewind)\" : \"\", reset);\n-\tfwrite(sb.buf, sb.len, 1, f);\n+\t\tstrbuf_addf(&sb, \"%s:\\n\", fast_backward ? \" (rewind)\" : \"\");\n+\tdiff_emit_submodule_header(o, sb.buf);\n \n \tstrbuf_release(&sb);\n }\n \n-void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset)\n+\t\tunsigned dirty_submodule)\n {\n \tstruct rev_info rev;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n-\t\t\t meta, reset, &left, &right, &merge_bases);\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n+\t\t\t &left, &right, &merge_bases);\n \n \t/*\n \t * If we don't have both a left and a right pointer, there is no\n@@ -508,11 +495,11 @@ void show_submodule_summary(FILE *f, const char *path,\n \n \t/* Treat revision walker failure the same as missing commits */\n \tif (prepare_submodule_summary(&rev, path, left, right, merge_bases)) {\n-\t\tfprintf(f, \"%s(revision walker failed)\\n\", line_prefix);\n+\t\tdiff_emit_submodule_error(o, \"(revision walker failed)\\n\");\n \t\tgoto out;\n \t}\n \n-\tprint_submodule_summary(&rev, f, line_prefix, del, add, reset);\n+\tprint_submodule_summary(&rev, o);\n \n out:\n \tif (merge_bases)\n@@ -521,21 +508,18 @@ void show_submodule_summary(FILE *f, const char *path,\n \tclear_commit_marks(right, ~0);\n }\n \n-void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *o)\n+\t\tunsigned dirty_submodule)\n {\n \tconst struct object_id *old = &empty_tree_oid, *new = &empty_tree_oid;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n-\tstruct strbuf submodule_dir = STRBUF_INIT;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n-\t\t\t meta, reset, &left, &right, &merge_bases);\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n+\t\t\t &left, &right, &merge_bases);\n \n \t/* We need a valid left and right commit to display a difference */\n \tif (!(left || is_null_oid(one)) ||\n@@ -547,15 +531,16 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \tif (right)\n \t\tnew = two;\n \n-\tfflush(f);\n \tcp.git_cmd = 1;\n \tcp.dir = path;\n-\tcp.out = dup(fileno(f));\n+\tcp.out = -1;\n \tcp.no_stdin = 1;\n \n \t/* TODO: other options may need to be passed here. */\n \targv_array_push(&cp.args, \"diff\");\n-\targv_array_pushf(&cp.args, \"--line-prefix=%s\", line_prefix);\n+\targv_array_pushf(&cp.args, \"--color=%s\", want_color(o->use_color) ?\n+\t\t\t \"always\" : \"never\");\n+\n \tif (DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\targv_array_pushf(&cp.args, \"--src-prefix=%s%s/\",\n \t\t\t\t o->b_prefix, path);\n@@ -578,11 +563,17 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \t\targv_array_push(&cp.args, oid_to_hex(new));\n \n \tprepare_submodule_repo_env(&cp.env_array);\n-\tif (run_command(&cp))\n-\t\tfprintf(f, \"(diff failed)\\n\");\n+\tif (start_command(&cp))\n+\t\tdiff_emit_submodule_error(o, \"(diff failed)\\n\");\n+\n+\twhile (strbuf_getwholeline_fd(&sb, cp.out, '\\n') != EOF)\n+\t\tdiff_emit_submodule_pipethrough(o, sb.buf, sb.len);\n+\n+\tif (finish_command(&cp))\n+\t\tdiff_emit_submodule_error(o, \"(diff failed)\\n\");\n \n done:\n-\tstrbuf_release(&submodule_dir);\n+\tstrbuf_release(&sb);\n \tif (merge_bases)\n \t\tfree_commit_list(merge_bases);\n \tif (left)\ndiff --git a/submodule.h b/submodule.h\nindex 1277480add..e7358863e9 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -53,17 +53,12 @@ extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n extern void handle_ignore_submodules_arg(struct diff_options *, const char *);\n-extern void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+extern void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset);\n-extern void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+\t\tunsigned dirty_submodule);\n+extern void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *opt);\n+\t\tunsigned dirty_submodule);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n /* Check if we want to update any submodule.*/\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323037","messageId":"20170623012919.28693-26-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 25/25] diff: document the new --color-moved setting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:19Z","receivedAt":"2017-06-23T01:30:47Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n Documentation/config.txt | 12 ++++++++++--\n Documentation/diff-options.txt | 27 +++++++++++++++++++++++++++\n 2 files changed, 37 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 475e874d51..29e0b9fa69 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1051,14 +1051,22 @@ This does not affect linkgit:git-format-patch[1] or the\n 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n command line with the `--color[=]` option.\n \n+diff.colorMoved::\n+\tIf set moved lines in a diff are colored differently,\n+\tfor details see '--color-moved' in linkgit:git-diff[1].\n+\n color.diff.::\n \tUse customized color for diff colorization. `` specifies\n \twhich part of the patch to use the specified color, and is one\n \tof `context` (context text - `plain` is a historical synonym),\n \t`meta` (metainformation), `frag`\n \t(hunk header), 'func' (function in hunk header), `old` (removed lines),\n-\t`new` (added lines), `commit` (commit headers), or `whitespace`\n-\t(highlighting whitespace errors).\n+\t`new` (added lines), `commit` (commit headers), `whitespace`\n+\t(highlighting whitespace errors), `oldMoved` (deleted lines),\n+\t`newMoved` (added lines), `oldMovedDimmed`, `oldMovedAlternative`,\n+\t`oldMovedAlternativeDimmed`, `newMovedDimmed`, `newMovedAlternative`\n+\tand `newMovedAlternativeDimmed` (See the ''\n+\tsetting of '--color-moved' in linkgit:git-diff[1] for details).\n \n color.decorate.::\n \tUse customized color for 'git log --decorate' output. `` is one\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex 89cc0f48de..058c8014ed 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -231,6 +231,33 @@ ifdef::git-diff[]\n endif::git-diff[]\n \tIt is the same as `--color=never`.\n \n+--color-moved[=]::\n+\tMoved lines of code are colored differently.\n+ifdef::git-diff[]\n+\tIt can be changed by the `diff.colorMoved` configuration setting.\n+endif::git-diff[]\n+\tThe defaults to 'no' if the option is not given\n+\tand to 'dimmed_zebra' if the option with no mode is given.\n+\tThe mode must be one of:\n++\n+--\n+no::\n+\tMoved lines are not highlighted.\n+plain::\n+\tAny line that is added in one location and was removed\n+\tin another location will be colored with 'color.diff.newMoved'.\n+\tSimilarly 'color.diff.oldMoved' will be used for removed lines\n+\tthat are added somewhere else in the diff.\n+zebra::\n+\tBlocks of moved code are detected. The detected blocks are\n+\tpainted using the 'color.diff.{old,new}Moved' alternating with\n+\t'color.diff.{old,new}MovedAlternative'.\n+dimmed_zebra::\n+\tSimilar to 'zebra', but additional dimming of uninteresting parts\n+\tof moved code is performed. The bordering lines of two adjacent\n+\tblocks are considered interesting, the rest is uninteresting.\n+--\n+\n --word-diff[=]::\n \tShow a word diff, using the to delimit changed words.\n \tBy default, words are delimited by whitespace; see\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323038","messageId":"20170623012919.28693-12-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 11/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:05Z","receivedAt":"2017-06-23T01:30:49Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We have to use fprintf instead of emit_line, because we want to emit the\ntab after the color. This is important for ancient versions of gnu patch\nAFAICT, although we probably do not want to feed colored output to the\npatch utility, such that it would not matter if the trailing tab is\ncolored. Keep the corner case as-is though.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 51 ++++++++++++++++++++++++++++++---------------------\n 1 file changed, 30 insertions(+), 21 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex f3d0918810..761ee581ad 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,8 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_FILEPAIR_PLUS,\n+\tDIFF_SYMBOL_FILEPAIR_MINUS,\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n \tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n@@ -610,7 +612,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset, *set;\n+\tconst char *context, *reset, *set, *meta;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -672,6 +674,20 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t}\n \t\temit_line(o, context, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_FILEPAIR_PLUS:\n+\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\", diff_line_prefix(o), meta,\n+\t\t\tline, reset,\n+\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_FILEPAIR_MINUS:\n+\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\", diff_line_prefix(o), meta,\n+\t\t\tline, reset,\n+\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -843,8 +859,6 @@ static void emit_rewrite_diff(const char *name_a,\n \t\t\t struct diff_options *o)\n {\n \tint lc_a, lc_b;\n-\tconst char *name_a_tab, *name_b_tab;\n-\tconst char *metainfo = diff_get_color(o->use_color, DIFF_METAINFO);\n \tconst char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n \tconst char *reset = diff_get_color(o->use_color, DIFF_RESET);\n \tstatic struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;\n@@ -864,8 +878,6 @@ static void emit_rewrite_diff(const char *name_a,\n \n \tname_a += (*name_a == '/');\n \tname_b += (*name_b == '/');\n-\tname_a_tab = strchr(name_a, ' ') ? \"\\t\" : \"\";\n-\tname_b_tab = strchr(name_b, ' ') ? \"\\t\" : \"\";\n \n \tstrbuf_reset(&a_name);\n \tstrbuf_reset(&b_name);\n@@ -892,11 +904,13 @@ static void emit_rewrite_diff(const char *name_a,\n \n \tlc_a = count_lines(data_one, size_one);\n \tlc_b = count_lines(data_two, size_two);\n-\tfprintf(o->file,\n-\t\t\"%s%s--- %s%s%s\\n%s%s+++ %s%s%s\\n%s%s@@ -\",\n-\t\tline_prefix, metainfo, a_name.buf, name_a_tab, reset,\n-\t\tline_prefix, metainfo, b_name.buf, name_b_tab, reset,\n-\t\tline_prefix, fraginfo);\n+\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,\n+\t\t\t a_name.buf, a_name.len, 0);\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n+\t\t\t b_name.buf, b_name.len, 0);\n+\n+\tfprintf(o->file, \"%s%s@@ -\", line_prefix, fraginfo);\n \tif (!o->irreversible_delete)\n \t\tprint_line_count(o->file, lc_a);\n \telse\n@@ -1365,10 +1379,8 @@ static void find_lno(const char *line, struct emit_callback *ecbdata)\n static void fn_out_consume(void *priv, char *line, unsigned long len)\n {\n \tstruct emit_callback *ecbdata = priv;\n-\tconst char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n-\tconst char *line_prefix = diff_line_prefix(o);\n \n \to->found_changes = 1;\n \n@@ -1379,15 +1391,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->label_path[0]) {\n-\t\tconst char *name_a_tab, *name_b_tab;\n-\n-\t\tname_a_tab = strchr(ecbdata->label_path[0], ' ') ? \"\\t\" : \"\";\n-\t\tname_b_tab = strchr(ecbdata->label_path[1], ' ') ? \"\\t\" : \"\";\n-\n-\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);\n-\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,\n+\t\t\t\t ecbdata->label_path[0],\n+\t\t\t\t strlen(ecbdata->label_path[0]), 0);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n+\t\t\t\t ecbdata->label_path[1],\n+\t\t\t\t strlen(ecbdata->label_path[1]), 0);\n \t\tecbdata->label_path[0] = ecbdata->label_path[1] = NULL;\n \t}\n \n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323039","messageId":"20170623012919.28693-25-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 24/25] diff.c: add dimming to moved line detection","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:18Z","receivedAt":"2017-06-23T01:30:51Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Any lines inside a moved block of code are not interesting. Boundaries\nof blocks are only interesting if they are next to another block of moved\ncode.\n\nSigned-off-by: Stefan Beller \n---\n color.h | 2 +\n diff.c | 139 ++++++++++++++++++++++++++++++++++++++++-----\n diff.h | 9 ++-\n t/t4015-diff-whitespace.sh | 124 ++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 259 insertions(+), 15 deletions(-)\n\ndiff --git a/color.h b/color.h\nindex 90627650fc..fd2b688dfb 100644\n--- a/color.h\n+++ b/color.h\n@@ -42,6 +42,8 @@ struct strbuf;\n #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n+#define GIT_COLOR_FAINT\t\t\"\\033[2m\"\n+#define GIT_COLOR_FAINT_ITALIC\t\"\\033[2;3m\"\n \n /* A special value meaning \"no color selected\" */\n #define GIT_COLOR_NIL \"NIL\"\ndiff --git a/diff.c b/diff.c\nindex 0eb744514f..82ace48c38 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -57,10 +57,14 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n-\tGIT_COLOR_MAGENTA,\t/* OLD_MOVED */\n-\tGIT_COLOR_BLUE,\t\t/* OLD_MOVED ALTERNATIVE */\n-\tGIT_COLOR_CYAN,\t\t/* NEW_MOVED */\n-\tGIT_COLOR_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n+\tGIT_COLOR_BOLD_MAGENTA,\t/* OLD_MOVED */\n+\tGIT_COLOR_BOLD_BLUE,\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_FAINT,\t/* OLD_MOVED_DIM */\n+\tGIT_COLOR_FAINT_ITALIC,\t/* OLD_MOVED_ALTERNATIVE_DIM */\n+\tGIT_COLOR_BOLD_CYAN,\t/* NEW_MOVED */\n+\tGIT_COLOR_BOLD_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n+\tGIT_COLOR_FAINT,\t/* NEW_MOVED_DIM */\n+\tGIT_COLOR_FAINT_ITALIC,\t/* NEW_MOVED_ALTERNATIVE_DIM */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -90,10 +94,18 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_FILE_OLD_MOVED;\n \tif (!strcasecmp(var, \"oldmovedalternative\"))\n \t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"oldmoveddimmed\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_DIM;\n+\tif (!strcasecmp(var, \"oldmovedalternativedimmed\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT_DIM;\n \tif (!strcasecmp(var, \"newmoved\"))\n \t\treturn DIFF_FILE_NEW_MOVED;\n \tif (!strcasecmp(var, \"newmovedalternative\"))\n \t\treturn DIFF_FILE_NEW_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoveddimmed\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_DIM;\n+\tif (!strcasecmp(var, \"newmovedalternativedimmed\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT_DIM;\n \treturn -1;\n }\n \n@@ -250,6 +262,8 @@ static int parse_color_moved(const char *arg)\n \t\treturn COLOR_MOVED_PLAIN;\n \telse if (!strcmp(arg, \"zebra\"))\n \t\treturn COLOR_MOVED_ZEBRA;\n+\telse if (!strcmp(arg, \"dimmed_zebra\"))\n+\t\treturn COLOR_MOVED_ZEBRA_DIM;\n \telse\n \t\treturn -1;\n }\n@@ -637,6 +651,7 @@ enum diff_symbol {\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n #define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n #define DIFF_SYMBOL_MOVED_LINE_ALT\t\t(1<<18)\n+#define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING\t(1<<19)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n /*\n@@ -890,6 +905,67 @@ static void mark_color_as_moved(struct diff_options *o,\n \tfree(pmb);\n }\n \n+#define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \\\n+ (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ALT)\n+static void dim_moved_lines(struct diff_options *o)\n+{\n+\tint n;\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct emitted_diff_symbol *prev = (n != 0) ?\n+\t\t\t\t&o->emitted_symbols->buf[n - 1] : NULL;\n+\t\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];\n+\t\tstruct emitted_diff_symbol *next =\n+\t\t\t\t(n < o->emitted_symbols->nr - 1) ?\n+\t\t\t\t&o->emitted_symbols->buf[n + 1] : NULL;\n+\n+\t\t/* Not a plus or minus line? */\n+\t\tif (l->s != DIFF_SYMBOL_PLUS && l->s != DIFF_SYMBOL_MINUS)\n+\t\t\tcontinue;\n+\n+\t\t/* Not a moved line? */\n+\t\tif (!(l->flags & DIFF_SYMBOL_MOVED_LINE))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * If prev or next are not a plus or minus line,\n+\t\t * pretend they don't exist\n+\t\t */\n+\t\tif (prev && prev->s != DIFF_SYMBOL_PLUS &&\n+\t\t\t prev->s != DIFF_SYMBOL_MINUS)\n+\t\t\tprev = NULL;\n+\t\tif (next && next->s != DIFF_SYMBOL_PLUS &&\n+\t\t\t next->s != DIFF_SYMBOL_MINUS)\n+\t\t\tnext = NULL;\n+\n+\t\t/* Inside a block? */\n+\t\tif ((prev &&\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK)) &&\n+\t\t (next &&\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK))) {\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Check if we are at an interesting bound: */\n+\t\tif (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))\n+\t\t\tcontinue;\n+\t\tif (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * The boundary to prev and next are not interesting,\n+\t\t * so this line is not interesting as a whole\n+\t\t */\n+\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;\n+\t}\n+}\n+\n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n \t\t\t\tconst char *line, int len, char sign,\n@@ -964,24 +1040,56 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n \t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n-\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n-\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);\n-\t\telse\n+\t\t\tbreak;\n+\t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\t}\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '+',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n-\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n-\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);\n-\t\telse\n+\t\t\tbreak;\n+\t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\t}\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n@@ -4534,9 +4642,12 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n-\telse if (!strcmp(arg, \"--color-moved\"))\n-\t\toptions->color_moved = COLOR_MOVED_ZEBRA;\n-\telse if (!strcmp(arg, \"--no-color-moved\"))\n+\telse if (!strcmp(arg, \"--color-moved\")) {\n+\t\tif (diff_color_moved_default)\n+\t\t\toptions->color_moved = diff_color_moved_default;\n+\t\tif (options->color_moved == COLOR_MOVED_NO)\n+\t\t\toptions->color_moved = COLOR_MOVED_ZEBRA_DIM;\n+\t} else if (!strcmp(arg, \"--no-color-moved\"))\n \t\toptions->color_moved = COLOR_MOVED_NO;\n \telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n \t\tint cm = parse_color_moved(arg);\n@@ -5381,6 +5492,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \n \t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n \t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\t\t\tif (o->color_moved == COLOR_MOVED_ZEBRA_DIM)\n+\t\t\t\tdim_moved_lines(o);\n \n \t\t\thashmap_free(&add_lines, 0);\n \t\t\thashmap_free(&del_lines, 0);\ndiff --git a/diff.h b/diff.h\nindex 1aae8738ca..98abd75521 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -192,6 +192,7 @@ struct diff_options {\n \t\tCOLOR_MOVED_NO = 0,\n \t\tCOLOR_MOVED_PLAIN = 1,\n \t\tCOLOR_MOVED_ZEBRA = 2,\n+\t\tCOLOR_MOVED_ZEBRA_DIM = 3,\n \t} color_moved;\n };\n \n@@ -216,8 +217,12 @@ enum color_diff {\n \tDIFF_FUNCINFO = 8,\n \tDIFF_FILE_OLD_MOVED = 9,\n \tDIFF_FILE_OLD_MOVED_ALT = 10,\n-\tDIFF_FILE_NEW_MOVED = 11,\n-\tDIFF_FILE_NEW_MOVED_ALT = 12\n+\tDIFF_FILE_OLD_MOVED_DIM = 11,\n+\tDIFF_FILE_OLD_MOVED_ALT_DIM = 12,\n+\tDIFF_FILE_NEW_MOVED = 13,\n+\tDIFF_FILE_NEW_MOVED_ALT = 14,\n+\tDIFF_FILE_NEW_MOVED_DIM = 15,\n+\tDIFF_FILE_NEW_MOVED_ALT_DIM = 16\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 1ca16435d6..ae8c686f3c 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -1179,6 +1179,130 @@ test_expect_success 'plain moved code, inside file' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'detect permutations inside moved code -- dimmed_zebra' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\tEOF\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\tEOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\ttest_config color.diff.oldMovedDimmed \"normal magenta\" &&\n+\ttest_config color.diff.newMovedDimmed \"normal cyan\" &&\n+\ttest_config color.diff.oldMovedAlternativeDimmed \"normal blue\" &&\n+\ttest_config color.diff.newMovedAlternativeDimmed \"normal yellow\" &&\n+\tgit diff HEAD --no-renames --color-moved=dimmed_zebra| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'cmd option assumes configured colored-moved' '\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\ttest_config color.diff.oldMovedDimmed \"normal magenta\" &&\n+\ttest_config color.diff.newMovedDimmed \"normal cyan\" &&\n+\ttest_config color.diff.oldMovedAlternativeDimmed \"normal blue\" &&\n+\ttest_config color.diff.newMovedAlternativeDimmed \"normal yellow\" &&\n+\ttest_config diff.colorMoved zebra &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'no effect from --color-moved with --word-diff' '\n \tcat <<-\\EOF >text.txt &&\n \tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323040","messageId":"20170623012919.28693-2-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 01/25] diff.c: readability fix","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:28:55Z","receivedAt":"2017-06-23T01:30:55Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We already have dereferenced 'p->two' into a local variable 'two'.\nUse that.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 74283d9001..3f5bf8b5a4 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3283,8 +3283,8 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \tconst char *other;\n \tconst char *attr_path;\n \n-\tname = p->one->path;\n-\tother = (strcmp(name, p->two->path) ? p->two->path : NULL);\n+\tname = one->path;\n+\tother = (strcmp(name, two->path) ? two->path : NULL);\n \tattr_path = name;\n \tif (o->prefix_length)\n \t\tstrip_prefix(o->prefix_length, &name, &other);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323041","messageId":"20170623012919.28693-11-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 10/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:04Z","receivedAt":"2017-06-23T01:30:56Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The context marker use the exact same output pattern, so reuse it.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 3d081edd12..f3d0918810 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -563,6 +563,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n \tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_CONTEXT_INCOMPLETE,\n \tDIFF_SYMBOL_PLUS,\n \tDIFF_SYMBOL_MINUS,\n \tDIFF_SYMBOL_NO_LF_EOF,\n@@ -621,6 +622,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT_INCOMPLETE:\n \tcase DIFF_SYMBOL_CONTEXT_MARKER:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -1448,8 +1450,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \tdefault:\n \t\t/* incomplete line at the end */\n \t\tecbdata->lno_in_preimage++;\n-\t\temit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),\n-\t\t\t reset, line, len);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_CONTEXT_INCOMPLETE,\n+\t\t\t\t line, len, 0);\n \t\tbreak;\n \t}\n }\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323042","messageId":"20170623012919.28693-22-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 21/25] diff.c: buffer all output if asked to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:15Z","receivedAt":"2017-06-23T01:30:58Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Introduce a new option 'emitted_symbols' in the struct diff_options which\ncontrols whether all output is buffered up until all output is available.\nIt is set internally in diff.c when necessary.\n\nWe'll have a new struct 'emitted_string' in diff.c which will be used to\nbuffer each line. The emitted_string will duplicate the memory of the\nline to buffer as that is easiest to reason about for now. In a future\npatch we may want to decrease the memory usage by not duplicating all\noutput for buffering but rather we may want to store offsets into the\nfile or in case of hunk descriptions such as the similarity score, we\ncould just store the relevant number and reproduce the text later on.\n\nThis approach was chosen as a first step because it is quite simple\ncompared to the alternative with less memory footprint.\n\nemit_diff_symbol factors out the emission part and depending on the\ndiff_options->emitted_symbols the emission will be performed directly\nwhen calling emit_diff_symbol or after the whole process is done, i.e.\nby buffering we have add the possibility for a second pass over the\nwhole output before doing the actual output.\n\nIn 6440d34 (2012-03-14, diff: tweak a _copy_ of diff_options with\nword-diff) we introduced a duplicate diff options struct for word\nemissions as we may have different regex settings in there.\nWhen buffering the output, we need to operate on just one buffer,\nso we have to copy back the emissions of the word buffer into the\nmain buffer.\n\nUnconditionally enable output via buffer in this patch as it yields\na great opportunity for testing, i.e. all the diff tests from the\ntest suite pass without having reordering issues (i.e. only parts\nof the output got buffered, and we forgot to buffer other parts).\nThe test suite passes, which gives confidence that we converted all\nfunctions to use emit_string for output.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--\n diff.h | 2 ++\n 2 files changed, 109 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex f9fb94b0d3..b344808c20 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -604,6 +604,47 @@ enum diff_symbol {\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n+/*\n+ * This struct is used when we need to buffer the output of the diff output.\n+ *\n+ * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer\n+ * into the pre/post image file. This pointer could be a union with the\n+ * line pointer. By storing an offset into the file instead of the literal line,\n+ * we can decrease the memory footprint for the buffered output. At first we\n+ * may want to only have indirection for the content lines, but we could also\n+ * enhance the state for emitting prefabricated lines, e.g. the similarity\n+ * score line or hunk/file headers would only need to store a number or path\n+ * and then the output can be constructed later on depending on state.\n+ */\n+struct emitted_diff_symbol {\n+\tconst char *line;\n+\tint len;\n+\tint flags;\n+\tenum diff_symbol s;\n+};\n+#define EMITTED_DIFF_SYMBOL_INIT {NULL}\n+\n+struct emitted_diff_symbols {\n+\tstruct emitted_diff_symbol *buf;\n+\tint nr, alloc;\n+};\n+#define EMITTED_DIFF_SYMBOLS_INIT {NULL, 0, 0}\n+\n+static void append_emitted_diff_symbol(struct diff_options *o,\n+\t\t\t\t struct emitted_diff_symbol *e)\n+{\n+\tstruct emitted_diff_symbol *f;\n+\n+\tALLOC_GROW(o->emitted_symbols->buf,\n+\t\t o->emitted_symbols->nr + 1,\n+\t\t o->emitted_symbols->alloc);\n+\tf = &o->emitted_symbols->buf[o->emitted_symbols->nr++];\n+\n+\tmemcpy(f, e, sizeof(struct emitted_diff_symbol));\n+\tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n+}\n+\n+\n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n \t\t\t\tconst char *line, int len, char sign,\n@@ -630,12 +671,18 @@ static void emit_line_ws_markup(struct diff_options *o,\n \t}\n }\n \n-static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n-\t\t\t const char *line, int len, unsigned flags)\n+static void emit_diff_symbol_from_struct(struct diff_options *o,\n+\t\t\t\t\t struct emitted_diff_symbol *eds)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n \tstruct strbuf sb = STRBUF_INIT;\n+\n+\tenum diff_symbol s = eds->s;\n+\tconst char *line = eds->line;\n+\tint len = eds->len;\n+\tunsigned flags = eds->flags;\n+\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -777,6 +824,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tstrbuf_release(&sb);\n }\n \n+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n+\t\t\t const char *line, int len, unsigned flags)\n+{\n+\tstruct emitted_diff_symbol e = {line, len, flags, s};\n+\n+\tif (o->emitted_symbols)\n+\t\tappend_emitted_diff_symbol(o, &e);\n+\telse\n+\t\temit_diff_symbol_from_struct(o, &e);\n+}\n+\n void diff_emit_submodule_del(struct diff_options *o, const char *line)\n {\n \temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);\n@@ -1373,9 +1431,29 @@ static void diff_words_show(struct diff_words_data *diff_words)\n /* In \"color-words\" mode, show word-diff of words accumulated in the buffer */\n static void diff_words_flush(struct emit_callback *ecbdata)\n {\n+\tstruct diff_options *wo = ecbdata->diff_words->opt;\n+\n \tif (ecbdata->diff_words->minus.text.size ||\n \t ecbdata->diff_words->plus.text.size)\n \t\tdiff_words_show(ecbdata->diff_words);\n+\n+\tif (wo->emitted_symbols) {\n+\t\tstruct diff_options *o = ecbdata->opt;\n+\t\tstruct emitted_diff_symbols *wol = wo->emitted_symbols;\n+\t\tint i;\n+\n+\t\t/*\n+\t\t * NEEDSWORK:\n+\t\t * Instead of appending each, concat all words to a line?\n+\t\t */\n+\t\tfor (i = 0; i < wol->nr; i++)\n+\t\t\tappend_emitted_diff_symbol(o, &wol->buf[i]);\n+\n+\t\tfor (i = 0; i < wol->nr; i++)\n+\t\t\tfree((void *)wol->buf[i].line);\n+\n+\t\twol->nr = 0;\n+\t}\n }\n \n static void diff_filespec_load_driver(struct diff_filespec *one)\n@@ -1411,6 +1489,11 @@ static void init_diff_words_data(struct emit_callback *ecbdata,\n \t\txcalloc(1, sizeof(struct diff_words_data));\n \tecbdata->diff_words->type = o->word_diff;\n \tecbdata->diff_words->opt = o;\n+\n+\tif (orig_opts->emitted_symbols)\n+\t\to->emitted_symbols =\n+\t\t\txcalloc(1, sizeof(struct emitted_diff_symbols));\n+\n \tif (!o->word_regex)\n \t\to->word_regex = userdiff_word_regex(one);\n \tif (!o->word_regex)\n@@ -1445,6 +1528,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)\n {\n \tif (ecbdata->diff_words) {\n \t\tdiff_words_flush(ecbdata);\n+\t\tfree (ecbdata->diff_words->opt->emitted_symbols);\n \t\tfree (ecbdata->diff_words->opt);\n \t\tfree (ecbdata->diff_words->minus.text.ptr);\n \t\tfree (ecbdata->diff_words->minus.orig);\n@@ -4999,16 +5083,37 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n+\tstatic struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n \n \tif (WSEH_NEW & WS_RULE_MASK)\n \t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n \n+\t/*\n+\t * For testing purposes we want to make sure the diff machinery\n+\t * works completely with the buffer. If there is anything emitted\n+\t * outside the emit_string, then the order is screwed\n+\t * up and the tests will fail.\n+\t *\n+\t * TODO (later in this series):\n+\t * We'll unset this pointer in a later patch.\n+\t */\n+\to->emitted_symbols = &esm;\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tif (check_pair_status(p))\n \t\t\tdiff_flush_patch(p, o);\n \t}\n+\n+\tif (o->emitted_symbols) {\n+\t\tfor (i = 0; i < esm.nr; i++)\n+\t\t\temit_diff_symbol_from_struct(o, &esm.buf[i]);\n+\n+\t\tfor (i = 0; i < esm.nr; i++)\n+\t\t\tfree((void *)esm.buf[i].line);\n+\t}\n+\tesm.nr = 0;\n }\n \n void diff_flush(struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex b73f7a1e75..65fc9dbb4b 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -186,6 +186,8 @@ struct diff_options {\n \tvoid *output_prefix_data;\n \n \tint diff_path_counter;\n+\n+\tstruct emitted_diff_symbols *emitted_symbols;\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323043","messageId":"20170623012919.28693-9-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 08/25] diff.c: migrate emit_line_checked to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:02Z","receivedAt":"2017-06-23T01:30:59Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a new flags field to emit_diff_symbol, that will be used by\ncontext lines for:\n* white space rules that are applicable (The first 12 bits)\n Take a note in cahe.c as well, when this ws rules are extended we have\n to fix the bits in the flags field.\n* how the rules are evaluated (actually this double encodes the sign\n of the line, but the code is easier to keep this way, bits 13,14,15)\n* if the line a blank line at EOF (bit 16)\n\nThe check if new lines need to be marked up as extra lines at the end of\nfile, is now done unconditionally. That should be ok, as\n'new_blank_line_at_eof' has a quick early return.\n\nSigned-off-by: Stefan Beller \n---\n cache.h | 2 ++\n diff.c | 116 +++++++++++++++++++++++++++++++++++++++++-----------------------\n diff.h | 6 ++--\n 3 files changed, 80 insertions(+), 44 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex e1f0e182ad..d2204bf6d1 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -2168,6 +2168,8 @@ void shift_tree_by(const struct object_id *, const struct object_id *, struct ob\n #define WS_TRAILING_SPACE (WS_BLANK_AT_EOL|WS_BLANK_AT_EOF)\n #define WS_DEFAULT_RULE (WS_TRAILING_SPACE|WS_SPACE_BEFORE_TAB|8)\n #define WS_TAB_WIDTH_MASK 077\n+/* All WS_* -- when extended, adapt diff.c emit_symbol */\n+#define WS_RULE_MASK 07777\n extern unsigned whitespace_rule_cfg;\n extern unsigned whitespace_rule(const char *);\n extern unsigned parse_whitespace_rule(const char *);\ndiff --git a/diff.c b/diff.c\nindex b78f698cad..e7583efca3 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,17 +560,54 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_PLUS,\n+\tDIFF_SYMBOL_MINUS,\n \tDIFF_SYMBOL_NO_LF_EOF,\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n+/*\n+ * Flags for content lines:\n+ * 0..12 are whitespace rules\n+ * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT\n+ * 16 is marking if the line is blank at EOF\n+ */\n+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n+#define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n+\n+static void emit_line_ws_markup(struct diff_options *o,\n+\t\t\t\tconst char *set, const char *reset,\n+\t\t\t\tconst char *line, int len, char sign,\n+\t\t\t\tunsigned ws_rule, int blank_at_eof)\n+{\n+\tconst char *ws = NULL;\n+\n+\tif (o->ws_error_highlight & ws_rule) {\n+\t\tws = diff_get_color_opt(o, DIFF_WHITESPACE);\n+\t\tif (!*ws)\n+\t\t\tws = NULL;\n+\t}\n+\n+\tif (!ws)\n+\t\temit_line_0(o, set, reset, sign, line, len);\n+\telse if (blank_at_eof)\n+\t\t/* Blank line at EOF - paint '+' as well */\n+\t\temit_line_0(o, ws, reset, sign, line, len);\n+\telse {\n+\t\t/* Emit just the prefix, then the rest. */\n+\t\temit_line_0(o, set, reset, sign, \"\", 0);\n+\t\tws_check_emit(line, len, ws_rule,\n+\t\t\t o->file, set, reset, ws);\n+\t}\n+}\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n-\t\t\t const char *line, int len)\n+\t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset;\n+\tconst char *context, *reset, *set;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -592,6 +629,25 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tdiff_line_prefix(o),\n \t\t\to->line_termination);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT:\n+\t\tset = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, ' ',\n+\t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_PLUS:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, '+',\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_MINUS:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, '-',\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -608,57 +664,31 @@ static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line\n \treturn ws_blank_line(line, len, ecbdata->ws_rule);\n }\n \n-static void emit_line_checked(const char *reset,\n-\t\t\t struct emit_callback *ecbdata,\n-\t\t\t const char *line, int len,\n-\t\t\t enum color_diff color,\n-\t\t\t unsigned ws_error_highlight,\n-\t\t\t char sign)\n-{\n-\tconst char *set = diff_get_color(ecbdata->color_diff, color);\n-\tconst char *ws = NULL;\n-\n-\tif (ecbdata->opt->ws_error_highlight & ws_error_highlight) {\n-\t\tws = diff_get_color(ecbdata->color_diff, DIFF_WHITESPACE);\n-\t\tif (!*ws)\n-\t\t\tws = NULL;\n-\t}\n-\n-\tif (!ws)\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, line, len);\n-\telse if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))\n-\t\t/* Blank line at EOF - paint '+' as well */\n-\t\temit_line_0(ecbdata->opt, ws, reset, sign, line, len);\n-\telse {\n-\t\t/* Emit just the prefix, then the rest. */\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, \"\", 0);\n-\t\tws_check_emit(line, len, ecbdata->ws_rule,\n-\t\t\t ecbdata->opt->file, set, reset, ws);\n-\t}\n-}\n-\n static void emit_add_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_FILE_NEW, WSEH_NEW, '+');\n+\tunsigned flags = WSEH_NEW | ecbdata->ws_rule;\n+\tif (new_blank_line_at_eof(ecbdata, line, len))\n+\t\tflags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;\n+\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);\n }\n \n static void emit_del_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_FILE_OLD, WSEH_OLD, '-');\n+\tunsigned flags = WSEH_OLD | ecbdata->ws_rule;\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_MINUS, line, len, flags);\n }\n \n static void emit_context_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_CONTEXT, WSEH_CONTEXT, ' ');\n+\tunsigned flags = WSEH_CONTEXT | ecbdata->ws_rule;\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_CONTEXT, line, len, flags);\n }\n \n static void emit_hunk_header(struct emit_callback *ecbdata,\n@@ -682,7 +712,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n \t\temit_diff_symbol(ecbdata->opt,\n-\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len);\n+\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len, 0);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n@@ -718,7 +748,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n \temit_diff_symbol(ecbdata->opt,\n-\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);\n+\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len, 0);\n \tstrbuf_release(&msgbuf);\n }\n \n@@ -777,7 +807,7 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tdata += len;\n \t}\n \tif (!endp)\n-\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);\n+\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0, 0);\n }\n \n static void emit_rewrite_diff(const char *name_a,\n@@ -4774,6 +4804,10 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n+\n+\tif (WSEH_NEW & WS_RULE_MASK)\n+\t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tif (check_pair_status(p))\n@@ -4866,7 +4900,7 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\ndiff --git a/diff.h b/diff.h\nindex 5be1ee77a7..8483ca0991 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -148,9 +148,9 @@ struct diff_options {\n \tint abbrev;\n \tint ita_invisible_in_index;\n /* white-space error highlighting */\n-#define WSEH_NEW 1\n-#define WSEH_CONTEXT 2\n-#define WSEH_OLD 4\n+#define WSEH_NEW (1<<12)\n+#define WSEH_CONTEXT (1<<13)\n+#define WSEH_OLD (1<<14)\n \tunsigned ws_error_highlight;\n \tconst char *prefix;\n \tint prefix_length;\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323044","messageId":"20170623012919.28693-19-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 18/25] diff.c: convert word diffing to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:12Z","receivedAt":"2017-06-23T01:31:01Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The word diffing is not line oriented and would need some serious\neffort to be transformed into a line oriented approach, so\njust go with a symbol DIFF_SYMBOL_WORD_DIFF that is a partial line.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 79 ++++++++++++++++++++++++++++++++++++++----------------------------\n 1 file changed, 46 insertions(+), 33 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2108c191a3..60630d2c3f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -569,6 +569,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n \tDIFF_SYMBOL_STATS_LINE,\n+\tDIFF_SYMBOL_WORD_DIFF,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -761,6 +762,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n \t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n \t\tbreak;\n+\tcase DIFF_SYMBOL_WORD_DIFF:\n+\t\tfprintf(o->file, \"%.*s\", len, line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1091,37 +1095,49 @@ struct diff_words_data {\n \tstruct diff_words_style *style;\n };\n \n-static int fn_out_diff_words_write_helper(FILE *fp,\n+static int fn_out_diff_words_write_helper(struct diff_options *o,\n \t\t\t\t\t struct diff_words_style_elem *st_el,\n \t\t\t\t\t const char *newline,\n-\t\t\t\t\t size_t count, const char *buf,\n-\t\t\t\t\t const char *line_prefix)\n+\t\t\t\t\t size_t count, const char *buf)\n {\n \tint print = 0;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n \twhile (count) {\n \t\tchar *p = memchr(buf, '\\n', count);\n \t\tif (print)\n-\t\t\tfputs(line_prefix, fp);\n+\t\t\tstrbuf_addstr(&sb, diff_line_prefix(o));\n+\n \t\tif (p != buf) {\n-\t\t\tif (st_el->color && fputs(st_el->color, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (fputs(st_el->prefix, fp) < 0 ||\n-\t\t\t fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||\n-\t\t\t fputs(st_el->suffix, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (st_el->color && *st_el->color\n-\t\t\t && fputs(GIT_COLOR_RESET, fp) < 0)\n-\t\t\t\treturn -1;\n+\t\t\tconst char *reset = st_el->color && *st_el->color ?\n+\t\t\t\t\t GIT_COLOR_RESET : NULL;\n+\t\t\tif (st_el->color && *st_el->color)\n+\t\t\t\tstrbuf_addstr(&sb, st_el->color);\n+\t\t\tstrbuf_addstr(&sb, st_el->prefix);\n+\t\t\tstrbuf_add(&sb, buf, p ? p - buf : count);\n+\t\t\tstrbuf_addstr(&sb, st_el->suffix);\n+\t\t\tif (reset)\n+\t\t\t\tstrbuf_addstr(&sb, reset);\n \t\t}\n \t\tif (!p)\n-\t\t\treturn 0;\n-\t\tif (fputs(newline, fp) < 0)\n-\t\t\treturn -1;\n+\t\t\tgoto out;\n+\n+\t\tstrbuf_addstr(&sb, newline);\n \t\tcount -= p + 1 - buf;\n \t\tbuf = p + 1;\n \t\tprint = 1;\n+\t\tif (count) {\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_reset(&sb);\n+\t\t}\n \t}\n+\n+out:\n+\tif (sb.len)\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\tstrbuf_release(&sb);\n \treturn 0;\n }\n \n@@ -1203,24 +1219,20 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)\n \t\tfputs(line_prefix, diff_words->opt->file);\n \t}\n \tif (diff_words->current_plus != plus_begin) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->ctx, style->newline,\n \t\t\t\tplus_begin - diff_words->current_plus,\n-\t\t\t\tdiff_words->current_plus, line_prefix);\n-\t\tif (*(plus_begin - 1) == '\\n')\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n+\t\t\t\tdiff_words->current_plus);\n \t}\n \tif (minus_begin != minus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->old, style->newline,\n-\t\t\t\tminus_end - minus_begin, minus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tminus_end - minus_begin, minus_begin);\n \t}\n \tif (plus_begin != plus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->new, style->newline,\n-\t\t\t\tplus_end - plus_begin, plus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tplus_end - plus_begin, plus_begin);\n \t}\n \n \tdiff_words->current_plus = plus_end;\n@@ -1314,11 +1326,12 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \n \t/* special case: only removal */\n \tif (!diff_words->plus.text.size) {\n-\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\temit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t line_prefix, strlen(line_prefix), 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->old, style->newline,\n \t\t\tdiff_words->minus.text.size,\n-\t\t\tdiff_words->minus.text.ptr, line_prefix);\n+\t\t\tdiff_words->minus.text.ptr);\n \t\tdiff_words->minus.text.size = 0;\n \t\treturn;\n \t}\n@@ -1341,12 +1354,12 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \tif (diff_words->current_plus != diff_words->plus.text.ptr +\n \t\t\tdiff_words->plus.text.size) {\n \t\tif (color_words_output_graph_prefix(diff_words))\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\t\temit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t\t line_prefix, strlen(line_prefix), 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->ctx, style->newline,\n \t\t\tdiff_words->plus.text.ptr + diff_words->plus.text.size\n-\t\t\t- diff_words->current_plus, diff_words->current_plus,\n-\t\t\tline_prefix);\n+\t\t\t- diff_words->current_plus, diff_words->current_plus);\n \t}\n \tdiff_words->minus.text.size = diff_words->plus.text.size = 0;\n }\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323045","messageId":"20170623012919.28693-18-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 17/25] diff.c: convert show_stats to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:11Z","receivedAt":"2017-06-23T01:31:03Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We call print_stat_summary from builtin/apply, so we still\nneed the version with a file pointer, so introduce\nprint_stat_summary_0 that uses emit_string machinery and\nkeep print_stat_summary with the same arguments around.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 114 +++++++++++++++++++++++++++++++++++++++++------------------------\n diff.h | 4 +--\n 2 files changed, 74 insertions(+), 44 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 7c92675f6f..2108c191a3 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -565,6 +565,10 @@ enum diff_symbol {\n \tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n \tDIFF_SYMBOL_BINARY_DIFF_BODY,\n \tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n+\tDIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n+\tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n+\tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n+\tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -628,6 +632,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n+\tstruct strbuf sb = STRBUF_INIT;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -639,6 +644,8 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n+\tcase DIFF_SYMBOL_STATS_LINE:\n \tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n@@ -747,9 +754,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\tfprintf(o->file, \"%sSubmodule %s contains modified content\\n\",\n \t\t\tdiff_line_prefix(o), line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:\n+\t\temit_line(o, \"\", \"\", \" 0 files changed\\n\",\n+\t\t\t strlen(\" 0 files changed\\n\"));\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n+\t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n+\tstrbuf_release(&sb);\n }\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line)\n@@ -1705,20 +1720,14 @@ static int scale_linear(int it, int width, int max_change)\n \treturn 1 + (it * (width - 1) / max_change);\n }\n \n-static void show_name(FILE *file,\n-\t\t const char *prefix, const char *name, int len)\n-{\n-\tfprintf(file, \" %s%-*s |\", prefix, len, name);\n-}\n-\n-static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)\n+static void show_graph(struct strbuf *out, char ch, int cnt,\n+\t\t const char *set, const char *reset)\n {\n \tif (cnt <= 0)\n \t\treturn;\n-\tfprintf(file, \"%s\", set);\n-\twhile (cnt--)\n-\t\tputc(ch, file);\n-\tfprintf(file, \"%s\", reset);\n+\tstrbuf_addstr(out, set);\n+\tstrbuf_addchars(out, ch, cnt);\n+\tstrbuf_addstr(out, reset);\n }\n \n static void fill_print_name(struct diffstat_file *file)\n@@ -1742,14 +1751,16 @@ static void fill_print_name(struct diffstat_file *file)\n \tfile->print_name = pname;\n }\n \n-int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n+static void print_stat_summary_inserts_deletes(struct diff_options *options,\n+\t\tint files, int insertions, int deletions)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint ret;\n \n \tif (!files) {\n \t\tassert(insertions == 0 && deletions == 0);\n-\t\treturn fprintf(fp, \"%s\\n\", \" 0 files changed\");\n+\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n+\t\t\t\t NULL, 0, 0);\n+\t\treturn;\n \t}\n \n \tstrbuf_addf(&sb,\n@@ -1776,9 +1787,19 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n \t\t\t deletions);\n \t}\n \tstrbuf_addch(&sb, '\\n');\n-\tret = fputs(sb.buf, fp);\n+\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n+\t\t\t sb.buf, sb.len, 0);\n \tstrbuf_release(&sb);\n-\treturn ret;\n+}\n+\n+void print_stat_summary(FILE *fp, int files,\n+\t\t\tint insertions, int deletions)\n+{\n+\tstruct diff_options o;\n+\tmemset(&o, 0, sizeof(o));\n+\to.file = fp;\n+\n+\tprint_stat_summary_inserts_deletes(&o, files, insertions, deletions);\n }\n \n static void show_stats(struct diffstat_t *data, struct diff_options *options)\n@@ -1788,13 +1809,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \tint total_files = data->nr, count;\n \tint width, name_width, graph_width, number_width = 0, bin_width = 0;\n \tconst char *reset, *add_c, *del_c;\n-\tconst char *line_prefix = \"\";\n \tint extra_shown = 0;\n+\tconst char *line_prefix = diff_line_prefix(options);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (data->nr == 0)\n \t\treturn;\n \n-\tline_prefix = diff_line_prefix(options);\n \tcount = options->stat_count ? options->stat_count : data->nr;\n \n \treset = diff_get_color_opt(options, DIFF_RESET);\n@@ -1948,26 +1969,32 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t}\n \n \t\tif (file->is_binary) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" %*s\", number_width, \"Bin\");\n+\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\t\tstrbuf_addf(&out, \" %*s\", number_width, \"Bin\");\n \t\t\tif (!added && !deleted) {\n-\t\t\t\tputc('\\n', options->file);\n+\t\t\t\tstrbuf_addch(&out, '\\n');\n+\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\t\tstrbuf_reset(&out);\n \t\t\t\tcontinue;\n \t\t\t}\n-\t\t\tfprintf(options->file, \" %s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addf(&out, \" %s%\"PRIuMAX\"%s\",\n \t\t\t\tdel_c, deleted, reset);\n-\t\t\tfprintf(options->file, \" -> \");\n-\t\t\tfprintf(options->file, \"%s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addstr(&out, \" -> \");\n+\t\t\tstrbuf_addf(&out, \"%s%\"PRIuMAX\"%s\",\n \t\t\t\tadd_c, added, reset);\n-\t\t\tfprintf(options->file, \" bytes\");\n-\t\t\tfprintf(options->file, \"\\n\");\n+\t\t\tstrbuf_addstr(&out, \" bytes\\n\");\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \t\telse if (file->is_unmerged) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" Unmerged\\n\");\n+\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\t\tstrbuf_addstr(&out, \" Unmerged\\n\");\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \n@@ -1990,14 +2017,16 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t\t\tadd = total - del;\n \t\t\t}\n \t\t}\n-\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\tshow_name(options->file, prefix, name, len);\n-\t\tfprintf(options->file, \" %*\"PRIuMAX\"%s\",\n+\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\tstrbuf_addf(&out, \" %*\"PRIuMAX\"%s\",\n \t\t\tnumber_width, added + deleted,\n \t\t\tadded + deleted ? \" \" : \"\");\n-\t\tshow_graph(options->file, '+', add, add_c, reset);\n-\t\tshow_graph(options->file, '-', del, del_c, reset);\n-\t\tfprintf(options->file, \"\\n\");\n+\t\tshow_graph(&out, '+', add, add_c, reset);\n+\t\tshow_graph(&out, '-', del, del_c, reset);\n+\t\tstrbuf_addch(&out, '\\n');\n+\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t out.buf, out.len, 0);\n+\t\tstrbuf_reset(&out);\n \t}\n \n \tfor (i = 0; i < data->nr; i++) {\n@@ -2018,11 +2047,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\tif (i < count)\n \t\t\tcontinue;\n \t\tif (!extra_shown)\n-\t\t\tfprintf(options->file, \"%s ...\\n\", line_prefix);\n+\t\t\temit_diff_symbol(options,\n+\t\t\t\t\t DIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n+\t\t\t\t\t NULL, 0, 0);\n \t\textra_shown = 1;\n \t}\n-\tfprintf(options->file, \"%s\", line_prefix);\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\n+\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n }\n \n static void show_shortstats(struct diffstat_t *data, struct diff_options *options)\n@@ -2034,7 +2065,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \n \tfor (i = 0; i < data->nr; i++) {\n \t\tint added = data->files[i]->added;\n-\t\tint deleted= data->files[i]->deleted;\n+\t\tint deleted = data->files[i]->deleted;\n \n \t\tif (data->files[i]->is_unmerged ||\n \t\t (!data->files[i]->is_interesting && (added + deleted == 0))) {\n@@ -2044,8 +2075,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \t\t\tdels += deleted;\n \t\t}\n \t}\n-\tfprintf(options->file, \"%s\", diff_line_prefix(options));\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n }\n \n static void show_numstat(struct diffstat_t *data, struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex 2ee0ef3908..b73f7a1e75 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -398,8 +398,8 @@ extern int parse_rename_score(const char **cp_p);\n \n extern long parse_algorithm_value(const char *value);\n \n-extern int print_stat_summary(FILE *fp, int files,\n-\t\t\t int insertions, int deletions);\n+extern void print_stat_summary(FILE *fp, int files,\n+\t\t\t int insertions, int deletions);\n extern void setup_diff_pager(struct diff_options *);\n \n #endif /* DIFF_H */\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323046","messageId":"20170623012919.28693-20-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 19/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:13Z","receivedAt":"2017-06-23T01:31:04Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 10 +++++++---\n 1 file changed, 7 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 60630d2c3f..0ed86ba984 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -570,6 +570,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n \tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_WORD_DIFF,\n+\tDIFF_SYMBOL_STAT_SEP,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -765,6 +766,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_WORD_DIFF:\n \t\tfprintf(o->file, \"%.*s\", len, line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_STAT_SEP:\n+\t\tfputs(o->stat_sep, o->file);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -5082,10 +5086,10 @@ void diff_flush(struct diff_options *options)\n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n \t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);\n-\t\t\tif (options->stat_sep) {\n+\t\t\tif (options->stat_sep)\n \t\t\t\t/* attach patch instead of inline */\n-\t\t\t\tfputs(options->stat_sep, options->file);\n-\t\t\t}\n+\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STAT_SEP,\n+\t\t\t\t\t\t NULL, 0, 0);\n \t\t}\n \n \t\tdiff_flush_patch_all_file_pairs(options);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323047","messageId":"20170623012919.28693-21-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 20/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:14Z","receivedAt":"2017-06-23T01:31:06Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 71 ++++++++++++++++++++++++++++++++++++++----------------------------\n 1 file changed, 41 insertions(+), 30 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 0ed86ba984..f9fb94b0d3 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -571,6 +571,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_WORD_DIFF,\n \tDIFF_SYMBOL_STAT_SEP,\n+\tDIFF_SYMBOL_SUMMARY,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -647,6 +648,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n \tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n+\tcase DIFF_SYMBOL_SUMMARY:\n \tcase DIFF_SYMBOL_STATS_LINE:\n \tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n@@ -4720,67 +4722,76 @@ static void flush_one_pair(struct diff_filepair *p, struct diff_options *opt)\n \t}\n }\n \n-static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)\n+static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tif (fs->mode)\n-\t\tfprintf(file, \" %s mode %06o \", newdelete, fs->mode);\n+\t\tstrbuf_addf(&sb, \" %s mode %06o \", newdelete, fs->mode);\n \telse\n-\t\tfprintf(file, \" %s \", newdelete);\n-\twrite_name_quoted(fs->path, file, '\\n');\n-}\n+\t\tstrbuf_addf(&sb, \" %s \", newdelete);\n \n+\tquote_c_style(fs->path, &sb, NULL, 0);\n+\tstrbuf_addch(&sb, '\\n');\n+\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t sb.buf, sb.len, 0);\n+\tstrbuf_release(&sb);\n+}\n \n-static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,\n-\t\tconst char *line_prefix)\n+static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,\n+\t\tint show_name)\n {\n \tif (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {\n-\t\tfprintf(file, \"%s mode change %06o => %06o%c\", line_prefix, p->one->mode,\n-\t\t\tp->two->mode, show_name ? ' ' : '\\n');\n+\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\tstrbuf_addf(&sb, \" mode change %06o => %06o\",\n+\t\t\t p->one->mode, p->two->mode);\n \t\tif (show_name) {\n-\t\t\twrite_name_quoted(p->two->path, file, '\\n');\n+\t\t\tstrbuf_addch(&sb, ' ');\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n \t\t}\n+\t\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\t\tstrbuf_release(&sb);\n \t}\n }\n \n-static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,\n-\t\t\tconst char *line_prefix)\n+static void show_rename_copy(struct diff_options *opt, const char *renamecopy,\n+\t\tstruct diff_filepair *p)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tchar *names = pprint_rename(p->one->path, p->two->path);\n-\n-\tfprintf(file, \" %s %s (%d%%)\\n\", renamecopy, names, similarity_index(p));\n+\tstrbuf_addf(&sb, \" %s %s (%d%%)\\n\",\n+\t\t\trenamecopy, names, similarity_index(p));\n \tfree(names);\n-\tshow_mode_change(file, p, 0, line_prefix);\n+\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\tshow_mode_change(opt, p, 0);\n }\n \n static void diff_summary(struct diff_options *opt, struct diff_filepair *p)\n {\n-\tFILE *file = opt->file;\n-\tconst char *line_prefix = diff_line_prefix(opt);\n-\n \tswitch(p->status) {\n \tcase DIFF_STATUS_DELETED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"delete\", p->one);\n+\t\tshow_file_mode_name(opt, \"delete\", p->one);\n \t\tbreak;\n \tcase DIFF_STATUS_ADDED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"create\", p->two);\n+\t\tshow_file_mode_name(opt, \"create\", p->two);\n \t\tbreak;\n \tcase DIFF_STATUS_COPIED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"copy\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"copy\", p);\n \t\tbreak;\n \tcase DIFF_STATUS_RENAMED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"rename\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"rename\", p);\n \t\tbreak;\n \tdefault:\n \t\tif (p->score) {\n-\t\t\tfprintf(file, \"%s rewrite \", line_prefix);\n-\t\t\twrite_name_quoted(p->two->path, file, ' ');\n-\t\t\tfprintf(file, \"(%d%%)\\n\", similarity_index(p));\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addstr(&sb, \" rewrite \");\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n+\t\t\tstrbuf_addf(&sb, \" (%d%%)\\n\", similarity_index(p));\n+\t\t\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n \t\t}\n-\t\tshow_mode_change(file, p, !p->score, line_prefix);\n+\t\tshow_mode_change(opt, p, !p->score);\n \t\tbreak;\n \t}\n }\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323048","messageId":"20170623012919.28693-17-sbeller@google.com","threadId":"46038","inReplyTo":"20170623012919.28693-1-sbeller@google.com","subject":"[PATCHv2 16/25] diff.c: convert emit_binary_diff_body to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T01:29:10Z","receivedAt":"2017-06-23T01:31:07Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n diff.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++-----------------\n 1 file changed, 46 insertions(+), 17 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex bc78a216ab..7c92675f6f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,11 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n+\tDIFF_SYMBOL_BINARY_DIFF_BODY,\n+\tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -634,6 +639,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n+\tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -705,6 +711,19 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER:\n+\t\tfprintf(o->file, \"%sGIT binary patch\\n\", diff_line_prefix(o));\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA:\n+\t\tfprintf(o->file, \"%sdelta %s\\n\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:\n+\t\tfprintf(o->file, \"%sliteral %s\\n\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_FOOTER:\n+\t\tfputs(diff_line_prefix(o), o->file);\n+\t\tfputc('\\n', o->file);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_REWRITE_DIFF:\n \t\tfraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -2390,8 +2409,8 @@ static unsigned char *deflate_it(char *data,\n \treturn deflated;\n }\n \n-static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t\t const char *prefix)\n+static void emit_binary_diff_body(struct diff_options *o,\n+\t\t\t\t mmfile_t *one, mmfile_t *two)\n {\n \tvoid *cp;\n \tvoid *delta;\n@@ -2420,13 +2439,18 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t}\n \n \tif (delta && delta_size < deflate_size) {\n-\t\tfprintf(file, \"%sdelta %lu\\n\", prefix, orig_size);\n+\t\tchar *s = xstrfmt(\"%lu\", orig_size);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n+\t\t\t\t s, strlen(s), 0);\n+\t\tfree(s);\n \t\tfree(deflated);\n \t\tdata = delta;\n \t\tdata_size = delta_size;\n-\t}\n-\telse {\n-\t\tfprintf(file, \"%sliteral %lu\\n\", prefix, two->size);\n+\t} else {\n+\t\tchar *s = xstrfmt(\"%lu\", two->size);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n+\t\t\t\t s, strlen(s), 0);\n+\t\tfree(s);\n \t\tfree(delta);\n \t\tdata = deflated;\n \t\tdata_size = deflate_size;\n@@ -2435,8 +2459,9 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t/* emit data encoded in base85 */\n \tcp = data;\n \twhile (data_size) {\n+\t\tint len;\n \t\tint bytes = (52 < data_size) ? 52 : data_size;\n-\t\tchar line[70];\n+\t\tchar line[71];\n \t\tdata_size -= bytes;\n \t\tif (bytes <= 26)\n \t\t\tline[0] = bytes + 'A' - 1;\n@@ -2444,20 +2469,24 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t\t\tline[0] = bytes - 26 + 'a' - 1;\n \t\tencode_85(line + 1, cp, bytes);\n \t\tcp = (char *) cp + bytes;\n-\t\tfprintf(file, \"%s\", prefix);\n-\t\tfputs(line, file);\n-\t\tfputc('\\n', file);\n+\n+\t\tlen = strlen(line);\n+\t\tline[len++] = '\\n';\n+\t\tline[len] = '\\0';\n+\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_BODY,\n+\t\t\t\t line, len, 0);\n \t}\n-\tfprintf(file, \"%s\\n\", prefix);\n+\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_FOOTER, NULL, 0, 0);\n \tfree(data);\n }\n \n-static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t const char *prefix)\n+static void emit_binary_diff(struct diff_options *o,\n+\t\t\t mmfile_t *one, mmfile_t *two)\n {\n-\tfprintf(file, \"%sGIT binary patch\\n\", prefix);\n-\temit_binary_diff_body(file, one, two, prefix);\n-\temit_binary_diff_body(file, two, one, prefix);\n+\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER, NULL, 0, 0);\n+\temit_binary_diff_body(o, one, two);\n+\temit_binary_diff_body(o, two, one);\n }\n \n int diff_filespec_is_binary(struct diff_filespec *one)\n@@ -2643,7 +2672,7 @@ static void builtin_diff(const char *name_a,\n \t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n-\t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n+\t\t\temit_binary_diff(o, &mf1, &mf2);\n \t\telse {\n \t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n \t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n-- \n2.12.2.575.gb14f27f917\n\n"},{"id":"323055","messageId":"xmqq1sqbgv3a.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"CAGZ79kben16T9nUpOjsq5-Sv8q7z+C1Nrq++jn2aO17CzqToHw@mail.gmail.com","subject":"Re: [PATCH 08/26] diff.c: migrate emit_line_checked to use emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-23T04:56:57Z","receivedAt":"2017-06-23T04:57:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Actually that function already has some quick return:\n>\n> static int new_blank_line_at_eof(struct emit_callback *ecbdata, const\n> char *line, int len)\n> {\n> if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n> ecbdata->blank_at_eof_in_preimage &&\n> ecbdata->blank_at_eof_in_postimage &&\n> ecbdata->blank_at_eof_in_preimage <= ecbdata->lno_in_preimage &&\n> ecbdata->blank_at_eof_in_postimage <= ecbdata->lno_in_postimage))\n> return 0;\n> return ws_blank_line(line, len, ecbdata->ws_rule);\n> }\n\nI wouldn't call the first if() statement \"quick return\". It\nprobably weighs as much as ws_blank_line() itself, which is not that\nexpensive. \n\nEven though new_blank_line_at_eof() may have been coded efficiently\nand not very expensive as a whole, the fact remains that it didn't\nget called when we know it was unneeded, and it now will get called\nbefore we figure out if it is necessary.\n\nAnd that is why I said it is a bit unsatisfactory (as opposed to\n\"horrible\" or with other more severe adjective).\n\n\n"},{"id":"323145","messageId":"xmqqa84yeads.fsf@gitster.mtv.corp.google.com","threadId":"46038","inReplyTo":"20170623012919.28693-5-sbeller@google.com","subject":"Re: [PATCHv2 04/25] diff.c: introduce emit_diff_symbol","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-23T20:07:11Z","receivedAt":"2017-06-23T20:07:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> diff --git a/new.txt b/new.txt\n> index fa69b07..412428c 100644\n> Binary files a/new.txt and b/new.txt differ\n>\n> could be buffered as\n> DIFF_SYMBOL_DIFF_START + new.txt\n> DIFF_SYMBOL_INDEX_MODE + fa69b07 412428c \"non-executable\" flag\n> DIFF_SYMBOL_BINARY_FILES + new.txt\n\nHopefully this is an oversimplified example and I'd assume that you\nwill do the right thing when showing a renamed filepair by allowing\nboth old and new paths that could be different?\n\n"},{"id":"323147","messageId":"CAGZ79kZhtS7HOPARLc+MGTkjRZ2+NK-SRDznEJJmVPJBDXkCkA@mail.gmail.com","threadId":"46038","inReplyTo":"xmqqa84yeads.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCHv2 04/25] diff.c: introduce emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-23T20:13:03Z","receivedAt":"2017-06-23T20:13:14Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jun 23, 2017 at 1:07 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> diff --git a/new.txt b/new.txt\n>> index fa69b07..412428c 100644\n>> Binary files a/new.txt and b/new.txt differ\n>>\n>> could be buffered as\n>> DIFF_SYMBOL_DIFF_START + new.txt\n>> DIFF_SYMBOL_INDEX_MODE + fa69b07 412428c \"non-executable\" flag\n>> DIFF_SYMBOL_BINARY_FILES + new.txt\n>\n> Hopefully this is an oversimplified example and I'd assume that you\n> will do the right thing when showing a renamed filepair by allowing\n> both old and new paths that could be different?\n\nYes, this is simplified to drive the point of having as little data as possible,\nso I thought we might have a flag that says \"same file name\" instead of giving\nthe file name twice.\n\nWe do not do such an optimization yet.\n"},{"id":"323158","messageId":"8737aqjs6q.fsf@gmail.com","threadId":"46038","inReplyTo":"CAGZ79kYLdpaXzn0T5547fRr=EPR691n2bPfS+mzaZuS-k39ayw@mail.gmail.com","subject":"Re: [showing-off RFC/PATCH 26/26] diff.c: have a \"machine parseable\" move coloring","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-06-23T21:43:41Z","receivedAt":"2017-06-23T21:43:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jun 20 2017, Stefan Beller jotted:\n\n> + Ævar, who was not part of the email where I copied all recipients\n> from for this series.\n\nI played around with this a bit, it would be great to have something\nlike this on top of --color-moved eventually. It's a lot easier to /^_\nin a pager than looking for color codes.\n\nOf course it's diff-incompatible output, but it's fair enough to have\nthat hidden behind some option IMO.\n\n> On Mon, Jun 19, 2017 at 7:48 PM, Stefan Beller wrote:\n>> Ævar asked for it, this is how you would do it.\n>> (plus documentation, tests, CLI knobs, options)\n>>\n>> Signed-off-by: Stefan Beller \n>> ---\n>> diff.c | 15 +++++++++++----\n>> diff.h | 2 ++\n>> 2 files changed, 13 insertions(+), 4 deletions(-)\n>>\n>> diff --git a/diff.c b/diff.c\n>> index 7756f7610c..61caa057ff 100644\n>> --- a/diff.c\n>> +++ b/diff.c\n>> @@ -997,6 +997,7 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n>> static const char *nneof = \" No newline at end of file\\n\";\n>> const char *context, *reset, *set, *meta, *fraginfo;\n>> struct strbuf sb = STRBUF_INIT;\n>> + int sign;\n>>\n>> enum diff_symbol s = eds->s;\n>> const char *line = eds->line;\n>> @@ -1058,8 +1059,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n>> default:\n>> set = diff_get_color_opt(o, DIFF_FILE_NEW);\n>> }\n>> + sign = '+';\n>> + if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)\n>> + sign = '*';\n>> reset = diff_get_color_opt(o, DIFF_RESET);\n>> - emit_line_ws_markup(o, set, reset, line, len, '+',\n>> + emit_line_ws_markup(o, set, reset, line, len, sign,\n>> flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n>> flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n>> break;\n>> @@ -1086,8 +1090,11 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n>> default:\n>> set = diff_get_color_opt(o, DIFF_FILE_OLD);\n>> }\n>> + sign = '-';\n>> + if (flags & DIFF_SYMBOL_MOVED_LINE && o->machine_readable_moves)\n>> + sign = '_';\n>> reset = diff_get_color_opt(o, DIFF_RESET);\n>> - emit_line_ws_markup(o, set, reset, line, len, '-',\n>> + emit_line_ws_markup(o, set, reset, line, len, sign,\n>> flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n>> break;\n>> case DIFF_SYMBOL_WORDS_PORCELAIN:\n>> @@ -5475,7 +5482,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n>> static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n>> struct diff_queue_struct *q = &diff_queued_diff;\n>>\n>> - if (o->color_moved)\n>> + if (o->color_moved || o->machine_readable_moves)\n>> o->emitted_symbols = &esm;\n>>\n>> for (i = 0; i < q->nr; i++) {\n>> @@ -5485,7 +5492,7 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n>> }\n>>\n>> if (o->emitted_symbols) {\n>> - if (o->color_moved) {\n>> + if (o->color_moved || o->machine_readable_moves) {\n>> struct hashmap add_lines, del_lines;\n>> unsigned ignore_ws = DIFF_XDL_TST(o, IGNORE_WHITESPACE);\n>>\n>> diff --git a/diff.h b/diff.h\n>> index 98abd75521..b6c4d7ab0f 100644\n>> --- a/diff.h\n>> +++ b/diff.h\n>> @@ -194,6 +194,8 @@ struct diff_options {\n>> COLOR_MOVED_ZEBRA = 2,\n>> COLOR_MOVED_ZEBRA_DIM = 3,\n>> } color_moved;\n>> +\n>> + int machine_readable_moves;\n>> };\n>>\n>> void diff_emit_submodule_del(struct diff_options *o, const char *line);\n>> --\n>> 2.12.2.575.gb14f27f917\n>>\n"}]}