{"thread":{"id":"46290","subject":"[PATCH 02/25] diff.c: move line ending check into emit_hunk_header","startedAt":"2017-06-30T00:07:48Z","lastAt":"2017-06-30T21:49:15Z","messageCount":63,"participants":["Stefan Beller","Simon Ruderich","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":25},"messages":[{"id":"323516","messageId":"20170630000710.10601-3-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 02/25] diff.c: move line ending check into emit_hunk_header","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:47Z","receivedAt":"2017-06-30T00:07:48Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The emit_hunk_header() function is responsible for assembling a\nhunk header and calling emit_line() to send the hunk header\nto the output file. Its only caller fn_out_consume() needs\nto prepare for a case where the function emits an incomplete\nline and add the terminating LF.\n\nInstead make sure emit_hunk_header() to always send a\ncompleted line to emit_line().\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2874dfc6fc..94fdb57927 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -678,6 +678,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t}\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n+\tstrbuf_complete_line(&msgbuf);\n+\n \temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n@@ -1315,8 +1317,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\tlen = sane_truncate_line(ecbdata, line, len);\n \t\tfind_lno(line, ecbdata);\n \t\temit_hunk_header(ecbdata, line, len);\n-\t\tif (line[len-1] != '\\n')\n-\t\t\tputc('\\n', o->file);\n \t\treturn;\n \t}\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323517","messageId":"20170630000710.10601-13-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 12/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:57Z","receivedAt":"2017-06-30T00:07:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The header is constructed lazily including line breaks, so just emit\nthe raw string as is.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 28 ++++++++++++++++++++--------\n 1 file changed, 20 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 49b45fef29..78f7c6f82f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n \tDIFF_SYMBOL_FILEPAIR_MINUS,\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n@@ -689,6 +690,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tline, reset,\n \t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n+\tcase DIFF_SYMBOL_HEADER:\n+\t\tfprintf(o->file, \"%s\", line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1385,7 +1389,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \to->found_changes = 1;\n \n \tif (ecbdata->header) {\n-\t\tfprintf(o->file, \"%s\", ecbdata->header->buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t ecbdata->header->buf, ecbdata->header->len, 0);\n \t\tstrbuf_reset(ecbdata->header);\n \t\tecbdata->header = NULL;\n \t}\n@@ -2519,7 +2524,8 @@ static void builtin_diff(const char *name_a,\n \t\tif (complete_rewrite &&\n \t\t (textconv_one || !diff_filespec_is_binary(one)) &&\n \t\t (textconv_two || !diff_filespec_is_binary(two))) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tstrbuf_reset(&header);\n \t\t\temit_rewrite_diff(name_a, name_b, one, two,\n \t\t\t\t\t\ttextconv_one, textconv_two, o);\n@@ -2529,7 +2535,8 @@ static void builtin_diff(const char *name_a,\n \t}\n \n \tif (o->irreversible_delete && lbl[1][0] == '/') {\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf,\n+\t\t\t\t header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tgoto free_ab_and_return;\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n@@ -2540,10 +2547,13 @@ static void builtin_diff(const char *name_a,\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n \t\t\tif (!oidcmp(&one->oid, &two->oid)) {\n \t\t\t\tif (must_show_header)\n-\t\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t\t\t header.buf, header.len,\n+\t\t\t\t\t\t\t 0);\n \t\t\t\tgoto free_ab_and_return;\n \t\t\t}\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n \t\t\t\tline_prefix, lbl[0], lbl[1]);\n \t\t\tgoto free_ab_and_return;\n@@ -2554,10 +2564,11 @@ static void builtin_diff(const char *name_a,\n \t\tif (mf1.size == mf2.size &&\n \t\t !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {\n \t\t\tif (must_show_header)\n-\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n@@ -2575,7 +2586,8 @@ static void builtin_diff(const char *name_a,\n \t\tconst struct userdiff_funcname *pe;\n \n \t\tif (must_show_header) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tstrbuf_reset(&header);\n \t\t}\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323518","messageId":"20170630000710.10601-5-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 04/25] diff.c: introduce emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:49Z","receivedAt":"2017-06-30T00:07:50Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we want to buffer all output before emitting it as a\nnew feature (\"markup moved lines\") conceptually cannot be implemented\nin a single pass over the output.\n\nThere are different approaches to buffer all output such as:\n* Buffering on the char level, i.e. we'd have a char[] which would\n grow at approximately 80 characters a line. This would keep the\n output completely unstructured, but might be very easy to implement,\n such as redirecting all output to a temporary file and working off\n that. The later passes over the buffer are quite complicated though,\n because we have to parse back any output and then decide if it should\n be modified.\n\n* Buffer on a line level. As the output is mostly line oriented already,\n this would make sense, but it still is a bit awkward as we'd have to\n make sense of it again by looking at the first characters of a line\n to decide what part of a diff a line is.\n\n* Buffer semantically. Imagine there is a formal grammar for the diff\n output and we'd keep the symbols of this grammar around. This keeps\n the highest level of structure in the buffered data, such that the\n actual memory requirements are less than say the first option. Instead\n of buffering the characters of the line, we'll buffer what we intend\n to do plus additional information for the specifics. An output of\n\n diff --git a/new.txt b/new.txt\n index fa69b07..412428c 100644\n Binary files a/new.txt and b/new.txt differ\n\n could be buffered as\n DIFF_SYMBOL_DIFF_START + new.txt\n DIFF_SYMBOL_INDEX_MODE + fa69b07 412428c \"non-executable\" flag\n DIFF_SYMBOL_BINARY_FILES + new.txt\n\nThis and the following patches introduce the third option of buffering\nby first moving any output to emit_diff_symbol, and then introducing the\nbuffering in this function.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 22 +++++++++++++++++++---\n 1 file changed, 19 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 77ef56a6e4..4637368d59 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n \temit_line_0(o, set, reset, line[0], line+1, len-1);\n }\n \n+enum diff_symbol {\n+\tDIFF_SYMBOL_SEPARATOR\n+};\n+\n+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n+\t\t\t const char *line, int len)\n+{\n+\tswitch (s) {\n+\tcase DIFF_SYMBOL_SEPARATOR:\n+\t\tfprintf(o->file, \"%s%c\",\n+\t\t\tdiff_line_prefix(o),\n+\t\t\to->line_termination);\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: unknown diff symbol\");\n+\t}\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -4828,9 +4846,7 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\tfprintf(options->file, \"%s%c\",\n-\t\t\t\tdiff_line_prefix(options),\n-\t\t\t\toptions->line_termination);\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323519","messageId":"20170630000710.10601-7-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 06/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:51Z","receivedAt":"2017-06-30T00:07:51Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 75b996c4cf..6e48a129ed 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n@@ -570,6 +571,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n {\n \tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_CONTEXT_MARKER:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -705,8 +709,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n-\n-\temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n+\temit_diff_symbol(ecbdata->opt,\n+\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323520","messageId":"20170630000710.10601-2-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 01/25] diff.c: readability fix","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:46Z","receivedAt":"2017-06-30T00:07:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We already have dereferenced 'p->two' into a local variable 'two'.\nUse that.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 00b4c86698..2874dfc6fc 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3282,8 +3282,8 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \tconst char *other;\n \tconst char *attr_path;\n \n-\tname = p->one->path;\n-\tother = (strcmp(name, p->two->path) ? p->two->path : NULL);\n+\tname = one->path;\n+\tother = (strcmp(name, two->path) ? two->path : NULL);\n \tattr_path = name;\n \tif (o->prefix_length)\n \t\tstrip_prefix(o->prefix_length, &name, &other);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323521","messageId":"20170630000710.10601-6-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 05/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:50Z","receivedAt":"2017-06-30T00:07:54Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 10 +++++++++-\n 1 file changed, 9 insertions(+), 1 deletion(-)\n\ndiff --git a/diff.c b/diff.c\nindex 4637368d59..75b996c4cf 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,13 +561,20 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len)\n {\n+\tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_CONTEXT_MARKER:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n \t\t\tdiff_line_prefix(o),\n@@ -662,7 +669,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tif (len < 10 ||\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n-\t\temit_line(ecbdata->opt, context, reset, line, len);\n+\t\temit_diff_symbol(ecbdata->opt,\n+\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323522","messageId":"20170630000710.10601-4-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 03/25] diff.c: factor out diff_flush_patch_all_file_pairs","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:48Z","receivedAt":"2017-06-30T00:07:55Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we want to do more things before and after all filepairs\nare flushed. So factor flushing out all file pairs into its own function\nthat the new code can be plugged in easily.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 17 ++++++++++++-----\n 1 file changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 94fdb57927..77ef56a6e4 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4734,6 +4734,17 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n \t\twarning(_(rename_limit_advice), varname, needed);\n }\n \n+static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n+{\n+\tint i;\n+\tstruct diff_queue_struct *q = &diff_queued_diff;\n+\tfor (i = 0; i < q->nr; i++) {\n+\t\tstruct diff_filepair *p = q->queue[i];\n+\t\tif (check_pair_status(p))\n+\t\t\tdiff_flush_patch(p, o);\n+\t}\n+}\n+\n void diff_flush(struct diff_options *options)\n {\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n@@ -4826,11 +4837,7 @@ void diff_flush(struct diff_options *options)\n \t\t\t}\n \t\t}\n \n-\t\tfor (i = 0; i < q->nr; i++) {\n-\t\t\tstruct diff_filepair *p = q->queue[i];\n-\t\t\tif (check_pair_status(p))\n-\t\t\t\tdiff_flush_patch(p, options);\n-\t\t}\n+\t\tdiff_flush_patch_all_file_pairs(options);\n \t}\n \n \tif (output_format & DIFF_FORMAT_CALLBACK)\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323523","messageId":"20170630000710.10601-8-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 07/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:52Z","receivedAt":"2017-06-30T00:07:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 19 +++++++++++--------\n 1 file changed, 11 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 6e48a129ed..488096b757 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_NO_LF_EOF,\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n@@ -569,8 +570,16 @@ enum diff_symbol {\n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len)\n {\n+\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_NO_LF_EOF:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tputc('\\n', o->file);\n+\t\temit_line_0(o, context, reset, '\\\\',\n+\t\t\t nneof, strlen(nneof));\n+\t\tbreak;\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -751,7 +760,6 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\t\t int prefix, const char *data, int size)\n {\n \tconst char *endp = NULL;\n-\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);\n \n \twhile (0 < size) {\n@@ -769,13 +777,8 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tsize -= len;\n \t\tdata += len;\n \t}\n-\tif (!endp) {\n-\t\tconst char *context = diff_get_color(ecb->color_diff,\n-\t\t\t\t\t\t DIFF_CONTEXT);\n-\t\tputc('\\n', ecb->opt->file);\n-\t\temit_line_0(ecb->opt, context, reset, '\\\\',\n-\t\t\t nneof, strlen(nneof));\n-\t}\n+\tif (!endp)\n+\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);\n }\n \n static void emit_rewrite_diff(const char *name_a,\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323524","messageId":"20170630000710.10601-19-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 18/25] diff.c: convert word diffing to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:03Z","receivedAt":"2017-06-30T00:07:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The word diffing is not line oriented and would need some serious\neffort to be transformed into a line oriented approach, so\njust go with a symbol DIFF_SYMBOL_WORD_DIFF that is a partial line.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 79 ++++++++++++++++++++++++++++++++++++++----------------------------\n 1 file changed, 46 insertions(+), 33 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 76d4b8ebf9..e0d39d04da 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -570,6 +570,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n \tDIFF_SYMBOL_STATS_LINE,\n+\tDIFF_SYMBOL_WORD_DIFF,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -762,6 +763,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n \t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n \t\tbreak;\n+\tcase DIFF_SYMBOL_WORD_DIFF:\n+\t\tfprintf(o->file, \"%.*s\", len, line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1092,37 +1096,49 @@ struct diff_words_data {\n \tstruct diff_words_style *style;\n };\n \n-static int fn_out_diff_words_write_helper(FILE *fp,\n+static int fn_out_diff_words_write_helper(struct diff_options *o,\n \t\t\t\t\t struct diff_words_style_elem *st_el,\n \t\t\t\t\t const char *newline,\n-\t\t\t\t\t size_t count, const char *buf,\n-\t\t\t\t\t const char *line_prefix)\n+\t\t\t\t\t size_t count, const char *buf)\n {\n \tint print = 0;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n \twhile (count) {\n \t\tchar *p = memchr(buf, '\\n', count);\n \t\tif (print)\n-\t\t\tfputs(line_prefix, fp);\n+\t\t\tstrbuf_addstr(&sb, diff_line_prefix(o));\n+\n \t\tif (p != buf) {\n-\t\t\tif (st_el->color && fputs(st_el->color, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (fputs(st_el->prefix, fp) < 0 ||\n-\t\t\t fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||\n-\t\t\t fputs(st_el->suffix, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (st_el->color && *st_el->color\n-\t\t\t && fputs(GIT_COLOR_RESET, fp) < 0)\n-\t\t\t\treturn -1;\n+\t\t\tconst char *reset = st_el->color && *st_el->color ?\n+\t\t\t\t\t GIT_COLOR_RESET : NULL;\n+\t\t\tif (st_el->color && *st_el->color)\n+\t\t\t\tstrbuf_addstr(&sb, st_el->color);\n+\t\t\tstrbuf_addstr(&sb, st_el->prefix);\n+\t\t\tstrbuf_add(&sb, buf, p ? p - buf : count);\n+\t\t\tstrbuf_addstr(&sb, st_el->suffix);\n+\t\t\tif (reset)\n+\t\t\t\tstrbuf_addstr(&sb, reset);\n \t\t}\n \t\tif (!p)\n-\t\t\treturn 0;\n-\t\tif (fputs(newline, fp) < 0)\n-\t\t\treturn -1;\n+\t\t\tgoto out;\n+\n+\t\tstrbuf_addstr(&sb, newline);\n \t\tcount -= p + 1 - buf;\n \t\tbuf = p + 1;\n \t\tprint = 1;\n+\t\tif (count) {\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_reset(&sb);\n+\t\t}\n \t}\n+\n+out:\n+\tif (sb.len)\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\tstrbuf_release(&sb);\n \treturn 0;\n }\n \n@@ -1204,24 +1220,20 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)\n \t\tfputs(line_prefix, diff_words->opt->file);\n \t}\n \tif (diff_words->current_plus != plus_begin) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->ctx, style->newline,\n \t\t\t\tplus_begin - diff_words->current_plus,\n-\t\t\t\tdiff_words->current_plus, line_prefix);\n-\t\tif (*(plus_begin - 1) == '\\n')\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n+\t\t\t\tdiff_words->current_plus);\n \t}\n \tif (minus_begin != minus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->old, style->newline,\n-\t\t\t\tminus_end - minus_begin, minus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tminus_end - minus_begin, minus_begin);\n \t}\n \tif (plus_begin != plus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->new, style->newline,\n-\t\t\t\tplus_end - plus_begin, plus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tplus_end - plus_begin, plus_begin);\n \t}\n \n \tdiff_words->current_plus = plus_end;\n@@ -1315,11 +1327,12 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \n \t/* special case: only removal */\n \tif (!diff_words->plus.text.size) {\n-\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\temit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t line_prefix, strlen(line_prefix), 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->old, style->newline,\n \t\t\tdiff_words->minus.text.size,\n-\t\t\tdiff_words->minus.text.ptr, line_prefix);\n+\t\t\tdiff_words->minus.text.ptr);\n \t\tdiff_words->minus.text.size = 0;\n \t\treturn;\n \t}\n@@ -1342,12 +1355,12 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \tif (diff_words->current_plus != diff_words->plus.text.ptr +\n \t\t\tdiff_words->plus.text.size) {\n \t\tif (color_words_output_graph_prefix(diff_words))\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\t\temit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t\t line_prefix, strlen(line_prefix), 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->ctx, style->newline,\n \t\t\tdiff_words->plus.text.ptr + diff_words->plus.text.size\n-\t\t\t- diff_words->current_plus, diff_words->current_plus,\n-\t\t\tline_prefix);\n+\t\t\t- diff_words->current_plus, diff_words->current_plus);\n \t}\n \tdiff_words->minus.text.size = diff_words->plus.text.size = 0;\n }\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323525","messageId":"20170630000710.10601-14-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 13/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:58Z","receivedAt":"2017-06-30T00:07:59Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"we could save a little bit of memory when buffering in a later mode\nby just passing the inner part (\"%s and %s\", file1, file 2), but\nthose a just a few bytes, so instead let's reuse the implementation from\nDIFF_SYMBOL_HEADER and keep the whole line around.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 20 +++++++++++++++-----\n 1 file changed, 15 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 78f7c6f82f..1d0f8b509f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n \tDIFF_SYMBOL_FILEPAIR_MINUS,\n@@ -690,6 +691,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tline, reset,\n \t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_FILES:\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n@@ -2542,6 +2544,7 @@ static void builtin_diff(const char *name_a,\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n \t ( (!textconv_one && diff_filespec_is_binary(one)) ||\n \t (!textconv_two && diff_filespec_is_binary(two)) )) {\n+\t\tstruct strbuf sb = STRBUF_INIT;\n \t\tif (!one->data && !two->data &&\n \t\t S_ISREG(one->mode) && S_ISREG(two->mode) &&\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n@@ -2554,8 +2557,11 @@ static void builtin_diff(const char *name_a,\n \t\t\t}\n \t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n \t\t\t\t\t header.buf, header.len, 0);\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n+\t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_release(&sb);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n \t\tif (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)\n@@ -2572,9 +2578,13 @@ static void builtin_diff(const char *name_a,\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n-\t\telse\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\telse {\n+\t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n+\t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_release(&sb);\n+\t\t}\n \t\to->found_changes = 1;\n \t} else {\n \t\t/* Crazy xdl interfaces.. */\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323526","messageId":"20170630000710.10601-22-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 21/25] diff.c: buffer all output if asked to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:06Z","receivedAt":"2017-06-30T00:08:04Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Introduce a new option 'emitted_symbols' in the struct diff_options which\ncontrols whether all output is buffered up until all output is available.\nIt is set internally in diff.c when necessary.\n\nWe'll have a new struct 'emitted_string' in diff.c which will be used to\nbuffer each line. The emitted_string will duplicate the memory of the\nline to buffer as that is easiest to reason about for now. In a future\npatch we may want to decrease the memory usage by not duplicating all\noutput for buffering but rather we may want to store offsets into the\nfile or in case of hunk descriptions such as the similarity score, we\ncould just store the relevant number and reproduce the text later on.\n\nThis approach was chosen as a first step because it is quite simple\ncompared to the alternative with less memory footprint.\n\nemit_diff_symbol factors out the emission part and depending on the\ndiff_options->emitted_symbols the emission will be performed directly\nwhen calling emit_diff_symbol or after the whole process is done, i.e.\nby buffering we have add the possibility for a second pass over the\nwhole output before doing the actual output.\n\nIn 6440d34 (2012-03-14, diff: tweak a _copy_ of diff_options with\nword-diff) we introduced a duplicate diff options struct for word\nemissions as we may have different regex settings in there.\nWhen buffering the output, we need to operate on just one buffer,\nso we have to copy back the emissions of the word buffer into the\nmain buffer.\n\nUnconditionally enable output via buffer in this patch as it yields\na great opportunity for testing, i.e. all the diff tests from the\ntest suite pass without having reordering issues (i.e. only parts\nof the output got buffered, and we forgot to buffer other parts).\nThe test suite passes, which gives confidence that we converted all\nfunctions to use emit_string for output.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--\n diff.h | 2 ++\n 2 files changed, 109 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2db0d7c0f5..31720abf8f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -605,6 +605,47 @@ enum diff_symbol {\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n+/*\n+ * This struct is used when we need to buffer the output of the diff output.\n+ *\n+ * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer\n+ * into the pre/post image file. This pointer could be a union with the\n+ * line pointer. By storing an offset into the file instead of the literal line,\n+ * we can decrease the memory footprint for the buffered output. At first we\n+ * may want to only have indirection for the content lines, but we could also\n+ * enhance the state for emitting prefabricated lines, e.g. the similarity\n+ * score line or hunk/file headers would only need to store a number or path\n+ * and then the output can be constructed later on depending on state.\n+ */\n+struct emitted_diff_symbol {\n+\tconst char *line;\n+\tint len;\n+\tint flags;\n+\tenum diff_symbol s;\n+};\n+#define EMITTED_DIFF_SYMBOL_INIT {NULL}\n+\n+struct emitted_diff_symbols {\n+\tstruct emitted_diff_symbol *buf;\n+\tint nr, alloc;\n+};\n+#define EMITTED_DIFF_SYMBOLS_INIT {NULL, 0, 0}\n+\n+static void append_emitted_diff_symbol(struct diff_options *o,\n+\t\t\t\t struct emitted_diff_symbol *e)\n+{\n+\tstruct emitted_diff_symbol *f;\n+\n+\tALLOC_GROW(o->emitted_symbols->buf,\n+\t\t o->emitted_symbols->nr + 1,\n+\t\t o->emitted_symbols->alloc);\n+\tf = &o->emitted_symbols->buf[o->emitted_symbols->nr++];\n+\n+\tmemcpy(f, e, sizeof(struct emitted_diff_symbol));\n+\tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n+}\n+\n+\n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n \t\t\t\tconst char *line, int len, char sign,\n@@ -631,12 +672,18 @@ static void emit_line_ws_markup(struct diff_options *o,\n \t}\n }\n \n-static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n-\t\t\t const char *line, int len, unsigned flags)\n+static void emit_diff_symbol_from_struct(struct diff_options *o,\n+\t\t\t\t\t struct emitted_diff_symbol *eds)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n \tstruct strbuf sb = STRBUF_INIT;\n+\n+\tenum diff_symbol s = eds->s;\n+\tconst char *line = eds->line;\n+\tint len = eds->len;\n+\tunsigned flags = eds->flags;\n+\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -778,6 +825,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tstrbuf_release(&sb);\n }\n \n+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n+\t\t\t const char *line, int len, unsigned flags)\n+{\n+\tstruct emitted_diff_symbol e = {line, len, flags, s};\n+\n+\tif (o->emitted_symbols)\n+\t\tappend_emitted_diff_symbol(o, &e);\n+\telse\n+\t\temit_diff_symbol_from_struct(o, &e);\n+}\n+\n void diff_emit_submodule_del(struct diff_options *o, const char *line)\n {\n \temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);\n@@ -1374,9 +1432,29 @@ static void diff_words_show(struct diff_words_data *diff_words)\n /* In \"color-words\" mode, show word-diff of words accumulated in the buffer */\n static void diff_words_flush(struct emit_callback *ecbdata)\n {\n+\tstruct diff_options *wo = ecbdata->diff_words->opt;\n+\n \tif (ecbdata->diff_words->minus.text.size ||\n \t ecbdata->diff_words->plus.text.size)\n \t\tdiff_words_show(ecbdata->diff_words);\n+\n+\tif (wo->emitted_symbols) {\n+\t\tstruct diff_options *o = ecbdata->opt;\n+\t\tstruct emitted_diff_symbols *wol = wo->emitted_symbols;\n+\t\tint i;\n+\n+\t\t/*\n+\t\t * NEEDSWORK:\n+\t\t * Instead of appending each, concat all words to a line?\n+\t\t */\n+\t\tfor (i = 0; i < wol->nr; i++)\n+\t\t\tappend_emitted_diff_symbol(o, &wol->buf[i]);\n+\n+\t\tfor (i = 0; i < wol->nr; i++)\n+\t\t\tfree((void *)wol->buf[i].line);\n+\n+\t\twol->nr = 0;\n+\t}\n }\n \n static void diff_filespec_load_driver(struct diff_filespec *one)\n@@ -1412,6 +1490,11 @@ static void init_diff_words_data(struct emit_callback *ecbdata,\n \t\txcalloc(1, sizeof(struct diff_words_data));\n \tecbdata->diff_words->type = o->word_diff;\n \tecbdata->diff_words->opt = o;\n+\n+\tif (orig_opts->emitted_symbols)\n+\t\to->emitted_symbols =\n+\t\t\txcalloc(1, sizeof(struct emitted_diff_symbols));\n+\n \tif (!o->word_regex)\n \t\to->word_regex = userdiff_word_regex(one);\n \tif (!o->word_regex)\n@@ -1446,6 +1529,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)\n {\n \tif (ecbdata->diff_words) {\n \t\tdiff_words_flush(ecbdata);\n+\t\tfree (ecbdata->diff_words->opt->emitted_symbols);\n \t\tfree (ecbdata->diff_words->opt);\n \t\tfree (ecbdata->diff_words->minus.text.ptr);\n \t\tfree (ecbdata->diff_words->minus.orig);\n@@ -4996,16 +5080,37 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n+\tstatic struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n \n \tif (WSEH_NEW & WS_RULE_MASK)\n \t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n \n+\t/*\n+\t * For testing purposes we want to make sure the diff machinery\n+\t * works completely with the buffer. If there is anything emitted\n+\t * outside the emit_string, then the order is screwed\n+\t * up and the tests will fail.\n+\t *\n+\t * TODO (later in this series):\n+\t * We'll unset this pointer in a later patch.\n+\t */\n+\to->emitted_symbols = &esm;\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tif (check_pair_status(p))\n \t\t\tdiff_flush_patch(p, o);\n \t}\n+\n+\tif (o->emitted_symbols) {\n+\t\tfor (i = 0; i < esm.nr; i++)\n+\t\t\temit_diff_symbol_from_struct(o, &esm.buf[i]);\n+\n+\t\tfor (i = 0; i < esm.nr; i++)\n+\t\t\tfree((void *)esm.buf[i].line);\n+\t}\n+\tesm.nr = 0;\n }\n \n void diff_flush(struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex d6094a1eed..4a3b9bde40 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -186,6 +186,8 @@ struct diff_options {\n \tvoid *output_prefix_data;\n \n \tint diff_path_counter;\n+\n+\tstruct emitted_diff_symbols *emitted_symbols;\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323527","messageId":"20170630000710.10601-21-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 20/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:05Z","receivedAt":"2017-06-30T00:08:07Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 71 ++++++++++++++++++++++++++++++++++++++----------------------------\n 1 file changed, 41 insertions(+), 30 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 5a9c55736d..2db0d7c0f5 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -572,6 +572,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_WORD_DIFF,\n \tDIFF_SYMBOL_STAT_SEP,\n+\tDIFF_SYMBOL_SUMMARY,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -648,6 +649,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n \tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n+\tcase DIFF_SYMBOL_SUMMARY:\n \tcase DIFF_SYMBOL_STATS_LINE:\n \tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n@@ -4717,67 +4719,76 @@ static void flush_one_pair(struct diff_filepair *p, struct diff_options *opt)\n \t}\n }\n \n-static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)\n+static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tif (fs->mode)\n-\t\tfprintf(file, \" %s mode %06o \", newdelete, fs->mode);\n+\t\tstrbuf_addf(&sb, \" %s mode %06o \", newdelete, fs->mode);\n \telse\n-\t\tfprintf(file, \" %s \", newdelete);\n-\twrite_name_quoted(fs->path, file, '\\n');\n-}\n+\t\tstrbuf_addf(&sb, \" %s \", newdelete);\n \n+\tquote_c_style(fs->path, &sb, NULL, 0);\n+\tstrbuf_addch(&sb, '\\n');\n+\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t sb.buf, sb.len, 0);\n+\tstrbuf_release(&sb);\n+}\n \n-static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,\n-\t\tconst char *line_prefix)\n+static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,\n+\t\tint show_name)\n {\n \tif (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {\n-\t\tfprintf(file, \"%s mode change %06o => %06o%c\", line_prefix, p->one->mode,\n-\t\t\tp->two->mode, show_name ? ' ' : '\\n');\n+\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\tstrbuf_addf(&sb, \" mode change %06o => %06o\",\n+\t\t\t p->one->mode, p->two->mode);\n \t\tif (show_name) {\n-\t\t\twrite_name_quoted(p->two->path, file, '\\n');\n+\t\t\tstrbuf_addch(&sb, ' ');\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n \t\t}\n+\t\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\t\tstrbuf_release(&sb);\n \t}\n }\n \n-static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,\n-\t\t\tconst char *line_prefix)\n+static void show_rename_copy(struct diff_options *opt, const char *renamecopy,\n+\t\tstruct diff_filepair *p)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tchar *names = pprint_rename(p->one->path, p->two->path);\n-\n-\tfprintf(file, \" %s %s (%d%%)\\n\", renamecopy, names, similarity_index(p));\n+\tstrbuf_addf(&sb, \" %s %s (%d%%)\\n\",\n+\t\t\trenamecopy, names, similarity_index(p));\n \tfree(names);\n-\tshow_mode_change(file, p, 0, line_prefix);\n+\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\tshow_mode_change(opt, p, 0);\n }\n \n static void diff_summary(struct diff_options *opt, struct diff_filepair *p)\n {\n-\tFILE *file = opt->file;\n-\tconst char *line_prefix = diff_line_prefix(opt);\n-\n \tswitch(p->status) {\n \tcase DIFF_STATUS_DELETED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"delete\", p->one);\n+\t\tshow_file_mode_name(opt, \"delete\", p->one);\n \t\tbreak;\n \tcase DIFF_STATUS_ADDED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"create\", p->two);\n+\t\tshow_file_mode_name(opt, \"create\", p->two);\n \t\tbreak;\n \tcase DIFF_STATUS_COPIED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"copy\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"copy\", p);\n \t\tbreak;\n \tcase DIFF_STATUS_RENAMED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"rename\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"rename\", p);\n \t\tbreak;\n \tdefault:\n \t\tif (p->score) {\n-\t\t\tfprintf(file, \"%s rewrite \", line_prefix);\n-\t\t\twrite_name_quoted(p->two->path, file, ' ');\n-\t\t\tfprintf(file, \"(%d%%)\\n\", similarity_index(p));\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addstr(&sb, \" rewrite \");\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n+\t\t\tstrbuf_addf(&sb, \" (%d%%)\\n\", similarity_index(p));\n+\t\t\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n \t\t}\n-\t\tshow_mode_change(file, p, !p->score, line_prefix);\n+\t\tshow_mode_change(opt, p, !p->score);\n \t\tbreak;\n \t}\n }\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323528","messageId":"20170630000710.10601-23-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 22/25] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:07Z","receivedAt":"2017-06-30T00:08:09Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a patch consists mostly of moving blocks of code around, it can\nbe quite tedious to ensure that the blocks are moved verbatim, and not\nundesirably modified in the move. To that end, color blocks that are\nmoved within the same patch differently. For example (OM, del, add,\nand NM are different colors):\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OM] - if (!is_authorized_user())\n [OM] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OM] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NM] + sensitive_stuff(spanning,\n [NM] + multiple,\n [NM] + lines);\n [NM] +}\n\nHowever adjacent blocks may be problematic. For example, in this\npotentially malicious patch, the swapping of blocks can be spotted:\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OMA] - if (!is_authorized_user())\n [OMA] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OMA] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NMA] + sensitive_stuff(spanning,\n [NMA] + multiple,\n [NMA] + lines);\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NMA] +}\n\nIf the moved code is larger, it is easier to hide some permutation in the\ncode, which is why some alternative coloring is needed.\n\nThis patch implements the first mode:\n* basic alternating 'Zebra' mode\n This conveys all information needed to the user. Defer customization to\n later patches.\n\nFirst I implemented an alternative design, which would try to fingerprint\na line by its neighbors to detect if we are in a block or at the boundary.\nThis idea iss error prone as it inspected each line and its neighboring\nlines to determine if the line was (a) moved and (b) if was deep inside\na hunk by having matching neighboring lines. This is unreliable as the\nwe can construct hunks which have equal neighbors that just exceed the\nnumber of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter\nas a line, that is permutated to AXYZCXYZBXYZD..').\n\nInstead this provides a dynamic programming greedy algorithm that finds\nthe largest moved hunk and then has several modes on highlighting bounds.\n\nA note on the options '--submodule=diff' and '--color-words/--word-diff':\nIn the conversion to use emit_line in the prior patches both submodules\nas well as word diff output carefully chose to call emit_line with sign=0.\nAll output with sign=0 is ignored for move detection purposes in this\npatch, such that no weird looking output will be generated for these\ncases. This leads to another thought: We could pass on '--color-moved' to\nsubmodules such that they color up moved lines for themselves. If we'd do\nso only line moves within a repository boundary are marked up.\n\nIt is useful to have moved lines colored, but there are annoying corner\ncases, such as a single line moved, that is very common. For example\nin a typical patch of C code, we have closing braces that end statement\nblocks or functions.\n\nWhile it is technically true that these lines are moved as they show up\nelsewhere, it is harmful for the review as the reviewers attention is\ndrawn to such a minor side annoyance.\n\nFor now let's have a simple solution of hardcoding the number of\nmoved lines to be at least 3 before coloring them. Note, that the\nlength is applied across all blocks to find the 'lonely' blocks\nthat pollute new code, but do not interfere with a permutated\nblock where each permutation has less lines than 3.\n\nHelped-by: Jonathan Tan \nSigned-off-by: Stefan Beller \n---\n diff.c | 341 +++++++++++++++++++++++++++++++++++++++++++--\n diff.h | 12 +-\n t/t4015-diff-whitespace.sh | 261 ++++++++++++++++++++++++++++++++++\n 3 files changed, 599 insertions(+), 15 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 31720abf8f..322a2bfd72 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -16,6 +16,7 @@\n #include \"userdiff.h\"\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n+#include \"hashmap.h\"\n #include \"ll-merge.h\"\n #include \"string-list.h\"\n #include \"argv-array.h\"\n@@ -32,6 +33,7 @@ static int diff_indent_heuristic = 1;\n static int diff_rename_limit_default = 400;\n static int diff_suppress_blank_empty;\n static int diff_use_color_default = -1;\n+static int diff_color_moved_default;\n static int diff_context_default = 3;\n static int diff_interhunk_context_default;\n static const char *diff_word_regex_cfg;\n@@ -56,6 +58,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n+\tGIT_COLOR_MAGENTA,\t/* OLD_MOVED */\n+\tGIT_COLOR_BLUE,\t\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_CYAN,\t\t/* NEW_MOVED */\n+\tGIT_COLOR_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -81,6 +87,14 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_WHITESPACE;\n \tif (!strcasecmp(var, \"func\"))\n \t\treturn DIFF_FUNCINFO;\n+\tif (!strcasecmp(var, \"oldmoved\"))\n+\t\treturn DIFF_FILE_OLD_MOVED;\n+\tif (!strcasecmp(var, \"oldmovedalternative\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoved\"))\n+\t\treturn DIFF_FILE_NEW_MOVED;\n+\tif (!strcasecmp(var, \"newmovedalternative\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT;\n \treturn -1;\n }\n \n@@ -229,12 +243,40 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)\n \treturn 0;\n }\n \n+static int parse_color_moved(const char *arg)\n+{\n+\tswitch (git_parse_maybe_bool(arg)) {\n+\tcase 0:\n+\t\treturn COLOR_MOVED_NO;\n+\tcase 1:\n+\t\treturn COLOR_MOVED_DEFAULT;\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\tif (!strcmp(arg, \"no\"))\n+\t\treturn COLOR_MOVED_NO;\n+\telse if (!strcmp(arg, \"zebra\"))\n+\t\treturn COLOR_MOVED_ZEBRA;\n+\telse if (!strcmp(arg, \"default\"))\n+\t\treturn COLOR_MOVED_DEFAULT;\n+\telse\n+\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra'\"));\n+}\n+\n int git_diff_ui_config(const char *var, const char *value, void *cb)\n {\n \tif (!strcmp(var, \"diff.color\") || !strcmp(var, \"color.diff\")) {\n \t\tdiff_use_color_default = git_config_colorbool(var, value);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"diff.colormoved\")) {\n+\t\tint cm = parse_color_moved(value);\n+\t\tif (cm < 0)\n+\t\t\treturn -1;\n+\t\tdiff_color_moved_default = cm;\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"diff.context\")) {\n \t\tdiff_context_default = git_config_int(var, value);\n \t\tif (diff_context_default < 0)\n@@ -602,7 +644,9 @@ enum diff_symbol {\n * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT\n * 16 is marking if the line is blank at EOF\n */\n-#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n+#define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n+#define DIFF_SYMBOL_MOVED_LINE_ALT\t\t(1<<18)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n /*\n@@ -645,6 +689,240 @@ static void append_emitted_diff_symbol(struct diff_options *o,\n \tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n }\n \n+struct moved_entry {\n+\tstruct hashmap_entry ent;\n+\tconst struct emitted_diff_symbol *es;\n+\tstruct moved_entry *next_line;\n+};\n+\n+static int next_byte(const char **cp, const char **endp,\n+\t\t const struct diff_options *diffopt)\n+{\n+\tint retval;\n+\n+\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_AT_EOL)) {\n+\t\twhile (*endp > *cp && isspace(**endp))\n+\t\t\t(*endp)--;\n+\t}\n+\n+\tif (*cp > *endp)\n+\t\treturn -1;\n+\n+\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_CHANGE)) {\n+\t\twhile (*cp < *endp && isspace(**cp))\n+\t\t\t(*cp)++;\n+\t\t/*\n+\t\t * After skipping a couple of whitespaces, we still have to\n+\t\t * account for one space.\n+\t\t */\n+\t\treturn (int)' ';\n+\t}\n+\n+\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE)) {\n+\t\twhile (*cp < *endp && isspace(**cp))\n+\t\t\t(*cp)++;\n+\t\t/* return the first non-ws character via the usual below */\n+\t}\n+\n+\tretval = **cp;\n+\t(*cp)++; /* advance */\n+\treturn retval;\n+}\n+\n+static int moved_entry_cmp(const struct moved_entry *a,\n+\t\t\t const struct moved_entry *b,\n+\t\t\t const void *keydata,\n+\t\t\t const void *data)\n+{\n+\tconst struct diff_options *diffopt = data;\n+\tconst char *ap = a->es->line, *ae = a->es->line + a->es->len;\n+\tconst char *bp = b->es->line, *be = b->es->line + b->es->len;\n+\n+\tif (!(diffopt->xdl_opts & XDF_WHITESPACE_FLAGS))\n+\t\treturn a->es->len != b->es->len || memcmp(ap, bp, a->es->len);\n+\n+\twhile (1) {\n+\t\tint ca, cb;\n+\t\tca = next_byte(&ap, &ae, diffopt);\n+\t\tcb = next_byte(&bp, &be, diffopt);\n+\t\tif (ca != cb)\n+\t\t\treturn 1; /* differs */\n+\t\tif (!ca)\n+\t\t\treturn 0;\n+\t};\n+}\n+\n+static unsigned get_string_hash(struct emitted_diff_symbol *es, struct diff_options *o)\n+{\n+\tif (o->xdl_opts & XDF_WHITESPACE_FLAGS) {\n+\t\tstatic struct strbuf sb = STRBUF_INIT;\n+\t\tconst char *ap = es->line, *ae = es->line + es->len;\n+\t\tint c;\n+\n+\t\tstrbuf_reset(&sb);\n+\t\twhile ((c = next_byte(&ap, &ae, o)) > 0)\n+\t\t\tstrbuf_addch(&sb, c);\n+\n+\t\treturn memhash(sb.buf, sb.len);\n+\t} else {\n+\t\treturn memhash(es->line, es->len);\n+\t}\n+}\n+\n+static struct moved_entry *prepare_entry(struct diff_options *o,\n+\t\t\t\t\t int line_no)\n+{\n+\tstruct moved_entry *ret = xmalloc(sizeof(*ret));\n+\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];\n+\n+\tret->ent.hash = get_string_hash(l, o);\n+\tret->es = l;\n+\tret->next_line = NULL;\n+\n+\treturn ret;\n+}\n+\n+static void add_lines_to_move_detection(struct diff_options *o,\n+\t\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry *prev_line = NULL;\n+\n+\tint n;\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct hashmap *hm;\n+\t\tstruct moved_entry *key;\n+\n+\t\tswitch (o->emitted_symbols->buf[n].s) {\n+\t\tcase DIFF_SYMBOL_PLUS:\n+\t\t\thm = add_lines;\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MINUS:\n+\t\t\thm = del_lines;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tkey = prepare_entry(o, n);\n+\t\tif (prev_line && prev_line->es->s == o->emitted_symbols->buf[n].s)\n+\t\t\tprev_line->next_line = key;\n+\n+\t\thashmap_add(hm, key);\n+\t\tprev_line = key;\n+\t}\n+}\n+\n+static int shrink_potential_moved_blocks(struct moved_entry **pmb,\n+\t\t\t\t\t int pmb_nr)\n+{\n+\tint lp, rp;\n+\n+\t/* Shrink the set of potential block to the remaining running */\n+\tfor (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n+\t\twhile (lp < pmb_nr && pmb[lp])\n+\t\t\tlp++;\n+\t\t/* lp points at the first NULL now */\n+\n+\t\twhile (rp > -1 && !pmb[rp])\n+\t\t\trp--;\n+\t\t/* rp points at the last non-NULL */\n+\n+\t\tif (lp < pmb_nr && rp > -1 && lp < rp) {\n+\t\t\tpmb[lp] = pmb[rp];\n+\t\t\tpmb[rp] = NULL;\n+\t\t\trp--;\n+\t\t\tlp++;\n+\t\t}\n+\t}\n+\n+\t/* Remember the number of running sets */\n+\treturn rp + 1;\n+}\n+\n+/* Find blocks of moved code, delegate actual coloring decision to helper */\n+static void mark_color_as_moved(struct diff_options *o,\n+\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry **pmb = NULL; /* potentially moved blocks */\n+\tint pmb_nr = 0, pmb_alloc = 0;\n+\tint n, flipped_block = 1, block_length = 0;\n+\n+\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct hashmap *hm = NULL;\n+\t\tstruct moved_entry *key;\n+\t\tstruct moved_entry *match = NULL;\n+\t\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];\n+\t\tint i;\n+\n+\t\tswitch (l->s) {\n+\t\tcase DIFF_SYMBOL_PLUS:\n+\t\t\thm = del_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MINUS:\n+\t\t\thm = add_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tflipped_block = 1;\n+\t\t}\n+\n+\t\tif (!match) {\n+\t\t\tif (block_length < COLOR_MOVED_MIN_BLOCK_LENGTH) {\n+\t\t\t\tfor (i = 0; i < block_length + 1; i++) {\n+\t\t\t\t\tl = &o->emitted_symbols->buf[n - i];\n+\t\t\t\t\tl->flags &= ~DIFF_SYMBOL_MOVED_LINE;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tpmb_nr = 0;\n+\t\t\tblock_length = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE;\n+\t\tblock_length++;\n+\n+\t\t/* Check any potential block runs, advance each or nullify */\n+\t\tfor (i = 0; i < pmb_nr; i++) {\n+\t\t\tstruct moved_entry *p = pmb[i];\n+\t\t\tstruct moved_entry *pnext = (p && p->next_line) ?\n+\t\t\t\t\tp->next_line : NULL;\n+\t\t\tif (pnext && !hm->cmpfn(pnext, match, NULL, o)) {\n+\t\t\t\tpmb[i] = p->next_line;\n+\t\t\t} else {\n+\t\t\t\tpmb[i] = NULL;\n+\t\t\t}\n+\t\t}\n+\n+\t\tpmb_nr = shrink_potential_moved_blocks(pmb, pmb_nr);\n+\n+\t\tif (pmb_nr == 0) {\n+\t\t\t/*\n+\t\t\t * The current line is the start of a new block.\n+\t\t\t * Setup the set of potential blocks.\n+\t\t\t */\n+\t\t\tfor (; match; match = hashmap_get_next(hm, match)) {\n+\t\t\t\tALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n+\t\t\t\tpmb[pmb_nr++] = match;\n+\t\t\t}\n+\n+\t\t\tflipped_block = (flipped_block + 1) % 2;\n+\t\t}\n+\n+\t\tif (flipped_block)\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;\n+\t}\n+\n+\tfree(pmb);\n+}\n \n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n@@ -720,14 +998,24 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n \t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n-\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n+\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);\n+\t\telse\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '+',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n-\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n+\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);\n+\t\telse\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n@@ -3740,6 +4028,8 @@ void diff_setup(struct diff_options *options)\n \t\toptions->a_prefix = \"a/\";\n \t\toptions->b_prefix = \"b/\";\n \t}\n+\n+\toptions->color_moved = diff_color_moved_default;\n }\n \n void diff_setup_done(struct diff_options *options)\n@@ -3849,6 +4139,9 @@ void diff_setup_done(struct diff_options *options)\n \n \tif (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)\n \t\tdie(_(\"--follow requires exactly one pathspec\"));\n+\n+\tif (!options->use_color || external_diff())\n+\t\toptions->color_moved = 0;\n }\n \n static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)\n@@ -4273,7 +4566,19 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n-\telse if (!strcmp(arg, \"--color-words\")) {\n+\telse if (!strcmp(arg, \"--color-moved\")) {\n+\t\tif (diff_color_moved_default)\n+\t\t\toptions->color_moved = diff_color_moved_default;\n+\t\tif (options->color_moved == COLOR_MOVED_NO)\n+\t\t\toptions->color_moved = COLOR_MOVED_DEFAULT;\n+\t} else if (!strcmp(arg, \"--no-color-moved\"))\n+\t\toptions->color_moved = COLOR_MOVED_NO;\n+\telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n+\t\tint cm = parse_color_moved(arg);\n+\t\tif (cm < 0)\n+\t\t\tdie(\"bad --color-moved argument: %s\", arg);\n+\t\toptions->color_moved = cm;\n+\t} else if (!strcmp(arg, \"--color-words\")) {\n \t\toptions->use_color = 1;\n \t\toptions->word_diff = DIFF_WORDS_COLOR;\n \t}\n@@ -5086,16 +5391,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \tif (WSEH_NEW & WS_RULE_MASK)\n \t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n \n-\t/*\n-\t * For testing purposes we want to make sure the diff machinery\n-\t * works completely with the buffer. If there is anything emitted\n-\t * outside the emit_string, then the order is screwed\n-\t * up and the tests will fail.\n-\t *\n-\t * TODO (later in this series):\n-\t * We'll unset this pointer in a later patch.\n-\t */\n-\to->emitted_symbols = &esm;\n+\tif (o->color_moved)\n+\t\to->emitted_symbols = &esm;\n \n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n@@ -5104,6 +5401,21 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \t}\n \n \tif (o->emitted_symbols) {\n+\t\tif (o->color_moved) {\n+\t\t\tstruct hashmap add_lines, del_lines;\n+\n+\t\t\thashmap_init(&del_lines,\n+\t\t\t\t (hashmap_cmp_fn)moved_entry_cmp, o, 0);\n+\t\t\thashmap_init(&add_lines,\n+\t\t\t\t (hashmap_cmp_fn)moved_entry_cmp, o, 0);\n+\n+\t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n+\t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\n+\t\t\thashmap_free(&add_lines, 0);\n+\t\t\thashmap_free(&del_lines, 0);\n+\t\t}\n+\n \t\tfor (i = 0; i < esm.nr; i++)\n \t\t\temit_diff_symbol_from_struct(o, &esm.buf[i]);\n \n@@ -5185,6 +5497,7 @@ void diff_flush(struct diff_options *options)\n \t\t\tfclose(options->file);\n \t\toptions->file = xfopen(\"/dev/null\", \"w\");\n \t\toptions->close_file = 1;\n+\t\toptions->color_moved = 0;\n \t\tfor (i = 0; i < q->nr; i++) {\n \t\t\tstruct diff_filepair *p = q->queue[i];\n \t\t\tif (check_pair_status(p))\ndiff --git a/diff.h b/diff.h\nindex 4a3b9bde40..3196802673 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,12 @@ struct diff_options {\n \tint diff_path_counter;\n \n \tstruct emitted_diff_symbols *emitted_symbols;\n+\tenum {\n+\t\tCOLOR_MOVED_NO = 0,\n+\t\tCOLOR_MOVED_ZEBRA = 2,\n+\t} color_moved;\n+\t#define COLOR_MOVED_DEFAULT COLOR_MOVED_ZEBRA\n+\t#define COLOR_MOVED_MIN_BLOCK_LENGTH 3\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n@@ -208,7 +214,11 @@ enum color_diff {\n \tDIFF_FILE_NEW = 5,\n \tDIFF_COMMIT = 6,\n \tDIFF_WHITESPACE = 7,\n-\tDIFF_FUNCINFO = 8\n+\tDIFF_FUNCINFO = 8,\n+\tDIFF_FILE_OLD_MOVED = 9,\n+\tDIFF_FILE_OLD_MOVED_ALT = 10,\n+\tDIFF_FILE_NEW_MOVED = 11,\n+\tDIFF_FILE_NEW_MOVED_ALT = 12\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 289806d0c7..29704ae14e 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -972,4 +972,265 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '\n \n '\n \n+test_expect_success 'detect moved code, complete file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >test.c &&\n+\t#include\n+\tmain()\n+\t{\n+\tprintf(\"Hello World\");\n+\t}\n+\tEOF\n+\tgit add test.c &&\n+\tgit commit -m \"add main function\" &&\n+\tgit mv test.c main.c &&\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tdiff --git a/main.c b/main.c\n+\tnew file mode 100644\n+\tindex 0000000..a986c57\n+\t--- /dev/null\n+\t+++ b/main.c\n+\t@@ -0,0 +1,5 @@\n+\t+#include\n+\t+main()\n+\t+{\n+\t+printf(\"Hello World\");\n+\t+}\n+\tdiff --git a/test.c b/test.c\n+\tdeleted file mode 100644\n+\tindex a986c57..0000000\n+\t--- a/test.c\n+\t+++ /dev/null\n+\t@@ -1,5 +0,0 @@\n+\t-#include\n+\t-main()\n+\t-{\n+\t-printf(\"Hello World\");\n+\t-}\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect malicious moved code, inside file' '\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit add main.c test.c &&\n+\tgit commit -m \"add main and test file\" &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tfoo(u);\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit diff HEAD --no-renames --color-moved=zebra| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'no effect from --color-moved with --word-diff' '\n+\tcat <<-\\EOF >text.txt &&\n+\tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n+\tEOF\n+\tgit add text.txt &&\n+\tgit commit -a -m \"clean state\" &&\n+\tcat <<-\\EOF >text.txt &&\n+\tsimply Lorem Ipsum dummy is text of the typesetting and printing industry.\n+\tEOF\n+\tgit diff --color-moved --word-diff >actual &&\n+\tgit diff --word-diff >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'move detection ignoring whitespace ' '\n+\tgit reset --hard &&\n+\tcat <<\\EOF >lines.txt &&\n+line 1\n+line 2\n+line 3\n+line 4\n+line 5\n+line 6\n+line 7\n+EOF\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<\\EOF >lines.txt &&\n+\tline 5\n+\tline 6\n+\tline 7\n+line 1\n+line 2\n+line 3\n+line 4\n+EOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 734156d..eb89ead 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,7 +1,7 @@\n+\t+\tline 5\n+\t+\tline 6\n+\t+\tline 7\n+\t line 1\n+\t line 2\n+\t line 3\n+\t line 4\n+\t-line 5\n+\t-line 6\n+\t-line 7\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\tgit diff HEAD --no-renames -w --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 734156d..eb89ead 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,7 +1,7 @@\n+\t+\tline 5\n+\t+\tline 6\n+\t+\tline 7\n+\t line 1\n+\t line 2\n+\t line 3\n+\t line 4\n+\t-line 5\n+\t-line 6\n+\t-line 7\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'move detection with submodules' '\n+\ttest_create_repo bananas &&\n+\techo ripe >bananas/recipe &&\n+\tgit -C bananas add recipe &&\n+\ttest_commit fruit &&\n+\ttest_commit -C bananas recipe &&\n+\tgit submodule add ./bananas &&\n+\tgit add bananas &&\n+\tgit commit -a -m \"bananas are like a heavy library?\" &&\n+\techo foul >bananas/recipe &&\n+\techo ripe >fruit.t &&\n+\n+\tgit diff --submodule=diff --color-moved >actual &&\n+\n+\t# no move detection as the moved line is across repository boundaries.\n+\ttest_decode_color decoded_actual &&\n+\t! grep BGREEN decoded_actual &&\n+\t! grep BRED decoded_actual &&\n+\n+\t# nor did we mess with it another way\n+\tgit diff --submodule=diff | test_decode_color >expect &&\n+\ttest_cmp expect decoded_actual\n+'\n+\n test_done\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323529","messageId":"20170630000710.10601-25-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 24/25] diff.c: add dimming to moved line detection","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:09Z","receivedAt":"2017-06-30T00:08:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Any lines inside a moved block of code are not interesting. Boundaries\nof blocks are only interesting if they are next to another block of moved\ncode.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n color.h | 2 +\n diff.c | 132 +++++++++++++++++++++++++++++++++++++++++----\n diff.h | 9 +++-\n t/t4015-diff-whitespace.sh | 124 ++++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 254 insertions(+), 13 deletions(-)\n\ndiff --git a/color.h b/color.h\nindex 90627650fc..fd2b688dfb 100644\n--- a/color.h\n+++ b/color.h\n@@ -42,6 +42,8 @@ struct strbuf;\n #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n+#define GIT_COLOR_FAINT\t\t\"\\033[2m\"\n+#define GIT_COLOR_FAINT_ITALIC\t\"\\033[2;3m\"\n \n /* A special value meaning \"no color selected\" */\n #define GIT_COLOR_NIL \"NIL\"\ndiff --git a/diff.c b/diff.c\nindex 41ee81d1d8..6b79e6a54f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -58,10 +58,14 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n-\tGIT_COLOR_MAGENTA,\t/* OLD_MOVED */\n-\tGIT_COLOR_BLUE,\t\t/* OLD_MOVED ALTERNATIVE */\n-\tGIT_COLOR_CYAN,\t\t/* NEW_MOVED */\n-\tGIT_COLOR_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n+\tGIT_COLOR_BOLD_MAGENTA,\t/* OLD_MOVED */\n+\tGIT_COLOR_BOLD_BLUE,\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_FAINT,\t/* OLD_MOVED_DIM */\n+\tGIT_COLOR_FAINT_ITALIC,\t/* OLD_MOVED_ALTERNATIVE_DIM */\n+\tGIT_COLOR_BOLD_CYAN,\t/* NEW_MOVED */\n+\tGIT_COLOR_BOLD_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n+\tGIT_COLOR_FAINT,\t/* NEW_MOVED_DIM */\n+\tGIT_COLOR_FAINT_ITALIC,\t/* NEW_MOVED_ALTERNATIVE_DIM */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -91,10 +95,18 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_FILE_OLD_MOVED;\n \tif (!strcasecmp(var, \"oldmovedalternative\"))\n \t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"oldmoveddimmed\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_DIM;\n+\tif (!strcasecmp(var, \"oldmovedalternativedimmed\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT_DIM;\n \tif (!strcasecmp(var, \"newmoved\"))\n \t\treturn DIFF_FILE_NEW_MOVED;\n \tif (!strcasecmp(var, \"newmovedalternative\"))\n \t\treturn DIFF_FILE_NEW_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoveddimmed\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_DIM;\n+\tif (!strcasecmp(var, \"newmovedalternativedimmed\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT_DIM;\n \treturn -1;\n }\n \n@@ -262,8 +274,10 @@ static int parse_color_moved(const char *arg)\n \t\treturn COLOR_MOVED_ZEBRA;\n \telse if (!strcmp(arg, \"default\"))\n \t\treturn COLOR_MOVED_DEFAULT;\n+\telse if (!strcmp(arg, \"dimmed_zebra\"))\n+\t\treturn COLOR_MOVED_ZEBRA_DIM;\n \telse\n-\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra', 'plain'\"));\n+\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra', 'dimmed_zebra', 'plain'\"));\n }\n \n int git_diff_ui_config(const char *var, const char *value, void *cb)\n@@ -649,6 +663,7 @@ enum diff_symbol {\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n #define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n #define DIFF_SYMBOL_MOVED_LINE_ALT\t\t(1<<18)\n+#define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING\t(1<<19)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n /*\n@@ -930,6 +945,67 @@ static void mark_color_as_moved(struct diff_options *o,\n \tfree(pmb);\n }\n \n+#define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \\\n+ (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ALT)\n+static void dim_moved_lines(struct diff_options *o)\n+{\n+\tint n;\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct emitted_diff_symbol *prev = (n != 0) ?\n+\t\t\t\t&o->emitted_symbols->buf[n - 1] : NULL;\n+\t\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];\n+\t\tstruct emitted_diff_symbol *next =\n+\t\t\t\t(n < o->emitted_symbols->nr - 1) ?\n+\t\t\t\t&o->emitted_symbols->buf[n + 1] : NULL;\n+\n+\t\t/* Not a plus or minus line? */\n+\t\tif (l->s != DIFF_SYMBOL_PLUS && l->s != DIFF_SYMBOL_MINUS)\n+\t\t\tcontinue;\n+\n+\t\t/* Not a moved line? */\n+\t\tif (!(l->flags & DIFF_SYMBOL_MOVED_LINE))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * If prev or next are not a plus or minus line,\n+\t\t * pretend they don't exist\n+\t\t */\n+\t\tif (prev && prev->s != DIFF_SYMBOL_PLUS &&\n+\t\t\t prev->s != DIFF_SYMBOL_MINUS)\n+\t\t\tprev = NULL;\n+\t\tif (next && next->s != DIFF_SYMBOL_PLUS &&\n+\t\t\t next->s != DIFF_SYMBOL_MINUS)\n+\t\t\tnext = NULL;\n+\n+\t\t/* Inside a block? */\n+\t\tif ((prev &&\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK)) &&\n+\t\t (next &&\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK))) {\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Check if we are at an interesting bound: */\n+\t\tif (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))\n+\t\t\tcontinue;\n+\t\tif (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * The boundary to prev and next are not interesting,\n+\t\t * so this line is not interesting as a whole\n+\t\t */\n+\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;\n+\t}\n+}\n+\n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n \t\t\t\tconst char *line, int len, char sign,\n@@ -1004,24 +1080,56 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n \t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n-\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n-\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);\n-\t\telse\n+\t\t\tbreak;\n+\t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\t}\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '+',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n-\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n-\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);\n-\t\telse\n+\t\t\tbreak;\n+\t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\t}\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n@@ -5417,6 +5525,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \n \t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n \t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\t\t\tif (o->color_moved == COLOR_MOVED_ZEBRA_DIM)\n+\t\t\t\tdim_moved_lines(o);\n \n \t\t\thashmap_free(&add_lines, 0);\n \t\t\thashmap_free(&del_lines, 0);\ndiff --git a/diff.h b/diff.h\nindex 4cfd609c54..5755f465de 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -192,6 +192,7 @@ struct diff_options {\n \t\tCOLOR_MOVED_NO = 0,\n \t\tCOLOR_MOVED_PLAIN = 1,\n \t\tCOLOR_MOVED_ZEBRA = 2,\n+\t\tCOLOR_MOVED_ZEBRA_DIM = 3,\n \t} color_moved;\n \t#define COLOR_MOVED_DEFAULT COLOR_MOVED_ZEBRA\n \t#define COLOR_MOVED_MIN_BLOCK_LENGTH 3\n@@ -218,8 +219,12 @@ enum color_diff {\n \tDIFF_FUNCINFO = 8,\n \tDIFF_FILE_OLD_MOVED = 9,\n \tDIFF_FILE_OLD_MOVED_ALT = 10,\n-\tDIFF_FILE_NEW_MOVED = 11,\n-\tDIFF_FILE_NEW_MOVED_ALT = 12\n+\tDIFF_FILE_OLD_MOVED_DIM = 11,\n+\tDIFF_FILE_OLD_MOVED_ALT_DIM = 12,\n+\tDIFF_FILE_NEW_MOVED = 13,\n+\tDIFF_FILE_NEW_MOVED_ALT = 14,\n+\tDIFF_FILE_NEW_MOVED_DIM = 15,\n+\tDIFF_FILE_NEW_MOVED_ALT_DIM = 16\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex d1d7b99990..c3b697411a 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -1179,6 +1179,130 @@ test_expect_success 'plain moved code, inside file' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'detect permutations inside moved code -- dimmed_zebra' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\tEOF\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\tEOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\ttest_config color.diff.oldMovedDimmed \"normal magenta\" &&\n+\ttest_config color.diff.newMovedDimmed \"normal cyan\" &&\n+\ttest_config color.diff.oldMovedAlternativeDimmed \"normal blue\" &&\n+\ttest_config color.diff.newMovedAlternativeDimmed \"normal yellow\" &&\n+\tgit diff HEAD --no-renames --color-moved=dimmed_zebra| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'cmd option assumes configured colored-moved' '\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\ttest_config color.diff.oldMovedDimmed \"normal magenta\" &&\n+\ttest_config color.diff.newMovedDimmed \"normal cyan\" &&\n+\ttest_config color.diff.oldMovedAlternativeDimmed \"normal blue\" &&\n+\ttest_config color.diff.newMovedAlternativeDimmed \"normal yellow\" &&\n+\ttest_config diff.colorMoved zebra &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'no effect from --color-moved with --word-diff' '\n \tcat <<-\\EOF >text.txt &&\n \tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323530","messageId":"20170630000710.10601-26-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 25/25] diff: document the new --color-moved setting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:10Z","receivedAt":"2017-06-30T00:08:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n Documentation/config.txt | 15 +++++++++++++--\n Documentation/diff-options.txt | 34 ++++++++++++++++++++++++++++++++++\n 2 files changed, 47 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 06898a7498..74382e5ff5 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1077,14 +1077,25 @@ This does not affect linkgit:git-format-patch[1] or the\n 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n command line with the `--color[=]` option.\n \n+diff.colorMoved::\n+\tIf set to either a valid `` or a true value, moved lines\n+\tin a diff are colored differently, for details of valid modes\n+\tsee '--color-moved' in linkgit:git-diff[1]. If simply set to\n+\ttrue the default color mode will be used. When set to false,\n+\tmoved lines are not colored.\n+\n color.diff.::\n \tUse customized color for diff colorization. `` specifies\n \twhich part of the patch to use the specified color, and is one\n \tof `context` (context text - `plain` is a historical synonym),\n \t`meta` (metainformation), `frag`\n \t(hunk header), 'func' (function in hunk header), `old` (removed lines),\n-\t`new` (added lines), `commit` (commit headers), or `whitespace`\n-\t(highlighting whitespace errors).\n+\t`new` (added lines), `commit` (commit headers), `whitespace`\n+\t(highlighting whitespace errors), `oldMoved` (deleted lines),\n+\t`newMoved` (added lines), `oldMovedDimmed`, `oldMovedAlternative`,\n+\t`oldMovedAlternativeDimmed`, `newMovedDimmed`, `newMovedAlternative`\n+\tand `newMovedAlternativeDimmed` (See the ''\n+\tsetting of '--color-moved' in linkgit:git-diff[1] for details).\n \n color.decorate.::\n \tUse customized color for 'git log --decorate' output. `` is one\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex 89cc0f48de..bb257a83b2 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -231,6 +231,40 @@ ifdef::git-diff[]\n endif::git-diff[]\n \tIt is the same as `--color=never`.\n \n+--color-moved[=]::\n+\tMoved lines of code are colored differently.\n+ifdef::git-diff[]\n+\tIt can be changed by the `diff.colorMoved` configuration setting.\n+endif::git-diff[]\n+\tThe defaults to 'no' if the option is not given\n+\tand to 'zebra' if the option with no mode is given.\n+\tThe mode must be one of:\n++\n+--\n+no::\n+\tMoved lines are not highlighted.\n+default::\n+\tIs a synonym for `zebra`. This may change to a more sensible mode\n+\tin the future.\n+plain::\n+\tAny line that is added in one location and was removed\n+\tin another location will be colored with 'color.diff.newMoved'.\n+\tSimilarly 'color.diff.oldMoved' will be used for removed lines\n+\tthat are added somewhere else in the diff. This mode picks up any\n+\tmoved line, but it is not very useful in a review to determine\n+\tif a block of code was moved without permutation.\n+zebra::\n+\tBlocks of moved code are detected greedily. The detected blocks are\n+\tpainted using either the 'color.diff.{old,new}Moved' color or\n+\t'color.diff.{old,new}MovedAlternative'. The change between\n+\tthe two colors indicates that a new block was detected.\n+\tSmall blocks of 3 moved lines or fewer are skipped.\n+dimmed_zebra::\n+\tSimilar to 'zebra', but additional dimming of uninteresting parts\n+\tof moved code is performed. The bordering lines of two adjacent\n+\tblocks are considered interesting, the rest is uninteresting.\n+--\n+\n --word-diff[=]::\n \tShow a word diff, using the to delimit changed words.\n \tBy default, words are delimited by whitespace; see\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323531","messageId":"20170630000710.10601-20-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 19/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:04Z","receivedAt":"2017-06-30T00:08:14Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 10 +++++++---\n 1 file changed, 7 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex e0d39d04da..5a9c55736d 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -571,6 +571,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n \tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_WORD_DIFF,\n+\tDIFF_SYMBOL_STAT_SEP,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -766,6 +767,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_WORD_DIFF:\n \t\tfprintf(o->file, \"%.*s\", len, line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_STAT_SEP:\n+\t\tfputs(o->stat_sep, o->file);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -5077,10 +5081,10 @@ void diff_flush(struct diff_options *options)\n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n \t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);\n-\t\t\tif (options->stat_sep) {\n+\t\t\tif (options->stat_sep)\n \t\t\t\t/* attach patch instead of inline */\n-\t\t\t\tfputs(options->stat_sep, options->file);\n-\t\t\t}\n+\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STAT_SEP,\n+\t\t\t\t\t\t NULL, 0, 0);\n \t\t}\n \n \t\tdiff_flush_patch_all_file_pairs(options);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323532","messageId":"20170630000710.10601-12-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 11/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:56Z","receivedAt":"2017-06-30T00:08:20Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We have to use fprintf instead of emit_line, because we want to emit the\ntab after the color. This is important for ancient versions of gnu patch\nAFAICT, although we probably do not want to feed colored output to the\npatch utility, such that it would not matter if the trailing tab is\ncolored. Keep the corner case as-is though.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 51 ++++++++++++++++++++++++++++++---------------------\n 1 file changed, 30 insertions(+), 21 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex b2b2a19fcf..49b45fef29 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,8 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_FILEPAIR_PLUS,\n+\tDIFF_SYMBOL_FILEPAIR_MINUS,\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n \tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n@@ -611,7 +613,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset, *set;\n+\tconst char *context, *reset, *set, *meta;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -673,6 +675,20 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t}\n \t\temit_line(o, context, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_FILEPAIR_PLUS:\n+\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\", diff_line_prefix(o), meta,\n+\t\t\tline, reset,\n+\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_FILEPAIR_MINUS:\n+\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\", diff_line_prefix(o), meta,\n+\t\t\tline, reset,\n+\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -844,8 +860,6 @@ static void emit_rewrite_diff(const char *name_a,\n \t\t\t struct diff_options *o)\n {\n \tint lc_a, lc_b;\n-\tconst char *name_a_tab, *name_b_tab;\n-\tconst char *metainfo = diff_get_color(o->use_color, DIFF_METAINFO);\n \tconst char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n \tconst char *reset = diff_get_color(o->use_color, DIFF_RESET);\n \tstatic struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;\n@@ -865,8 +879,6 @@ static void emit_rewrite_diff(const char *name_a,\n \n \tname_a += (*name_a == '/');\n \tname_b += (*name_b == '/');\n-\tname_a_tab = strchr(name_a, ' ') ? \"\\t\" : \"\";\n-\tname_b_tab = strchr(name_b, ' ') ? \"\\t\" : \"\";\n \n \tstrbuf_reset(&a_name);\n \tstrbuf_reset(&b_name);\n@@ -893,11 +905,13 @@ static void emit_rewrite_diff(const char *name_a,\n \n \tlc_a = count_lines(data_one, size_one);\n \tlc_b = count_lines(data_two, size_two);\n-\tfprintf(o->file,\n-\t\t\"%s%s--- %s%s%s\\n%s%s+++ %s%s%s\\n%s%s@@ -\",\n-\t\tline_prefix, metainfo, a_name.buf, name_a_tab, reset,\n-\t\tline_prefix, metainfo, b_name.buf, name_b_tab, reset,\n-\t\tline_prefix, fraginfo);\n+\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,\n+\t\t\t a_name.buf, a_name.len, 0);\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n+\t\t\t b_name.buf, b_name.len, 0);\n+\n+\tfprintf(o->file, \"%s%s@@ -\", line_prefix, fraginfo);\n \tif (!o->irreversible_delete)\n \t\tprint_line_count(o->file, lc_a);\n \telse\n@@ -1365,10 +1379,8 @@ static void find_lno(const char *line, struct emit_callback *ecbdata)\n static void fn_out_consume(void *priv, char *line, unsigned long len)\n {\n \tstruct emit_callback *ecbdata = priv;\n-\tconst char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n-\tconst char *line_prefix = diff_line_prefix(o);\n \n \to->found_changes = 1;\n \n@@ -1379,15 +1391,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->label_path[0]) {\n-\t\tconst char *name_a_tab, *name_b_tab;\n-\n-\t\tname_a_tab = strchr(ecbdata->label_path[0], ' ') ? \"\\t\" : \"\";\n-\t\tname_b_tab = strchr(ecbdata->label_path[1], ' ') ? \"\\t\" : \"\";\n-\n-\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);\n-\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,\n+\t\t\t\t ecbdata->label_path[0],\n+\t\t\t\t strlen(ecbdata->label_path[0]), 0);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n+\t\t\t\t ecbdata->label_path[1],\n+\t\t\t\t strlen(ecbdata->label_path[1]), 0);\n \t\tecbdata->label_path[0] = ecbdata->label_path[1] = NULL;\n \t}\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323533","messageId":"20170630000710.10601-24-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 23/25] diff.c: color moved lines differently, plain mode","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:08Z","receivedAt":"2017-06-30T00:08:22Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add the 'plain' mode for move detection of code. This omits the checking\nfor adjacent blocks, so it is not as useful. If you have a lot of the\nsame blocks moved in the same patch, the 'Zebra' would end up slow as it\nis O(n^2) (n is number of same blocks). So this may be useful there and\nis generally easy to add. Instead be very literal at the move detection,\ndo not skip over short blocks here.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 10 +++++++--\n diff.h | 1 +\n t/t4015-diff-whitespace.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++-\n 3 files changed, 59 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 322a2bfd72..41ee81d1d8 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -256,12 +256,14 @@ static int parse_color_moved(const char *arg)\n \n \tif (!strcmp(arg, \"no\"))\n \t\treturn COLOR_MOVED_NO;\n+\telse if (!strcmp(arg, \"plain\"))\n+\t\treturn COLOR_MOVED_PLAIN;\n \telse if (!strcmp(arg, \"zebra\"))\n \t\treturn COLOR_MOVED_ZEBRA;\n \telse if (!strcmp(arg, \"default\"))\n \t\treturn COLOR_MOVED_DEFAULT;\n \telse\n-\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra'\"));\n+\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra', 'plain'\"));\n }\n \n int git_diff_ui_config(const char *var, const char *value, void *cb)\n@@ -876,7 +878,8 @@ static void mark_color_as_moved(struct diff_options *o,\n \t\t}\n \n \t\tif (!match) {\n-\t\t\tif (block_length < COLOR_MOVED_MIN_BLOCK_LENGTH) {\n+\t\t\tif (block_length < COLOR_MOVED_MIN_BLOCK_LENGTH &&\n+\t\t\t o->color_moved != COLOR_MOVED_PLAIN) {\n \t\t\t\tfor (i = 0; i < block_length + 1; i++) {\n \t\t\t\t\tl = &o->emitted_symbols->buf[n - i];\n \t\t\t\t\tl->flags &= ~DIFF_SYMBOL_MOVED_LINE;\n@@ -890,6 +893,9 @@ static void mark_color_as_moved(struct diff_options *o,\n \t\tl->flags |= DIFF_SYMBOL_MOVED_LINE;\n \t\tblock_length++;\n \n+\t\tif (o->color_moved == COLOR_MOVED_PLAIN)\n+\t\t\tcontinue;\n+\n \t\t/* Check any potential block runs, advance each or nullify */\n \t\tfor (i = 0; i < pmb_nr; i++) {\n \t\t\tstruct moved_entry *p = pmb[i];\ndiff --git a/diff.h b/diff.h\nindex 3196802673..4cfd609c54 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -190,6 +190,7 @@ struct diff_options {\n \tstruct emitted_diff_symbols *emitted_symbols;\n \tenum {\n \t\tCOLOR_MOVED_NO = 0,\n+\t\tCOLOR_MOVED_PLAIN = 1,\n \t\tCOLOR_MOVED_ZEBRA = 2,\n \t} color_moved;\n \t#define COLOR_MOVED_DEFAULT COLOR_MOVED_ZEBRA\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 29704ae14e..d1d7b99990 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -986,7 +986,7 @@ test_expect_success 'detect moved code, complete file' '\n \tgit mv test.c main.c &&\n \ttest_config color.diff.oldMoved \"normal red\" &&\n \ttest_config color.diff.newMoved \"normal green\" &&\n-\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tgit diff HEAD --color-moved=zebra --no-renames | test_decode_color >actual &&\n \tcat >expected <<-\\EOF &&\n \tdiff --git a/main.c b/main.c\n \tnew file mode 100644\n@@ -1130,6 +1130,55 @@ test_expect_success 'detect malicious moved code, inside file' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'plain moved code, inside file' '\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\t# needs previous test as setup\n+\tgit diff HEAD --no-renames --color-moved=plain| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'no effect from --color-moved with --word-diff' '\n \tcat <<-\\EOF >text.txt &&\n \tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323534","messageId":"20170630000710.10601-16-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 15/25] submodule.c: migrate diff output to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:00Z","receivedAt":"2017-06-30T00:08:23Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"As the submodule process is no longer attached to the same file pointer\n'o->file' as the superprojects process, there is a different result in\ncolor.c::check_auto_color. That is why we need to pass coloring explicitly,\nsuch that the submodule coloring decision will be made by the child process\nprocessing the submodule. Only DIFF_SYMBOL_SUBMODULE_PIPETHROUGH contains\ncolor, the other symbols are for embedding the submodule output into the\nsuperprojects output.\n\nRemove the colors from the function signatures, as all the coloring\ndecisions will be made either inside the child process or the final\nemit_diff_symbol, but not in the functions driving the submodule diff.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++---------\n diff.h | 9 +++++++\n submodule.c | 84 +++++++++++++++++++++++++++----------------------------------\n submodule.h | 13 +++-------\n 4 files changed, 121 insertions(+), 68 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 5c428e02b6..48f719fb07 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,13 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_SUBMODULE_ADD,\n+\tDIFF_SYMBOL_SUBMODULE_DEL,\n+\tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\tDIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\tDIFF_SYMBOL_SUBMODULE_HEADER,\n+\tDIFF_SYMBOL_SUBMODULE_ERROR,\n+\tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n \tDIFF_SYMBOL_REWRITE_DIFF,\n \tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n@@ -625,6 +632,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_0(o, context, reset, '\\\\',\n \t\t\t nneof, strlen(nneof));\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n+\tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n+\tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -701,11 +711,68 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line(o, fraginfo, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_ADD:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, set, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_DEL:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, set, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_UNTRACKED:\n+\t\tfprintf(o->file, \"%sSubmodule %s contains untracked content\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_MODIFIED:\n+\t\tfprintf(o->file, \"%sSubmodule %s contains modified content\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n }\n \n+void diff_emit_submodule_del(struct diff_options *o, const char *line)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);\n+}\n+\n+void diff_emit_submodule_add(struct diff_options *o, const char *line)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ADD, line, strlen(line), 0);\n+}\n+\n+void diff_emit_submodule_untracked(struct diff_options *o, const char *path)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\t\t\t path, strlen(path), 0);\n+}\n+\n+void diff_emit_submodule_modified(struct diff_options *o, const char *path)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\t\t\t path, strlen(path), 0);\n+}\n+\n+void diff_emit_submodule_header(struct diff_options *o, const char *header)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_HEADER,\n+\t\t\t header, strlen(header), 0);\n+}\n+\n+void diff_emit_submodule_error(struct diff_options *o, const char *err)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ERROR, err, strlen(err), 0);\n+}\n+\n+void diff_emit_submodule_pipethrough(struct diff_options *o,\n+\t\t\t\t const char *line, int len)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_PIPETHROUGH, line, len, 0);\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -2467,24 +2534,16 @@ static void builtin_diff(const char *name_a,\n \tif (o->submodule_format == DIFF_SUBMODULE_LOG &&\n \t (!one->mode || S_ISGITLINK(one->mode)) &&\n \t (!two->mode || S_ISGITLINK(two->mode))) {\n-\t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n-\t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_summary(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_summary(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n-\t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset);\n+\t\t\t\ttwo->dirty_submodule);\n \t\treturn;\n \t} else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&\n \t\t (!one->mode || S_ISGITLINK(one->mode)) &&\n \t\t (!two->mode || S_ISGITLINK(two->mode))) {\n-\t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n-\t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_inline_diff(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_inline_diff(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n-\t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset, o);\n+\t\t\t\ttwo->dirty_submodule);\n \t\treturn;\n \t}\n \ndiff --git a/diff.h b/diff.h\nindex ea66168454..ae40bac7a9 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,15 @@ struct diff_options {\n \tint diff_path_counter;\n };\n \n+void diff_emit_submodule_del(struct diff_options *o, const char *line);\n+void diff_emit_submodule_add(struct diff_options *o, const char *line);\n+void diff_emit_submodule_untracked(struct diff_options *o, const char *path);\n+void diff_emit_submodule_modified(struct diff_options *o, const char *path);\n+void diff_emit_submodule_header(struct diff_options *o, const char *header);\n+void diff_emit_submodule_error(struct diff_options *o, const char *err);\n+void diff_emit_submodule_pipethrough(struct diff_options *o,\n+\t\t\t\t const char *line, int len);\n+\n enum color_diff {\n \tDIFF_RESET = 0,\n \tDIFF_CONTEXT = 1,\ndiff --git a/submodule.c b/submodule.c\nindex da0b805493..07a3183150 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -479,9 +479,7 @@ static int prepare_submodule_summary(struct rev_info *rev, const char *path,\n \treturn prepare_revision_walk(rev);\n }\n \n-static void print_submodule_summary(struct rev_info *rev, FILE *f,\n-\t\tconst char *line_prefix,\n-\t\tconst char *del, const char *add, const char *reset)\n+static void print_submodule_summary(struct rev_info *rev, struct diff_options *o)\n {\n \tstatic const char format[] = \" %m %s\";\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -492,18 +490,12 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \t\tctx.date_mode = rev->date_mode;\n \t\tctx.output_encoding = get_log_output_encoding();\n \t\tstrbuf_setlen(&sb, 0);\n-\t\tstrbuf_addstr(&sb, line_prefix);\n-\t\tif (commit->object.flags & SYMMETRIC_LEFT) {\n-\t\t\tif (del)\n-\t\t\t\tstrbuf_addstr(&sb, del);\n-\t\t}\n-\t\telse if (add)\n-\t\t\tstrbuf_addstr(&sb, add);\n \t\tformat_commit_message(commit, format, &sb, &ctx);\n-\t\tif (reset)\n-\t\t\tstrbuf_addstr(&sb, reset);\n \t\tstrbuf_addch(&sb, '\\n');\n-\t\tfprintf(f, \"%s\", sb.buf);\n+\t\tif (commit->object.flags & SYMMETRIC_LEFT)\n+\t\t\tdiff_emit_submodule_del(o, sb.buf);\n+\t\telse\n+\t\t\tdiff_emit_submodule_add(o, sb.buf);\n \t}\n \tstrbuf_release(&sb);\n }\n@@ -530,11 +522,9 @@ void prepare_submodule_repo_env(struct argv_array *out)\n * attempt to lookup both the left and right commits and put them into the\n * left and right pointers.\n */\n-static void show_submodule_header(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+static void show_submodule_header(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *reset,\n+\t\tunsigned dirty_submodule,\n \t\tstruct commit **left, struct commit **right,\n \t\tstruct commit_list **merge_bases)\n {\n@@ -543,11 +533,10 @@ static void show_submodule_header(FILE *f, const char *path,\n \tint fast_forward = 0, fast_backward = 0;\n \n \tif (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED)\n-\t\tfprintf(f, \"%sSubmodule %s contains untracked content\\n\",\n-\t\t\tline_prefix, path);\n+\t\tdiff_emit_submodule_untracked(o, path);\n+\n \tif (dirty_submodule & DIRTY_SUBMODULE_MODIFIED)\n-\t\tfprintf(f, \"%sSubmodule %s contains modified content\\n\",\n-\t\t\tline_prefix, path);\n+\t\tdiff_emit_submodule_modified(o, path);\n \n \tif (is_null_oid(one))\n \t\tmessage = \"(new submodule)\";\n@@ -589,31 +578,29 @@ static void show_submodule_header(FILE *f, const char *path,\n \t}\n \n output_header:\n-\tstrbuf_addf(&sb, \"%s%sSubmodule %s \", line_prefix, meta, path);\n+\tstrbuf_addf(&sb, \"Submodule %s \", path);\n \tstrbuf_add_unique_abbrev(&sb, one->hash, DEFAULT_ABBREV);\n \tstrbuf_addstr(&sb, (fast_backward || fast_forward) ? \"..\" : \"...\");\n \tstrbuf_add_unique_abbrev(&sb, two->hash, DEFAULT_ABBREV);\n \tif (message)\n-\t\tstrbuf_addf(&sb, \" %s%s\\n\", message, reset);\n+\t\tstrbuf_addf(&sb, \" %s\\n\", message);\n \telse\n-\t\tstrbuf_addf(&sb, \"%s:%s\\n\", fast_backward ? \" (rewind)\" : \"\", reset);\n-\tfwrite(sb.buf, sb.len, 1, f);\n+\t\tstrbuf_addf(&sb, \"%s:\\n\", fast_backward ? \" (rewind)\" : \"\");\n+\tdiff_emit_submodule_header(o, sb.buf);\n \n \tstrbuf_release(&sb);\n }\n \n-void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset)\n+\t\tunsigned dirty_submodule)\n {\n \tstruct rev_info rev;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n-\t\t\t meta, reset, &left, &right, &merge_bases);\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n+\t\t\t &left, &right, &merge_bases);\n \n \t/*\n \t * If we don't have both a left and a right pointer, there is no\n@@ -625,11 +612,11 @@ void show_submodule_summary(FILE *f, const char *path,\n \n \t/* Treat revision walker failure the same as missing commits */\n \tif (prepare_submodule_summary(&rev, path, left, right, merge_bases)) {\n-\t\tfprintf(f, \"%s(revision walker failed)\\n\", line_prefix);\n+\t\tdiff_emit_submodule_error(o, \"(revision walker failed)\\n\");\n \t\tgoto out;\n \t}\n \n-\tprint_submodule_summary(&rev, f, line_prefix, del, add, reset);\n+\tprint_submodule_summary(&rev, o);\n \n out:\n \tif (merge_bases)\n@@ -638,21 +625,18 @@ void show_submodule_summary(FILE *f, const char *path,\n \tclear_commit_marks(right, ~0);\n }\n \n-void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *o)\n+\t\tunsigned dirty_submodule)\n {\n \tconst struct object_id *old = &empty_tree_oid, *new = &empty_tree_oid;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n-\tstruct strbuf submodule_dir = STRBUF_INIT;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n-\t\t\t meta, reset, &left, &right, &merge_bases);\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n+\t\t\t &left, &right, &merge_bases);\n \n \t/* We need a valid left and right commit to display a difference */\n \tif (!(left || is_null_oid(one)) ||\n@@ -664,16 +648,16 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \tif (right)\n \t\tnew = two;\n \n-\tfflush(f);\n \tcp.git_cmd = 1;\n \tcp.dir = path;\n-\tcp.out = dup(fileno(f));\n+\tcp.out = -1;\n \tcp.no_stdin = 1;\n \n \t/* TODO: other options may need to be passed here. */\n \targv_array_pushl(&cp.args, \"diff\", \"--submodule=diff\", NULL);\n+\targv_array_pushf(&cp.args, \"--color=%s\", want_color(o->use_color) ?\n+\t\t\t \"always\" : \"never\");\n \n-\targv_array_pushf(&cp.args, \"--line-prefix=%s\", line_prefix);\n \tif (DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\targv_array_pushf(&cp.args, \"--src-prefix=%s%s/\",\n \t\t\t\t o->b_prefix, path);\n@@ -696,11 +680,17 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \t\targv_array_push(&cp.args, oid_to_hex(new));\n \n \tprepare_submodule_repo_env(&cp.env_array);\n-\tif (run_command(&cp))\n-\t\tfprintf(f, \"(diff failed)\\n\");\n+\tif (start_command(&cp))\n+\t\tdiff_emit_submodule_error(o, \"(diff failed)\\n\");\n+\n+\twhile (strbuf_getwholeline_fd(&sb, cp.out, '\\n') != EOF)\n+\t\tdiff_emit_submodule_pipethrough(o, sb.buf, sb.len);\n+\n+\tif (finish_command(&cp))\n+\t\tdiff_emit_submodule_error(o, \"(diff failed)\\n\");\n \n done:\n-\tstrbuf_release(&submodule_dir);\n+\tstrbuf_release(&sb);\n \tif (merge_bases)\n \t\tfree_commit_list(merge_bases);\n \tif (left)\ndiff --git a/submodule.h b/submodule.h\nindex cbe5c1726f..829d975585 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -63,17 +63,12 @@ extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n extern void handle_ignore_submodules_arg(struct diff_options *, const char *);\n-extern void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+extern void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset);\n-extern void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+\t\tunsigned dirty_submodule);\n+extern void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *opt);\n+\t\tunsigned dirty_submodule);\n extern void set_config_fetch_recurse_submodules(int value);\n /* Check if we want to update any submodule.*/\n extern int should_update_submodules(void);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323535","messageId":"20170630000710.10601-15-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 14/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:59Z","receivedAt":"2017-06-30T00:08:25Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 35 +++++++++++++++++++++--------------\n 1 file changed, 21 insertions(+), 14 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 1d0f8b509f..5c428e02b6 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_REWRITE_DIFF,\n \tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n@@ -615,7 +616,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset, *set, *meta;\n+\tconst char *context, *reset, *set, *meta, *fraginfo;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -695,6 +696,11 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_REWRITE_DIFF:\n+\t\tfraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, fraginfo, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -817,17 +823,17 @@ static void remove_tempfile(void)\n \t}\n }\n \n-static void print_line_count(FILE *file, int count)\n+static void add_line_count(struct strbuf *out, int count)\n {\n \tswitch (count) {\n \tcase 0:\n-\t\tfprintf(file, \"0,0\");\n+\t\tstrbuf_addstr(out, \"0,0\");\n \t\tbreak;\n \tcase 1:\n-\t\tfprintf(file, \"1\");\n+\t\tstrbuf_addstr(out, \"1\");\n \t\tbreak;\n \tdefault:\n-\t\tfprintf(file, \"1,%d\", count);\n+\t\tstrbuf_addf(out, \"1,%d\", count);\n \t\tbreak;\n \t}\n }\n@@ -866,14 +872,12 @@ static void emit_rewrite_diff(const char *name_a,\n \t\t\t struct diff_options *o)\n {\n \tint lc_a, lc_b;\n-\tconst char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n-\tconst char *reset = diff_get_color(o->use_color, DIFF_RESET);\n \tstatic struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;\n \tconst char *a_prefix, *b_prefix;\n \tchar *data_one, *data_two;\n \tsize_t size_one, size_two;\n \tstruct emit_callback ecbdata;\n-\tconst char *line_prefix = diff_line_prefix(o);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\ta_prefix = o->b_prefix;\n@@ -917,14 +921,17 @@ static void emit_rewrite_diff(const char *name_a,\n \temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n \t\t\t b_name.buf, b_name.len, 0);\n \n-\tfprintf(o->file, \"%s%s@@ -\", line_prefix, fraginfo);\n+\tstrbuf_addstr(&out, \"@@ -\");\n \tif (!o->irreversible_delete)\n-\t\tprint_line_count(o->file, lc_a);\n+\t\tadd_line_count(&out, lc_a);\n \telse\n-\t\tfprintf(o->file, \"?,?\");\n-\tfprintf(o->file, \" +\");\n-\tprint_line_count(o->file, lc_b);\n-\tfprintf(o->file, \" @@%s\\n\", reset);\n+\t\tstrbuf_addstr(&out, \"?,?\");\n+\tstrbuf_addstr(&out, \" +\");\n+\tadd_line_count(&out, lc_b);\n+\tstrbuf_addstr(&out, \" @@\\n\");\n+\temit_diff_symbol(o, DIFF_SYMBOL_REWRITE_DIFF, out.buf, out.len, 0);\n+\tstrbuf_release(&out);\n+\n \tif (lc_a && !o->irreversible_delete)\n \t\temit_rewrite_lines(&ecbdata, '-', data_one, size_one);\n \tif (lc_b)\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323536","messageId":"20170630000710.10601-17-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 16/25] diff.c: convert emit_binary_diff_body to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:01Z","receivedAt":"2017-06-30T00:08:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++-----------------\n 1 file changed, 46 insertions(+), 17 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 48f719fb07..f5a14359ae 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,11 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n+\tDIFF_SYMBOL_BINARY_DIFF_BODY,\n+\tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -635,6 +640,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n+\tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -706,6 +712,19 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER:\n+\t\tfprintf(o->file, \"%sGIT binary patch\\n\", diff_line_prefix(o));\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA:\n+\t\tfprintf(o->file, \"%sdelta %s\\n\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:\n+\t\tfprintf(o->file, \"%sliteral %s\\n\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_FOOTER:\n+\t\tfputs(diff_line_prefix(o), o->file);\n+\t\tfputc('\\n', o->file);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_REWRITE_DIFF:\n \t\tfraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -2390,8 +2409,8 @@ static unsigned char *deflate_it(char *data,\n \treturn deflated;\n }\n \n-static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t\t const char *prefix)\n+static void emit_binary_diff_body(struct diff_options *o,\n+\t\t\t\t mmfile_t *one, mmfile_t *two)\n {\n \tvoid *cp;\n \tvoid *delta;\n@@ -2420,13 +2439,18 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t}\n \n \tif (delta && delta_size < deflate_size) {\n-\t\tfprintf(file, \"%sdelta %lu\\n\", prefix, orig_size);\n+\t\tchar *s = xstrfmt(\"%lu\", orig_size);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n+\t\t\t\t s, strlen(s), 0);\n+\t\tfree(s);\n \t\tfree(deflated);\n \t\tdata = delta;\n \t\tdata_size = delta_size;\n-\t}\n-\telse {\n-\t\tfprintf(file, \"%sliteral %lu\\n\", prefix, two->size);\n+\t} else {\n+\t\tchar *s = xstrfmt(\"%lu\", two->size);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n+\t\t\t\t s, strlen(s), 0);\n+\t\tfree(s);\n \t\tfree(delta);\n \t\tdata = deflated;\n \t\tdata_size = deflate_size;\n@@ -2435,8 +2459,9 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t/* emit data encoded in base85 */\n \tcp = data;\n \twhile (data_size) {\n+\t\tint len;\n \t\tint bytes = (52 < data_size) ? 52 : data_size;\n-\t\tchar line[70];\n+\t\tchar line[71];\n \t\tdata_size -= bytes;\n \t\tif (bytes <= 26)\n \t\t\tline[0] = bytes + 'A' - 1;\n@@ -2444,20 +2469,24 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t\t\tline[0] = bytes - 26 + 'a' - 1;\n \t\tencode_85(line + 1, cp, bytes);\n \t\tcp = (char *) cp + bytes;\n-\t\tfprintf(file, \"%s\", prefix);\n-\t\tfputs(line, file);\n-\t\tfputc('\\n', file);\n+\n+\t\tlen = strlen(line);\n+\t\tline[len++] = '\\n';\n+\t\tline[len] = '\\0';\n+\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_BODY,\n+\t\t\t\t line, len, 0);\n \t}\n-\tfprintf(file, \"%s\\n\", prefix);\n+\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_FOOTER, NULL, 0, 0);\n \tfree(data);\n }\n \n-static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t const char *prefix)\n+static void emit_binary_diff(struct diff_options *o,\n+\t\t\t mmfile_t *one, mmfile_t *two)\n {\n-\tfprintf(file, \"%sGIT binary patch\\n\", prefix);\n-\temit_binary_diff_body(file, one, two, prefix);\n-\temit_binary_diff_body(file, two, one, prefix);\n+\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER, NULL, 0, 0);\n+\temit_binary_diff_body(o, one, two);\n+\temit_binary_diff_body(o, two, one);\n }\n \n int diff_filespec_is_binary(struct diff_filespec *one)\n@@ -2643,7 +2672,7 @@ static void builtin_diff(const char *name_a,\n \t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n-\t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n+\t\t\temit_binary_diff(o, &mf1, &mf2);\n \t\telse {\n \t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n \t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323537","messageId":"20170630000710.10601-11-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 10/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:55Z","receivedAt":"2017-06-30T00:08:33Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The context marker use the exact same output pattern, so reuse it.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 85bfd9310d..b2b2a19fcf 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -564,6 +564,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n \tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_CONTEXT_INCOMPLETE,\n \tDIFF_SYMBOL_PLUS,\n \tDIFF_SYMBOL_MINUS,\n \tDIFF_SYMBOL_NO_LF_EOF,\n@@ -622,6 +623,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT_INCOMPLETE:\n \tcase DIFF_SYMBOL_CONTEXT_MARKER:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -1448,8 +1450,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \tdefault:\n \t\t/* incomplete line at the end */\n \t\tecbdata->lno_in_preimage++;\n-\t\temit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),\n-\t\t\t reset, line, len);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_CONTEXT_INCOMPLETE,\n+\t\t\t\t line, len, 0);\n \t\tbreak;\n \t}\n }\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323538","messageId":"20170630000710.10601-18-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 17/25] diff.c: convert show_stats to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:07:02Z","receivedAt":"2017-06-30T00:08:37Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We call print_stat_summary from builtin/apply, so we still\nneed the version with a file pointer, so introduce\nprint_stat_summary_0 that uses emit_string machinery and\nkeep print_stat_summary with the same arguments around.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 114 +++++++++++++++++++++++++++++++++++++++++------------------------\n diff.h | 4 +--\n 2 files changed, 74 insertions(+), 44 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex f5a14359ae..76d4b8ebf9 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -566,6 +566,10 @@ enum diff_symbol {\n \tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n \tDIFF_SYMBOL_BINARY_DIFF_BODY,\n \tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n+\tDIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n+\tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n+\tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n+\tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -629,6 +633,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n+\tstruct strbuf sb = STRBUF_INIT;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -640,6 +645,8 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n+\tcase DIFF_SYMBOL_STATS_LINE:\n \tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n@@ -748,9 +755,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\tfprintf(o->file, \"%sSubmodule %s contains modified content\\n\",\n \t\t\tdiff_line_prefix(o), line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:\n+\t\temit_line(o, \"\", \"\", \" 0 files changed\\n\",\n+\t\t\t strlen(\" 0 files changed\\n\"));\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n+\t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n+\tstrbuf_release(&sb);\n }\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line)\n@@ -1705,20 +1720,14 @@ static int scale_linear(int it, int width, int max_change)\n \treturn 1 + (it * (width - 1) / max_change);\n }\n \n-static void show_name(FILE *file,\n-\t\t const char *prefix, const char *name, int len)\n-{\n-\tfprintf(file, \" %s%-*s |\", prefix, len, name);\n-}\n-\n-static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)\n+static void show_graph(struct strbuf *out, char ch, int cnt,\n+\t\t const char *set, const char *reset)\n {\n \tif (cnt <= 0)\n \t\treturn;\n-\tfprintf(file, \"%s\", set);\n-\twhile (cnt--)\n-\t\tputc(ch, file);\n-\tfprintf(file, \"%s\", reset);\n+\tstrbuf_addstr(out, set);\n+\tstrbuf_addchars(out, ch, cnt);\n+\tstrbuf_addstr(out, reset);\n }\n \n static void fill_print_name(struct diffstat_file *file)\n@@ -1742,14 +1751,16 @@ static void fill_print_name(struct diffstat_file *file)\n \tfile->print_name = pname;\n }\n \n-int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n+static void print_stat_summary_inserts_deletes(struct diff_options *options,\n+\t\tint files, int insertions, int deletions)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint ret;\n \n \tif (!files) {\n \t\tassert(insertions == 0 && deletions == 0);\n-\t\treturn fprintf(fp, \"%s\\n\", \" 0 files changed\");\n+\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n+\t\t\t\t NULL, 0, 0);\n+\t\treturn;\n \t}\n \n \tstrbuf_addf(&sb,\n@@ -1776,9 +1787,19 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n \t\t\t deletions);\n \t}\n \tstrbuf_addch(&sb, '\\n');\n-\tret = fputs(sb.buf, fp);\n+\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n+\t\t\t sb.buf, sb.len, 0);\n \tstrbuf_release(&sb);\n-\treturn ret;\n+}\n+\n+void print_stat_summary(FILE *fp, int files,\n+\t\t\tint insertions, int deletions)\n+{\n+\tstruct diff_options o;\n+\tmemset(&o, 0, sizeof(o));\n+\to.file = fp;\n+\n+\tprint_stat_summary_inserts_deletes(&o, files, insertions, deletions);\n }\n \n static void show_stats(struct diffstat_t *data, struct diff_options *options)\n@@ -1788,13 +1809,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \tint total_files = data->nr, count;\n \tint width, name_width, graph_width, number_width = 0, bin_width = 0;\n \tconst char *reset, *add_c, *del_c;\n-\tconst char *line_prefix = \"\";\n \tint extra_shown = 0;\n+\tconst char *line_prefix = diff_line_prefix(options);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (data->nr == 0)\n \t\treturn;\n \n-\tline_prefix = diff_line_prefix(options);\n \tcount = options->stat_count ? options->stat_count : data->nr;\n \n \treset = diff_get_color_opt(options, DIFF_RESET);\n@@ -1948,26 +1969,32 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t}\n \n \t\tif (file->is_binary) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" %*s\", number_width, \"Bin\");\n+\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\t\tstrbuf_addf(&out, \" %*s\", number_width, \"Bin\");\n \t\t\tif (!added && !deleted) {\n-\t\t\t\tputc('\\n', options->file);\n+\t\t\t\tstrbuf_addch(&out, '\\n');\n+\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\t\tstrbuf_reset(&out);\n \t\t\t\tcontinue;\n \t\t\t}\n-\t\t\tfprintf(options->file, \" %s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addf(&out, \" %s%\"PRIuMAX\"%s\",\n \t\t\t\tdel_c, deleted, reset);\n-\t\t\tfprintf(options->file, \" -> \");\n-\t\t\tfprintf(options->file, \"%s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addstr(&out, \" -> \");\n+\t\t\tstrbuf_addf(&out, \"%s%\"PRIuMAX\"%s\",\n \t\t\t\tadd_c, added, reset);\n-\t\t\tfprintf(options->file, \" bytes\");\n-\t\t\tfprintf(options->file, \"\\n\");\n+\t\t\tstrbuf_addstr(&out, \" bytes\\n\");\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \t\telse if (file->is_unmerged) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" Unmerged\\n\");\n+\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\t\tstrbuf_addstr(&out, \" Unmerged\\n\");\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \n@@ -1990,14 +2017,16 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t\t\tadd = total - del;\n \t\t\t}\n \t\t}\n-\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\tshow_name(options->file, prefix, name, len);\n-\t\tfprintf(options->file, \" %*\"PRIuMAX\"%s\",\n+\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\tstrbuf_addf(&out, \" %*\"PRIuMAX\"%s\",\n \t\t\tnumber_width, added + deleted,\n \t\t\tadded + deleted ? \" \" : \"\");\n-\t\tshow_graph(options->file, '+', add, add_c, reset);\n-\t\tshow_graph(options->file, '-', del, del_c, reset);\n-\t\tfprintf(options->file, \"\\n\");\n+\t\tshow_graph(&out, '+', add, add_c, reset);\n+\t\tshow_graph(&out, '-', del, del_c, reset);\n+\t\tstrbuf_addch(&out, '\\n');\n+\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t out.buf, out.len, 0);\n+\t\tstrbuf_reset(&out);\n \t}\n \n \tfor (i = 0; i < data->nr; i++) {\n@@ -2018,11 +2047,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\tif (i < count)\n \t\t\tcontinue;\n \t\tif (!extra_shown)\n-\t\t\tfprintf(options->file, \"%s ...\\n\", line_prefix);\n+\t\t\temit_diff_symbol(options,\n+\t\t\t\t\t DIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n+\t\t\t\t\t NULL, 0, 0);\n \t\textra_shown = 1;\n \t}\n-\tfprintf(options->file, \"%s\", line_prefix);\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\n+\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n }\n \n static void show_shortstats(struct diffstat_t *data, struct diff_options *options)\n@@ -2034,7 +2065,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \n \tfor (i = 0; i < data->nr; i++) {\n \t\tint added = data->files[i]->added;\n-\t\tint deleted= data->files[i]->deleted;\n+\t\tint deleted = data->files[i]->deleted;\n \n \t\tif (data->files[i]->is_unmerged ||\n \t\t (!data->files[i]->is_interesting && (added + deleted == 0))) {\n@@ -2044,8 +2075,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \t\t\tdels += deleted;\n \t\t}\n \t}\n-\tfprintf(options->file, \"%s\", diff_line_prefix(options));\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n }\n \n static void show_numstat(struct diffstat_t *data, struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex ae40bac7a9..d6094a1eed 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -405,8 +405,8 @@ extern int parse_rename_score(const char **cp_p);\n \n extern long parse_algorithm_value(const char *value);\n \n-extern int print_stat_summary(FILE *fp, int files,\n-\t\t\t int insertions, int deletions);\n+extern void print_stat_summary(FILE *fp, int files,\n+\t\t\t int insertions, int deletions);\n extern void setup_diff_pager(struct diff_options *);\n \n #endif /* DIFF_H */\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323539","messageId":"20170630000710.10601-10-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 09/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:54Z","receivedAt":"2017-06-30T00:08:42Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 42 ++++++++++++++++++++++++++----------------\n 1 file changed, 26 insertions(+), 16 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex e5430d56da..85bfd9310d 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,8 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_WORDS_PORCELAIN,\n+\tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n \tDIFF_SYMBOL_PLUS,\n \tDIFF_SYMBOL_MINUS,\n@@ -649,6 +651,26 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_WORDS_PORCELAIN:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tfputs(\"~\\n\", o->file);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_WORDS:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\t/*\n+\t\t * Skip the prefix character, if any. With\n+\t\t * diff_suppress_blank_empty, there may be\n+\t\t * none.\n+\t\t */\n+\t\tif (line[0] != '\\n') {\n+\t\t\tline++;\n+\t\t\tlen--;\n+\t\t}\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1342,7 +1364,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n {\n \tstruct emit_callback *ecbdata = priv;\n \tconst char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);\n-\tconst char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n \tconst char *line_prefix = diff_line_prefix(o);\n@@ -1384,6 +1405,9 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->diff_words) {\n+\t\tenum diff_symbol s =\n+\t\t\tecbdata->diff_words->type == DIFF_WORDS_PORCELAIN ?\n+\t\t\tDIFF_SYMBOL_WORDS_PORCELAIN : DIFF_SYMBOL_WORDS;\n \t\tif (line[0] == '-') {\n \t\t\tdiff_words_append(line, len,\n \t\t\t\t\t &ecbdata->diff_words->minus);\n@@ -1403,21 +1427,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t\treturn;\n \t\t}\n \t\tdiff_words_flush(ecbdata);\n-\t\tif (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {\n-\t\t\temit_line(o, context, reset, line, len);\n-\t\t\tfputs(\"~\\n\", o->file);\n-\t\t} else {\n-\t\t\t/*\n-\t\t\t * Skip the prefix character, if any. With\n-\t\t\t * diff_suppress_blank_empty, there may be\n-\t\t\t * none.\n-\t\t\t */\n-\t\t\tif (line[0] != '\\n') {\n-\t\t\t line++;\n-\t\t\t len--;\n-\t\t\t}\n-\t\t\temit_line(o, context, reset, line, len);\n-\t\t}\n+\t\temit_diff_symbol(o, s, line, len, 0);\n \t\treturn;\n \t}\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323540","messageId":"20170630000710.10601-1-sbeller@google.com","threadId":"46290","inReplyTo":null,"subject":"[PATCH 00/25] Reroll of sb/diff-color-moved.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:45Z","receivedAt":"2017-06-30T00:08:46Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Reroll of sb/diff-color-moved, applies on top of the hashmap patches[1].\n\n* Supports different whitespace modes correctly now (and has a test for trivial\n white space changes)\n* The fixups[2] have been squashed.\n\nThanks,\nStefan\n\n[1] https://public-inbox.org/git/20170629235336.28460-1-sbeller@google.com/\n[2] https://public-inbox.org/git/20170628005651.8110-1-sbeller@google.com/\n\n\nStefan Beller (25):\n diff.c: readability fix\n diff.c: move line ending check into emit_hunk_header\n diff.c: factor out diff_flush_patch_all_file_pairs\n diff.c: introduce emit_diff_symbol\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF\n diff.c: migrate emit_line_checked to use emit_diff_symbol\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF\n submodule.c: migrate diff output to use emit_diff_symbol\n diff.c: convert emit_binary_diff_body to use emit_diff_symbol\n diff.c: convert show_stats to use emit_diff_symbol\n diff.c: convert word diffing to use emit_diff_symbol\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY\n diff.c: buffer all output if asked to\n diff.c: color moved lines differently\n diff.c: color moved lines differently, plain mode\n diff.c: add dimming to moved line detection\n diff: document the new --color-moved setting\n\n Documentation/config.txt | 15 +-\n Documentation/diff-options.txt | 34 ++\n cache.h | 2 +\n color.h | 2 +\n diff.c | 1306 ++++++++++++++++++++++++++++++++--------\n diff.h | 39 +-\n submodule.c | 84 ++-\n submodule.h | 13 +-\n t/t4015-diff-whitespace.sh | 434 +++++++++++++\n 9 files changed, 1613 insertions(+), 316 deletions(-)\n\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323541","messageId":"20170630000710.10601-9-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCH 08/25] diff.c: migrate emit_line_checked to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T00:06:53Z","receivedAt":"2017-06-30T00:08:48Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a new flags field to emit_diff_symbol, that will be used by\ncontext lines for:\n* white space rules that are applicable (The first 12 bits)\n Take a note in cahe.c as well, when this ws rules are extended we have\n to fix the bits in the flags field.\n* how the rules are evaluated (actually this double encodes the sign\n of the line, but the code is easier to keep this way, bits 13,14,15)\n* if the line a blank line at EOF (bit 16)\n\nThe check if new lines need to be marked up as extra lines at the end of\nfile, is now done unconditionally. That should be ok, as\n'new_blank_line_at_eof' has a quick early return.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n cache.h | 2 ++\n diff.c | 116 +++++++++++++++++++++++++++++++++++++++++-----------------------\n diff.h | 6 ++--\n 3 files changed, 80 insertions(+), 44 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 96055c2229..18aabacba5 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1996,6 +1996,8 @@ void shift_tree_by(const struct object_id *, const struct object_id *, struct ob\n #define WS_TRAILING_SPACE (WS_BLANK_AT_EOL|WS_BLANK_AT_EOF)\n #define WS_DEFAULT_RULE (WS_TRAILING_SPACE|WS_SPACE_BEFORE_TAB|8)\n #define WS_TAB_WIDTH_MASK 077\n+/* All WS_* -- when extended, adapt diff.c emit_symbol */\n+#define WS_RULE_MASK 07777\n extern unsigned whitespace_rule_cfg;\n extern unsigned whitespace_rule(const char *);\n extern unsigned parse_whitespace_rule(const char *);\ndiff --git a/diff.c b/diff.c\nindex 488096b757..e5430d56da 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,17 +561,54 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_PLUS,\n+\tDIFF_SYMBOL_MINUS,\n \tDIFF_SYMBOL_NO_LF_EOF,\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n+/*\n+ * Flags for content lines:\n+ * 0..12 are whitespace rules\n+ * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT\n+ * 16 is marking if the line is blank at EOF\n+ */\n+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n+#define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n+\n+static void emit_line_ws_markup(struct diff_options *o,\n+\t\t\t\tconst char *set, const char *reset,\n+\t\t\t\tconst char *line, int len, char sign,\n+\t\t\t\tunsigned ws_rule, int blank_at_eof)\n+{\n+\tconst char *ws = NULL;\n+\n+\tif (o->ws_error_highlight & ws_rule) {\n+\t\tws = diff_get_color_opt(o, DIFF_WHITESPACE);\n+\t\tif (!*ws)\n+\t\t\tws = NULL;\n+\t}\n+\n+\tif (!ws)\n+\t\temit_line_0(o, set, reset, sign, line, len);\n+\telse if (blank_at_eof)\n+\t\t/* Blank line at EOF - paint '+' as well */\n+\t\temit_line_0(o, ws, reset, sign, line, len);\n+\telse {\n+\t\t/* Emit just the prefix, then the rest. */\n+\t\temit_line_0(o, set, reset, sign, \"\", 0);\n+\t\tws_check_emit(line, len, ws_rule,\n+\t\t\t o->file, set, reset, ws);\n+\t}\n+}\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n-\t\t\t const char *line, int len)\n+\t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset;\n+\tconst char *context, *reset, *set;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -593,6 +630,25 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tdiff_line_prefix(o),\n \t\t\to->line_termination);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT:\n+\t\tset = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, ' ',\n+\t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_PLUS:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, '+',\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_MINUS:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, '-',\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -609,57 +665,31 @@ static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line\n \treturn ws_blank_line(line, len, ecbdata->ws_rule);\n }\n \n-static void emit_line_checked(const char *reset,\n-\t\t\t struct emit_callback *ecbdata,\n-\t\t\t const char *line, int len,\n-\t\t\t enum color_diff color,\n-\t\t\t unsigned ws_error_highlight,\n-\t\t\t char sign)\n-{\n-\tconst char *set = diff_get_color(ecbdata->color_diff, color);\n-\tconst char *ws = NULL;\n-\n-\tif (ecbdata->opt->ws_error_highlight & ws_error_highlight) {\n-\t\tws = diff_get_color(ecbdata->color_diff, DIFF_WHITESPACE);\n-\t\tif (!*ws)\n-\t\t\tws = NULL;\n-\t}\n-\n-\tif (!ws)\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, line, len);\n-\telse if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))\n-\t\t/* Blank line at EOF - paint '+' as well */\n-\t\temit_line_0(ecbdata->opt, ws, reset, sign, line, len);\n-\telse {\n-\t\t/* Emit just the prefix, then the rest. */\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, \"\", 0);\n-\t\tws_check_emit(line, len, ecbdata->ws_rule,\n-\t\t\t ecbdata->opt->file, set, reset, ws);\n-\t}\n-}\n-\n static void emit_add_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_FILE_NEW, WSEH_NEW, '+');\n+\tunsigned flags = WSEH_NEW | ecbdata->ws_rule;\n+\tif (new_blank_line_at_eof(ecbdata, line, len))\n+\t\tflags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;\n+\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);\n }\n \n static void emit_del_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_FILE_OLD, WSEH_OLD, '-');\n+\tunsigned flags = WSEH_OLD | ecbdata->ws_rule;\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_MINUS, line, len, flags);\n }\n \n static void emit_context_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_CONTEXT, WSEH_CONTEXT, ' ');\n+\tunsigned flags = WSEH_CONTEXT | ecbdata->ws_rule;\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_CONTEXT, line, len, flags);\n }\n \n static void emit_hunk_header(struct emit_callback *ecbdata,\n@@ -683,7 +713,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n \t\temit_diff_symbol(ecbdata->opt,\n-\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len);\n+\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len, 0);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n@@ -719,7 +749,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n \temit_diff_symbol(ecbdata->opt,\n-\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);\n+\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len, 0);\n \tstrbuf_release(&msgbuf);\n }\n \n@@ -778,7 +808,7 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tdata += len;\n \t}\n \tif (!endp)\n-\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);\n+\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0, 0);\n }\n \n static void emit_rewrite_diff(const char *name_a,\n@@ -4771,6 +4801,10 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n+\n+\tif (WSEH_NEW & WS_RULE_MASK)\n+\t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tif (check_pair_status(p))\n@@ -4861,7 +4895,7 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\ndiff --git a/diff.h b/diff.h\nindex 2d442e296f..ea66168454 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -148,9 +148,9 @@ struct diff_options {\n \tint abbrev;\n \tint ita_invisible_in_index;\n /* white-space error highlighting */\n-#define WSEH_NEW 1\n-#define WSEH_CONTEXT 2\n-#define WSEH_OLD 4\n+#define WSEH_NEW (1<<12)\n+#define WSEH_CONTEXT (1<<13)\n+#define WSEH_OLD (1<<14)\n \tunsigned ws_error_highlight;\n \tconst char *prefix;\n \tint prefix_length;\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323550","messageId":"20170630072611.q733inuxdvjmhjgw@ruderich.org","threadId":"46290","inReplyTo":"20170630000710.10601-26-sbeller@google.com","subject":"Re: [PATCH 25/25] diff: document the new --color-moved setting","fromName":"Simon Ruderich","fromEmail":"simon@ruderich.org","sentAt":"2017-06-30T07:26:11Z","receivedAt":"2017-06-30T07:26:17Z","isPatch":true,"sender":{"key":"simon@ruderich.org","avatar":"https://avatars.githubusercontent.com/u/390994?v=4"},"body":"On Thu, Jun 29, 2017 at 05:07:10PM -0700, Stefan Beller wrote:\n> +\tSmall blocks of 3 moved lines or fewer are skipped.\n\nIf I read the commit messages correctly, this \"skipping\" process\napplies to the move detection in general for those smaller blocks\nand therefore doesn't mean a malicious move can hide smaller\nchanges, correct? If so, I find this sentence misleading. Maybe\nsomething like:\n\n Small blocks of 3 moved lines or fewer are excluded from move\n detection and colored as regular diff.\n\nRegards\nSimon\n-- \n+ privacy is necessary\n+ using gnupg http://gnupg.org\n+ public key id: 0x92FEFDB7E44C32F9\n"},{"id":"323571","messageId":"CAGZ79kYONNMrB7L65WjwZjSuKOf3rAOWPwjb1bH26MU0owTJ4w@mail.gmail.com","threadId":"46290","inReplyTo":"20170630072611.q733inuxdvjmhjgw@ruderich.org","subject":"Re: [PATCH 25/25] diff: document the new --color-moved setting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T16:04:50Z","receivedAt":"2017-06-30T16:04:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jun 30, 2017 at 12:26 AM, Simon Ruderich wrote:\n> On Thu, Jun 29, 2017 at 05:07:10PM -0700, Stefan Beller wrote:\n>> + Small blocks of 3 moved lines or fewer are skipped.\n>\n> If I read the commit messages correctly, this \"skipping\" process\n> applies to the move detection in general for those smaller blocks\n> and therefore doesn't mean a malicious move can hide smaller\n> changes, correct? If so, I find this sentence misleading. Maybe\n> something like:\n>\n> Small blocks of 3 moved lines or fewer are excluded from move\n> detection and colored as regular diff.\n\nWell, this reads as if \"blocks of 3 lines\" are excluded, but what I\nmean is \"if all adjacent blocks combined are 3 lines or fewer\"\n\nExample of how I understand the code:\n\n context\n + moved line, block A\n + moved line, block A\n + moved line, block B\n + moved line, block A\n + moved line, block A\n context\n\nThese five lines are colored, because 5>3, but each\nindividual block is smaller than 3 lines. However we\nalready want to tell the reviewer that the middle line is not part of\na contiguous 5 line block, so we have to use alternative color\nin the middle.\n\nHowever\n\n context\n + moved line, block A or B\n + moved line, block A or B\n context\n\nis omitted, because the number of lines\nhere is fewer than 3 ignoring the block\ntype.\n\nMaybe\n\n If there are fewer than 3 adjacent lines of\n moved code, they are skipped.\n\nThanks,\nStefan\n\n>\n> Regards\n> Simon\n> --\n> + privacy is necessary\n> + using gnupg http://gnupg.org\n> + public key id: 0x92FEFDB7E44C32F9\n"},{"id":"323583","messageId":"xmqqd19ls6no.fsf@gitster.mtv.corp.google.com","threadId":"46290","inReplyTo":"20170630000710.10601-23-sbeller@google.com","subject":"Re: [PATCH 22/25] diff.c: color moved lines differently","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-30T17:54:19Z","receivedAt":"2017-06-30T17:54:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> +static int next_byte(const char **cp, const char **endp,\n> +\t\t const struct diff_options *diffopt)\n> +{\n> +\tint retval;\n> +\n> +\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_AT_EOL)) {\n> +\t\twhile (*endp > *cp && isspace(**endp))\n> +\t\t\t(*endp)--;\n> +\t}\n\nThis should be done by the callers (both moved_entry_cmp() and\nget_string_hash()) before starting to iterate over the bytes from\nthe beginning, no?\n\n> +\tif (*cp > *endp)\n> +\t\treturn -1;\n> +\n> +\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_CHANGE)) {\n> +\t\twhile (*cp < *endp && isspace(**cp))\n> +\t\t\t(*cp)++;\n> +\t\t/*\n> +\t\t * After skipping a couple of whitespaces, we still have to\n> +\t\t * account for one space.\n> +\t\t */\n> +\t\treturn (int)' ';\n> +\t}\n> +\n> +\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE)) {\n> +\t\twhile (*cp < *endp && isspace(**cp))\n> +\t\t\t(*cp)++;\n> +\t\t/* return the first non-ws character via the usual below */\n> +\t}\n> +\n> +\tretval = **cp;\n\nThe char could be signed, and byte 0xff may become indistinguishable\nfrom the EOF (i.e. -1) you returned earlier.\n\n> +\t(*cp)++; /* advance */\n> +\treturn retval;\n> +}\n> +\n> +static int moved_entry_cmp(const struct moved_entry *a,\n> +\t\t\t const struct moved_entry *b,\n> +\t\t\t const void *keydata,\n> +\t\t\t const void *data)\n> +{\n> +\tconst struct diff_options *diffopt = data;\n> +\tconst char *ap = a->es->line, *ae = a->es->line + a->es->len;\n> +\tconst char *bp = b->es->line, *be = b->es->line + b->es->len;\n> +\n> +\tif (!(diffopt->xdl_opts & XDF_WHITESPACE_FLAGS))\n> +\t\treturn a->es->len != b->es->len || memcmp(ap, bp, a->es->len);\n> +\n> +\twhile (1) {\n> +\t\tint ca, cb;\n> +\t\tca = next_byte(&ap, &ae, diffopt);\n> +\t\tcb = next_byte(&bp, &be, diffopt);\n> +\t\tif (ca != cb)\n> +\t\t\treturn 1; /* differs */\n> +\t\tif (!ca)\n\nShouldn't this check for \"ca == -1\", as we are not dealing with NUL\nterminated string but a thing?\n\n> +\t\t\treturn 0;\n> +\t};\n> +}\n> +\n> +static unsigned get_string_hash(struct emitted_diff_symbol *es, struct diff_options *o)\n> +{\n> +\tif (o->xdl_opts & XDF_WHITESPACE_FLAGS) {\n> +\t\tstatic struct strbuf sb = STRBUF_INIT;\n> +\t\tconst char *ap = es->line, *ae = es->line + es->len;\n> +\t\tint c;\n> +\n> +\t\tstrbuf_reset(&sb);\n> +\t\twhile ((c = next_byte(&ap, &ae, o)) > 0)\n> +\t\t\tstrbuf_addch(&sb, c);\n> +\n> +\t\treturn memhash(sb.buf, sb.len);\n> +\t} else {\n> +\t\treturn memhash(es->line, es->len);\n> +\t}\n> +}\n"},{"id":"323586","messageId":"CAGZ79kaK00CpXOtXnx_u7_KHbZq4Mz8vWHKy2a8p1gQ8ogE-OA@mail.gmail.com","threadId":"46290","inReplyTo":"xmqqd19ls6no.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 22/25] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T18:13:40Z","receivedAt":"2017-06-30T18:13:52Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jun 30, 2017 at 10:54 AM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> +static int next_byte(const char **cp, const char **endp,\n>> + const struct diff_options *diffopt)\n>> +{\n>> + int retval;\n>> +\n>> + if (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_AT_EOL)) {\n>> + while (*endp > *cp && isspace(**endp))\n>> + (*endp)--;\n>> + }\n>\n> This should be done by the callers (both moved_entry_cmp() and\n> get_string_hash()) before starting to iterate over the bytes from\n> the beginning, no?\n\nGood point.\n\n>> +\n>> + retval = **cp;\n>\n> The char could be signed, and byte 0xff may become indistinguishable\n> from the EOF (i.e. -1) you returned earlier.\n\nAh, I messed up there. I think EOF is wrong, too.\nSo maybe we'll just return 256 to indicate the end of memory chunk\nto not have to deal with signedness\n\n>> + if (ca != cb)\n>> + return 1; /* differs */\n>> + if (!ca)\n>\n> Shouldn't this check for \"ca == -1\", as we are not dealing with NUL\n> terminated string but a thing?\n\nYes, we'd check for the ending symbol instead of 0.\n"},{"id":"323591","messageId":"xmqqvandqpwx.fsf@gitster.mtv.corp.google.com","threadId":"46290","inReplyTo":"CAGZ79kaK00CpXOtXnx_u7_KHbZq4Mz8vWHKy2a8p1gQ8ogE-OA@mail.gmail.com","subject":"Re: [PATCH 22/25] diff.c: color moved lines differently","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-30T18:41:18Z","receivedAt":"2017-06-30T18:41:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> On Fri, Jun 30, 2017 at 10:54 AM, Junio C Hamano wrote:\n>> Stefan Beller writes:\n>>\n>>> +static int next_byte(const char **cp, const char **endp,\n>>> + const struct diff_options *diffopt)\n>>> +{\n>>> + int retval;\n>>> +\n>>> + if (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_AT_EOL)) {\n>>> + while (*endp > *cp && isspace(**endp))\n>>> + (*endp)--;\n>>> + }\n>>\n>> This should be done by the callers (both moved_entry_cmp() and\n>> get_string_hash()) before starting to iterate over the bytes from\n>> the beginning, no?\n>\n> Good point.\n>\n>>> +\n>>> + retval = **cp;\n>>\n>> The char could be signed, and byte 0xff may become indistinguishable\n>> from the EOF (i.e. -1) you returned earlier.\n>\n> Ah, I messed up there. I think EOF is wrong, too.\n> So maybe we'll just return 256 to indicate the end of memory chunk\n> to not have to deal with signedness\n\nI would just say that next_byte() returns -1 (at end of string) or\n0-255 if it is returning a byte that matters.\n"},{"id":"323606","messageId":"20170630203117.ktobhqnvgxgjujzi@ruderich.org","threadId":"46290","inReplyTo":"CAGZ79kYONNMrB7L65WjwZjSuKOf3rAOWPwjb1bH26MU0owTJ4w@mail.gmail.com","subject":"Re: [PATCH 25/25] diff: document the new --color-moved setting","fromName":"Simon Ruderich","fromEmail":"simon@ruderich.org","sentAt":"2017-06-30T20:31:17Z","receivedAt":"2017-06-30T20:31:24Z","isPatch":true,"sender":{"key":"simon@ruderich.org","avatar":"https://avatars.githubusercontent.com/u/390994?v=4"},"body":"On Fri, Jun 30, 2017 at 09:04:50AM -0700, Stefan Beller wrote:\n> [snip]\n>\n> However\n>\n> context\n> + moved line, block A or B\n> + moved line, block A or B\n> context\n>\n> is omitted, because the number of lines\n> here is fewer than 3 ignoring the block\n> type.\n>\n> Maybe\n>\n> If there are fewer than 3 adjacent lines of\n> moved code, they are skipped.\n\nMy issue with \"skipped\" is that it's not clear what exactly is\nskipped. Move detection? Inclusion in the diff? Something else?\nThat's why I tried to add the \"excluded from move detection\" to\nmake it clear that the change is still shown to the user, just\nnot handled by move detection and using the usual diff colors.\n\nIn your example above, what exactly is \"omitted\"? The complete\nhunk from the output or the special coloring? That's what\nconfuses me and what might confuse a future reader of the\ndocumentation.\n\nRegards\nSimon\n-- \n+ privacy is necessary\n+ using gnupg http://gnupg.org\n+ public key id: 0x92FEFDB7E44C32F9\n"},{"id":"323607","messageId":"CAGZ79kYejnJoSPMhaTwXy01dM2YC2FW0+_+tedTqvZ5Tqk+BGA@mail.gmail.com","threadId":"46290","inReplyTo":"20170630203117.ktobhqnvgxgjujzi@ruderich.org","subject":"Re: [PATCH 25/25] diff: document the new --color-moved setting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:33:31Z","receivedAt":"2017-06-30T20:33:37Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jun 30, 2017 at 1:31 PM, Simon Ruderich wrote:\n> On Fri, Jun 30, 2017 at 09:04:50AM -0700, Stefan Beller wrote:\n>> [snip]\n>>\n>> However\n>>\n>> context\n>> + moved line, block A or B\n>> + moved line, block A or B\n>> context\n>>\n>> is omitted, because the number of lines\n>> here is fewer than 3 ignoring the block\n>> type.\n>>\n>> Maybe\n>>\n>> If there are fewer than 3 adjacent lines of\n>> moved code, they are skipped.\n>\n> My issue with \"skipped\" is that it's not clear what exactly is\n> skipped. Move detection? Inclusion in the diff? Something else?\n> That's why I tried to add the \"excluded from move detection\" to\n> make it clear that the change is still shown to the user, just\n> not handled by move detection and using the usual diff colors.\n>\n> In your example above, what exactly is \"omitted\"? The complete\n> hunk from the output or the special coloring? That's what\n> confuses me and what might confuse a future reader of the\n> documentation.\n\nOh. I see. That was not the point of confusion that I thought.\n\nskipped/omitted both refer to the move detection and the lines\nwill be included in the diff, but just as regular added/removed lines.\n\nWill fix.\n\nThanks for bearing with me,\nStefan\n"},{"id":"323617","messageId":"20170630205310.7380-1-sbeller@google.com","threadId":"46290","inReplyTo":"20170630000710.10601-1-sbeller@google.com","subject":"[PATCHv2 00/25] Reroll of sb/diff-color-moved.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:45Z","receivedAt":"2017-06-30T20:53:21Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"v2:\n\n* on top of the new hashmap patches[3]. It compiles when put on older versions\n of the hashmap series, as the hashmap compare function just swapped void pointers\n in the definition.\n \n* better documentation, Thanks Simon!\n ... If there\n are fewer than 3 adjacent moved lines, they are not marked up\n as moved, but the regular colors 'color.diff.{old,new}' will be\n used. ...\n \n* clarified the compare function (casting the return to unsigned char, such\n that 0-255 is from the string, -1 indicated EOF. Adapted the checks for that.\n \nThanks,\nStefan\n\n[3] https://public-inbox.org/git/20170630191407.5381-1-sbeller@google.com/\n\nv1:\n\nReroll of sb/diff-color-moved, applies on top of the hashmap patches[1].\n\n* Supports different whitespace modes correctly now (and has a test for trivial\n white space changes)\n* The fixups[2] have been squashed.\n\nThanks,\nStefan\n\n[1] https://public-inbox.org/git/20170629235336.28460-1-sbeller@google.com/\n[2] https://public-inbox.org/git/20170628005651.8110-1-sbeller@google.com/\n\nStefan Beller (25):\n diff.c: readability fix\n diff.c: move line ending check into emit_hunk_header\n diff.c: factor out diff_flush_patch_all_file_pairs\n diff.c: introduce emit_diff_symbol\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF\n diff.c: migrate emit_line_checked to use emit_diff_symbol\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES\n diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF\n submodule.c: migrate diff output to use emit_diff_symbol\n diff.c: convert emit_binary_diff_body to use emit_diff_symbol\n diff.c: convert show_stats to use emit_diff_symbol\n diff.c: convert word diffing to use emit_diff_symbol\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP\n diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY\n diff.c: buffer all output if asked to\n diff.c: color moved lines differently\n diff.c: color moved lines differently, plain mode\n diff.c: add dimming to moved line detection\n diff: document the new --color-moved setting\n\n Documentation/config.txt | 15 +-\n Documentation/diff-options.txt | 36 ++\n cache.h | 2 +\n color.h | 2 +\n diff.c | 1309 ++++++++++++++++++++++++++++++++--------\n diff.h | 39 +-\n submodule.c | 84 ++-\n submodule.h | 13 +-\n t/t4015-diff-whitespace.sh | 434 +++++++++++++\n 9 files changed, 1618 insertions(+), 316 deletions(-)\n\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323618","messageId":"20170630205310.7380-2-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 01/25] diff.c: readability fix","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:46Z","receivedAt":"2017-06-30T20:53:23Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We already have dereferenced 'p->two' into a local variable 'two'.\nUse that.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 00b4c86698..2874dfc6fc 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3282,8 +3282,8 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \tconst char *other;\n \tconst char *attr_path;\n \n-\tname = p->one->path;\n-\tother = (strcmp(name, p->two->path) ? p->two->path : NULL);\n+\tname = one->path;\n+\tother = (strcmp(name, two->path) ? two->path : NULL);\n \tattr_path = name;\n \tif (o->prefix_length)\n \t\tstrip_prefix(o->prefix_length, &name, &other);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323619","messageId":"20170630205310.7380-3-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 02/25] diff.c: move line ending check into emit_hunk_header","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:47Z","receivedAt":"2017-06-30T20:53:24Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The emit_hunk_header() function is responsible for assembling a\nhunk header and calling emit_line() to send the hunk header\nto the output file. Its only caller fn_out_consume() needs\nto prepare for a case where the function emits an incomplete\nline and add the terminating LF.\n\nInstead make sure emit_hunk_header() to always send a\ncompleted line to emit_line().\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2874dfc6fc..94fdb57927 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -678,6 +678,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t}\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n+\tstrbuf_complete_line(&msgbuf);\n+\n \temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n@@ -1315,8 +1317,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\tlen = sane_truncate_line(ecbdata, line, len);\n \t\tfind_lno(line, ecbdata);\n \t\temit_hunk_header(ecbdata, line, len);\n-\t\tif (line[len-1] != '\\n')\n-\t\t\tputc('\\n', o->file);\n \t\treturn;\n \t}\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323620","messageId":"20170630205310.7380-7-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 06/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:51Z","receivedAt":"2017-06-30T20:53:28Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 75b996c4cf..6e48a129ed 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n@@ -570,6 +571,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n {\n \tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n+\t\temit_line(o, \"\", \"\", line, len);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_CONTEXT_MARKER:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -705,8 +709,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n-\n-\temit_line(ecbdata->opt, \"\", \"\", msgbuf.buf, msgbuf.len);\n+\temit_diff_symbol(ecbdata->opt,\n+\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);\n \tstrbuf_release(&msgbuf);\n }\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323621","messageId":"20170630205310.7380-8-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 07/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:52Z","receivedAt":"2017-06-30T20:53:31Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 19 +++++++++++--------\n 1 file changed, 11 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 6e48a129ed..488096b757 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_NO_LF_EOF,\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n@@ -569,8 +570,16 @@ enum diff_symbol {\n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len)\n {\n+\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_NO_LF_EOF:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tputc('\\n', o->file);\n+\t\temit_line_0(o, context, reset, '\\\\',\n+\t\t\t nneof, strlen(nneof));\n+\t\tbreak;\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -751,7 +760,6 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\t\t int prefix, const char *data, int size)\n {\n \tconst char *endp = NULL;\n-\tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);\n \n \twhile (0 < size) {\n@@ -769,13 +777,8 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tsize -= len;\n \t\tdata += len;\n \t}\n-\tif (!endp) {\n-\t\tconst char *context = diff_get_color(ecb->color_diff,\n-\t\t\t\t\t\t DIFF_CONTEXT);\n-\t\tputc('\\n', ecb->opt->file);\n-\t\temit_line_0(ecb->opt, context, reset, '\\\\',\n-\t\t\t nneof, strlen(nneof));\n-\t}\n+\tif (!endp)\n+\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);\n }\n \n static void emit_rewrite_diff(const char *name_a,\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323622","messageId":"20170630205310.7380-6-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 05/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_MARKER","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:50Z","receivedAt":"2017-06-30T20:53:32Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 10 +++++++++-\n 1 file changed, 9 insertions(+), 1 deletion(-)\n\ndiff --git a/diff.c b/diff.c\nindex 4637368d59..75b996c4cf 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,13 +561,20 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len)\n {\n+\tconst char *context, *reset;\n \tswitch (s) {\n+\tcase DIFF_SYMBOL_CONTEXT_MARKER:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_SEPARATOR:\n \t\tfprintf(o->file, \"%s%c\",\n \t\t\tdiff_line_prefix(o),\n@@ -662,7 +669,8 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tif (len < 10 ||\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n-\t\temit_line(ecbdata->opt, context, reset, line, len);\n+\t\temit_diff_symbol(ecbdata->opt,\n+\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323623","messageId":"20170630205310.7380-13-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 12/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:57Z","receivedAt":"2017-06-30T20:53:35Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The header is constructed lazily including line breaks, so just emit\nthe raw string as is.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 28 ++++++++++++++++++++--------\n 1 file changed, 20 insertions(+), 8 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 49b45fef29..78f7c6f82f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n \tDIFF_SYMBOL_FILEPAIR_MINUS,\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n@@ -689,6 +690,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tline, reset,\n \t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n+\tcase DIFF_SYMBOL_HEADER:\n+\t\tfprintf(o->file, \"%s\", line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1385,7 +1389,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \to->found_changes = 1;\n \n \tif (ecbdata->header) {\n-\t\tfprintf(o->file, \"%s\", ecbdata->header->buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t ecbdata->header->buf, ecbdata->header->len, 0);\n \t\tstrbuf_reset(ecbdata->header);\n \t\tecbdata->header = NULL;\n \t}\n@@ -2519,7 +2524,8 @@ static void builtin_diff(const char *name_a,\n \t\tif (complete_rewrite &&\n \t\t (textconv_one || !diff_filespec_is_binary(one)) &&\n \t\t (textconv_two || !diff_filespec_is_binary(two))) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tstrbuf_reset(&header);\n \t\t\temit_rewrite_diff(name_a, name_b, one, two,\n \t\t\t\t\t\ttextconv_one, textconv_two, o);\n@@ -2529,7 +2535,8 @@ static void builtin_diff(const char *name_a,\n \t}\n \n \tif (o->irreversible_delete && lbl[1][0] == '/') {\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf,\n+\t\t\t\t header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tgoto free_ab_and_return;\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n@@ -2540,10 +2547,13 @@ static void builtin_diff(const char *name_a,\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n \t\t\tif (!oidcmp(&one->oid, &two->oid)) {\n \t\t\t\tif (must_show_header)\n-\t\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t\t\t header.buf, header.len,\n+\t\t\t\t\t\t\t 0);\n \t\t\t\tgoto free_ab_and_return;\n \t\t\t}\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n \t\t\t\tline_prefix, lbl[0], lbl[1]);\n \t\t\tgoto free_ab_and_return;\n@@ -2554,10 +2564,11 @@ static void builtin_diff(const char *name_a,\n \t\tif (mf1.size == mf2.size &&\n \t\t !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {\n \t\t\tif (must_show_header)\n-\t\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n-\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n@@ -2575,7 +2586,8 @@ static void builtin_diff(const char *name_a,\n \t\tconst struct userdiff_funcname *pe;\n \n \t\tif (must_show_header) {\n-\t\t\tfprintf(o->file, \"%s\", header.buf);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n+\t\t\t\t\t header.buf, header.len, 0);\n \t\t\tstrbuf_reset(&header);\n \t\t}\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323624","messageId":"20170630205310.7380-14-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 13/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:58Z","receivedAt":"2017-06-30T20:53:37Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"we could save a little bit of memory when buffering in a later mode\nby just passing the inner part (\"%s and %s\", file1, file 2), but\nthose a just a few bytes, so instead let's reuse the implementation from\nDIFF_SYMBOL_HEADER and keep the whole line around.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 20 +++++++++++++++-----\n 1 file changed, 15 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 78f7c6f82f..1d0f8b509f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n \tDIFF_SYMBOL_FILEPAIR_MINUS,\n@@ -690,6 +691,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tline, reset,\n \t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n \t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_FILES:\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n@@ -2542,6 +2544,7 @@ static void builtin_diff(const char *name_a,\n \t} else if (!DIFF_OPT_TST(o, TEXT) &&\n \t ( (!textconv_one && diff_filespec_is_binary(one)) ||\n \t (!textconv_two && diff_filespec_is_binary(two)) )) {\n+\t\tstruct strbuf sb = STRBUF_INIT;\n \t\tif (!one->data && !two->data &&\n \t\t S_ISREG(one->mode) && S_ISREG(two->mode) &&\n \t\t !DIFF_OPT_TST(o, BINARY)) {\n@@ -2554,8 +2557,11 @@ static void builtin_diff(const char *name_a,\n \t\t\t}\n \t\t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER,\n \t\t\t\t\t header.buf, header.len, 0);\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n+\t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_release(&sb);\n \t\t\tgoto free_ab_and_return;\n \t\t}\n \t\tif (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)\n@@ -2572,9 +2578,13 @@ static void builtin_diff(const char *name_a,\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n \t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n-\t\telse\n-\t\t\tfprintf(o->file, \"%sBinary files %s and %s differ\\n\",\n-\t\t\t\tline_prefix, lbl[0], lbl[1]);\n+\t\telse {\n+\t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n+\t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_release(&sb);\n+\t\t}\n \t\to->found_changes = 1;\n \t} else {\n \t\t/* Crazy xdl interfaces.. */\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323625","messageId":"20170630205310.7380-15-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 14/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:59Z","receivedAt":"2017-06-30T20:53:41Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 35 +++++++++++++++++++++--------------\n 1 file changed, 21 insertions(+), 14 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 1d0f8b509f..5c428e02b6 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,7 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_REWRITE_DIFF,\n \tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n \tDIFF_SYMBOL_FILEPAIR_PLUS,\n@@ -615,7 +616,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset, *set, *meta;\n+\tconst char *context, *reset, *set, *meta, *fraginfo;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -695,6 +696,11 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_REWRITE_DIFF:\n+\t\tfraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, fraginfo, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -817,17 +823,17 @@ static void remove_tempfile(void)\n \t}\n }\n \n-static void print_line_count(FILE *file, int count)\n+static void add_line_count(struct strbuf *out, int count)\n {\n \tswitch (count) {\n \tcase 0:\n-\t\tfprintf(file, \"0,0\");\n+\t\tstrbuf_addstr(out, \"0,0\");\n \t\tbreak;\n \tcase 1:\n-\t\tfprintf(file, \"1\");\n+\t\tstrbuf_addstr(out, \"1\");\n \t\tbreak;\n \tdefault:\n-\t\tfprintf(file, \"1,%d\", count);\n+\t\tstrbuf_addf(out, \"1,%d\", count);\n \t\tbreak;\n \t}\n }\n@@ -866,14 +872,12 @@ static void emit_rewrite_diff(const char *name_a,\n \t\t\t struct diff_options *o)\n {\n \tint lc_a, lc_b;\n-\tconst char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n-\tconst char *reset = diff_get_color(o->use_color, DIFF_RESET);\n \tstatic struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;\n \tconst char *a_prefix, *b_prefix;\n \tchar *data_one, *data_two;\n \tsize_t size_one, size_two;\n \tstruct emit_callback ecbdata;\n-\tconst char *line_prefix = diff_line_prefix(o);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\ta_prefix = o->b_prefix;\n@@ -917,14 +921,17 @@ static void emit_rewrite_diff(const char *name_a,\n \temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n \t\t\t b_name.buf, b_name.len, 0);\n \n-\tfprintf(o->file, \"%s%s@@ -\", line_prefix, fraginfo);\n+\tstrbuf_addstr(&out, \"@@ -\");\n \tif (!o->irreversible_delete)\n-\t\tprint_line_count(o->file, lc_a);\n+\t\tadd_line_count(&out, lc_a);\n \telse\n-\t\tfprintf(o->file, \"?,?\");\n-\tfprintf(o->file, \" +\");\n-\tprint_line_count(o->file, lc_b);\n-\tfprintf(o->file, \" @@%s\\n\", reset);\n+\t\tstrbuf_addstr(&out, \"?,?\");\n+\tstrbuf_addstr(&out, \" +\");\n+\tadd_line_count(&out, lc_b);\n+\tstrbuf_addstr(&out, \" @@\\n\");\n+\temit_diff_symbol(o, DIFF_SYMBOL_REWRITE_DIFF, out.buf, out.len, 0);\n+\tstrbuf_release(&out);\n+\n \tif (lc_a && !o->irreversible_delete)\n \t\temit_rewrite_lines(&ecbdata, '-', data_one, size_one);\n \tif (lc_b)\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323626","messageId":"20170630205310.7380-10-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 09/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:54Z","receivedAt":"2017-06-30T20:53:42Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 42 ++++++++++++++++++++++++++----------------\n 1 file changed, 26 insertions(+), 16 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex e5430d56da..85bfd9310d 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,8 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_WORDS_PORCELAIN,\n+\tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n \tDIFF_SYMBOL_PLUS,\n \tDIFF_SYMBOL_MINUS,\n@@ -649,6 +651,26 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_WORDS_PORCELAIN:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, context, reset, line, len);\n+\t\tfputs(\"~\\n\", o->file);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_WORDS:\n+\t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\t/*\n+\t\t * Skip the prefix character, if any. With\n+\t\t * diff_suppress_blank_empty, there may be\n+\t\t * none.\n+\t\t */\n+\t\tif (line[0] != '\\n') {\n+\t\t\tline++;\n+\t\t\tlen--;\n+\t\t}\n+\t\temit_line(o, context, reset, line, len);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1342,7 +1364,6 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n {\n \tstruct emit_callback *ecbdata = priv;\n \tconst char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);\n-\tconst char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n \tconst char *line_prefix = diff_line_prefix(o);\n@@ -1384,6 +1405,9 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->diff_words) {\n+\t\tenum diff_symbol s =\n+\t\t\tecbdata->diff_words->type == DIFF_WORDS_PORCELAIN ?\n+\t\t\tDIFF_SYMBOL_WORDS_PORCELAIN : DIFF_SYMBOL_WORDS;\n \t\tif (line[0] == '-') {\n \t\t\tdiff_words_append(line, len,\n \t\t\t\t\t &ecbdata->diff_words->minus);\n@@ -1403,21 +1427,7 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t\t\treturn;\n \t\t}\n \t\tdiff_words_flush(ecbdata);\n-\t\tif (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {\n-\t\t\temit_line(o, context, reset, line, len);\n-\t\t\tfputs(\"~\\n\", o->file);\n-\t\t} else {\n-\t\t\t/*\n-\t\t\t * Skip the prefix character, if any. With\n-\t\t\t * diff_suppress_blank_empty, there may be\n-\t\t\t * none.\n-\t\t\t */\n-\t\t\tif (line[0] != '\\n') {\n-\t\t\t line++;\n-\t\t\t len--;\n-\t\t\t}\n-\t\t\temit_line(o, context, reset, line, len);\n-\t\t}\n+\t\temit_diff_symbol(o, s, line, len, 0);\n \t\treturn;\n \t}\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323627","messageId":"20170630205310.7380-21-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 20/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:05Z","receivedAt":"2017-06-30T20:53:46Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 71 ++++++++++++++++++++++++++++++++++++++----------------------------\n 1 file changed, 41 insertions(+), 30 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 5a9c55736d..2db0d7c0f5 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -572,6 +572,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_WORD_DIFF,\n \tDIFF_SYMBOL_STAT_SEP,\n+\tDIFF_SYMBOL_SUMMARY,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -648,6 +649,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n \tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n+\tcase DIFF_SYMBOL_SUMMARY:\n \tcase DIFF_SYMBOL_STATS_LINE:\n \tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n@@ -4717,67 +4719,76 @@ static void flush_one_pair(struct diff_filepair *p, struct diff_options *opt)\n \t}\n }\n \n-static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)\n+static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tif (fs->mode)\n-\t\tfprintf(file, \" %s mode %06o \", newdelete, fs->mode);\n+\t\tstrbuf_addf(&sb, \" %s mode %06o \", newdelete, fs->mode);\n \telse\n-\t\tfprintf(file, \" %s \", newdelete);\n-\twrite_name_quoted(fs->path, file, '\\n');\n-}\n+\t\tstrbuf_addf(&sb, \" %s \", newdelete);\n \n+\tquote_c_style(fs->path, &sb, NULL, 0);\n+\tstrbuf_addch(&sb, '\\n');\n+\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t sb.buf, sb.len, 0);\n+\tstrbuf_release(&sb);\n+}\n \n-static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,\n-\t\tconst char *line_prefix)\n+static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,\n+\t\tint show_name)\n {\n \tif (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {\n-\t\tfprintf(file, \"%s mode change %06o => %06o%c\", line_prefix, p->one->mode,\n-\t\t\tp->two->mode, show_name ? ' ' : '\\n');\n+\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\tstrbuf_addf(&sb, \" mode change %06o => %06o\",\n+\t\t\t p->one->mode, p->two->mode);\n \t\tif (show_name) {\n-\t\t\twrite_name_quoted(p->two->path, file, '\\n');\n+\t\t\tstrbuf_addch(&sb, ' ');\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n \t\t}\n+\t\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\t\tstrbuf_release(&sb);\n \t}\n }\n \n-static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,\n-\t\t\tconst char *line_prefix)\n+static void show_rename_copy(struct diff_options *opt, const char *renamecopy,\n+\t\tstruct diff_filepair *p)\n {\n+\tstruct strbuf sb = STRBUF_INIT;\n \tchar *names = pprint_rename(p->one->path, p->two->path);\n-\n-\tfprintf(file, \" %s %s (%d%%)\\n\", renamecopy, names, similarity_index(p));\n+\tstrbuf_addf(&sb, \" %s %s (%d%%)\\n\",\n+\t\t\trenamecopy, names, similarity_index(p));\n \tfree(names);\n-\tshow_mode_change(file, p, 0, line_prefix);\n+\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\tshow_mode_change(opt, p, 0);\n }\n \n static void diff_summary(struct diff_options *opt, struct diff_filepair *p)\n {\n-\tFILE *file = opt->file;\n-\tconst char *line_prefix = diff_line_prefix(opt);\n-\n \tswitch(p->status) {\n \tcase DIFF_STATUS_DELETED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"delete\", p->one);\n+\t\tshow_file_mode_name(opt, \"delete\", p->one);\n \t\tbreak;\n \tcase DIFF_STATUS_ADDED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_file_mode_name(file, \"create\", p->two);\n+\t\tshow_file_mode_name(opt, \"create\", p->two);\n \t\tbreak;\n \tcase DIFF_STATUS_COPIED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"copy\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"copy\", p);\n \t\tbreak;\n \tcase DIFF_STATUS_RENAMED:\n-\t\tfputs(line_prefix, file);\n-\t\tshow_rename_copy(file, \"rename\", p, line_prefix);\n+\t\tshow_rename_copy(opt, \"rename\", p);\n \t\tbreak;\n \tdefault:\n \t\tif (p->score) {\n-\t\t\tfprintf(file, \"%s rewrite \", line_prefix);\n-\t\t\twrite_name_quoted(p->two->path, file, ' ');\n-\t\t\tfprintf(file, \"(%d%%)\\n\", similarity_index(p));\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addstr(&sb, \" rewrite \");\n+\t\t\tquote_c_style(p->two->path, &sb, NULL, 0);\n+\t\t\tstrbuf_addf(&sb, \" (%d%%)\\n\", similarity_index(p));\n+\t\t\temit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n \t\t}\n-\t\tshow_mode_change(file, p, !p->score, line_prefix);\n+\t\tshow_mode_change(opt, p, !p->score);\n \t\tbreak;\n \t}\n }\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323628","messageId":"20170630205310.7380-19-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 18/25] diff.c: convert word diffing to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:03Z","receivedAt":"2017-06-30T20:53:48Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The word diffing is not line oriented and would need some serious\neffort to be transformed into a line oriented approach, so\njust go with a symbol DIFF_SYMBOL_WORD_DIFF that is a partial line.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 79 ++++++++++++++++++++++++++++++++++++++----------------------------\n 1 file changed, 46 insertions(+), 33 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 76d4b8ebf9..e0d39d04da 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -570,6 +570,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n \tDIFF_SYMBOL_STATS_LINE,\n+\tDIFF_SYMBOL_WORD_DIFF,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -762,6 +763,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n \t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n \t\tbreak;\n+\tcase DIFF_SYMBOL_WORD_DIFF:\n+\t\tfprintf(o->file, \"%.*s\", len, line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -1092,37 +1096,49 @@ struct diff_words_data {\n \tstruct diff_words_style *style;\n };\n \n-static int fn_out_diff_words_write_helper(FILE *fp,\n+static int fn_out_diff_words_write_helper(struct diff_options *o,\n \t\t\t\t\t struct diff_words_style_elem *st_el,\n \t\t\t\t\t const char *newline,\n-\t\t\t\t\t size_t count, const char *buf,\n-\t\t\t\t\t const char *line_prefix)\n+\t\t\t\t\t size_t count, const char *buf)\n {\n \tint print = 0;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n \twhile (count) {\n \t\tchar *p = memchr(buf, '\\n', count);\n \t\tif (print)\n-\t\t\tfputs(line_prefix, fp);\n+\t\t\tstrbuf_addstr(&sb, diff_line_prefix(o));\n+\n \t\tif (p != buf) {\n-\t\t\tif (st_el->color && fputs(st_el->color, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (fputs(st_el->prefix, fp) < 0 ||\n-\t\t\t fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||\n-\t\t\t fputs(st_el->suffix, fp) < 0)\n-\t\t\t\treturn -1;\n-\t\t\tif (st_el->color && *st_el->color\n-\t\t\t && fputs(GIT_COLOR_RESET, fp) < 0)\n-\t\t\t\treturn -1;\n+\t\t\tconst char *reset = st_el->color && *st_el->color ?\n+\t\t\t\t\t GIT_COLOR_RESET : NULL;\n+\t\t\tif (st_el->color && *st_el->color)\n+\t\t\t\tstrbuf_addstr(&sb, st_el->color);\n+\t\t\tstrbuf_addstr(&sb, st_el->prefix);\n+\t\t\tstrbuf_add(&sb, buf, p ? p - buf : count);\n+\t\t\tstrbuf_addstr(&sb, st_el->suffix);\n+\t\t\tif (reset)\n+\t\t\t\tstrbuf_addstr(&sb, reset);\n \t\t}\n \t\tif (!p)\n-\t\t\treturn 0;\n-\t\tif (fputs(newline, fp) < 0)\n-\t\t\treturn -1;\n+\t\t\tgoto out;\n+\n+\t\tstrbuf_addstr(&sb, newline);\n \t\tcount -= p + 1 - buf;\n \t\tbuf = p + 1;\n \t\tprint = 1;\n+\t\tif (count) {\n+\t\t\temit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t\t sb.buf, sb.len, 0);\n+\t\t\tstrbuf_reset(&sb);\n+\t\t}\n \t}\n+\n+out:\n+\tif (sb.len)\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t sb.buf, sb.len, 0);\n+\tstrbuf_release(&sb);\n \treturn 0;\n }\n \n@@ -1204,24 +1220,20 @@ static void fn_out_diff_words_aux(void *priv, char *line, unsigned long len)\n \t\tfputs(line_prefix, diff_words->opt->file);\n \t}\n \tif (diff_words->current_plus != plus_begin) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->ctx, style->newline,\n \t\t\t\tplus_begin - diff_words->current_plus,\n-\t\t\t\tdiff_words->current_plus, line_prefix);\n-\t\tif (*(plus_begin - 1) == '\\n')\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n+\t\t\t\tdiff_words->current_plus);\n \t}\n \tif (minus_begin != minus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->old, style->newline,\n-\t\t\t\tminus_end - minus_begin, minus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tminus_end - minus_begin, minus_begin);\n \t}\n \tif (plus_begin != plus_end) {\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t\t&style->new, style->newline,\n-\t\t\t\tplus_end - plus_begin, plus_begin,\n-\t\t\t\tline_prefix);\n+\t\t\t\tplus_end - plus_begin, plus_begin);\n \t}\n \n \tdiff_words->current_plus = plus_end;\n@@ -1315,11 +1327,12 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \n \t/* special case: only removal */\n \tif (!diff_words->plus.text.size) {\n-\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\temit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t line_prefix, strlen(line_prefix), 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->old, style->newline,\n \t\t\tdiff_words->minus.text.size,\n-\t\t\tdiff_words->minus.text.ptr, line_prefix);\n+\t\t\tdiff_words->minus.text.ptr);\n \t\tdiff_words->minus.text.size = 0;\n \t\treturn;\n \t}\n@@ -1342,12 +1355,12 @@ static void diff_words_show(struct diff_words_data *diff_words)\n \tif (diff_words->current_plus != diff_words->plus.text.ptr +\n \t\t\tdiff_words->plus.text.size) {\n \t\tif (color_words_output_graph_prefix(diff_words))\n-\t\t\tfputs(line_prefix, diff_words->opt->file);\n-\t\tfn_out_diff_words_write_helper(diff_words->opt->file,\n+\t\t\temit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,\n+\t\t\t\t\t line_prefix, strlen(line_prefix), 0);\n+\t\tfn_out_diff_words_write_helper(diff_words->opt,\n \t\t\t&style->ctx, style->newline,\n \t\t\tdiff_words->plus.text.ptr + diff_words->plus.text.size\n-\t\t\t- diff_words->current_plus, diff_words->current_plus,\n-\t\t\tline_prefix);\n+\t\t\t- diff_words->current_plus, diff_words->current_plus);\n \t}\n \tdiff_words->minus.text.size = diff_words->plus.text.size = 0;\n }\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323629","messageId":"20170630205310.7380-18-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 17/25] diff.c: convert show_stats to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:02Z","receivedAt":"2017-06-30T20:53:52Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We call print_stat_summary from builtin/apply, so we still\nneed the version with a file pointer, so introduce\nprint_stat_summary_0 that uses emit_string machinery and\nkeep print_stat_summary with the same arguments around.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 114 +++++++++++++++++++++++++++++++++++++++++------------------------\n diff.h | 4 +--\n 2 files changed, 74 insertions(+), 44 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex f5a14359ae..76d4b8ebf9 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -566,6 +566,10 @@ enum diff_symbol {\n \tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n \tDIFF_SYMBOL_BINARY_DIFF_BODY,\n \tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n+\tDIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n+\tDIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n+\tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n+\tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -629,6 +633,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n+\tstruct strbuf sb = STRBUF_INIT;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -640,6 +645,8 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:\n+\tcase DIFF_SYMBOL_STATS_LINE:\n \tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n@@ -748,9 +755,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\tfprintf(o->file, \"%sSubmodule %s contains modified content\\n\",\n \t\t\tdiff_line_prefix(o), line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:\n+\t\temit_line(o, \"\", \"\", \" 0 files changed\\n\",\n+\t\t\t strlen(\" 0 files changed\\n\"));\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_STATS_SUMMARY_ABBREV:\n+\t\temit_line(o, \"\", \"\", \" ...\\n\", strlen(\" ...\\n\"));\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n+\tstrbuf_release(&sb);\n }\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line)\n@@ -1705,20 +1720,14 @@ static int scale_linear(int it, int width, int max_change)\n \treturn 1 + (it * (width - 1) / max_change);\n }\n \n-static void show_name(FILE *file,\n-\t\t const char *prefix, const char *name, int len)\n-{\n-\tfprintf(file, \" %s%-*s |\", prefix, len, name);\n-}\n-\n-static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)\n+static void show_graph(struct strbuf *out, char ch, int cnt,\n+\t\t const char *set, const char *reset)\n {\n \tif (cnt <= 0)\n \t\treturn;\n-\tfprintf(file, \"%s\", set);\n-\twhile (cnt--)\n-\t\tputc(ch, file);\n-\tfprintf(file, \"%s\", reset);\n+\tstrbuf_addstr(out, set);\n+\tstrbuf_addchars(out, ch, cnt);\n+\tstrbuf_addstr(out, reset);\n }\n \n static void fill_print_name(struct diffstat_file *file)\n@@ -1742,14 +1751,16 @@ static void fill_print_name(struct diffstat_file *file)\n \tfile->print_name = pname;\n }\n \n-int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n+static void print_stat_summary_inserts_deletes(struct diff_options *options,\n+\t\tint files, int insertions, int deletions)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n-\tint ret;\n \n \tif (!files) {\n \t\tassert(insertions == 0 && deletions == 0);\n-\t\treturn fprintf(fp, \"%s\\n\", \" 0 files changed\");\n+\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,\n+\t\t\t\t NULL, 0, 0);\n+\t\treturn;\n \t}\n \n \tstrbuf_addf(&sb,\n@@ -1776,9 +1787,19 @@ int print_stat_summary(FILE *fp, int files, int insertions, int deletions)\n \t\t\t deletions);\n \t}\n \tstrbuf_addch(&sb, '\\n');\n-\tret = fputs(sb.buf, fp);\n+\temit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n+\t\t\t sb.buf, sb.len, 0);\n \tstrbuf_release(&sb);\n-\treturn ret;\n+}\n+\n+void print_stat_summary(FILE *fp, int files,\n+\t\t\tint insertions, int deletions)\n+{\n+\tstruct diff_options o;\n+\tmemset(&o, 0, sizeof(o));\n+\to.file = fp;\n+\n+\tprint_stat_summary_inserts_deletes(&o, files, insertions, deletions);\n }\n \n static void show_stats(struct diffstat_t *data, struct diff_options *options)\n@@ -1788,13 +1809,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \tint total_files = data->nr, count;\n \tint width, name_width, graph_width, number_width = 0, bin_width = 0;\n \tconst char *reset, *add_c, *del_c;\n-\tconst char *line_prefix = \"\";\n \tint extra_shown = 0;\n+\tconst char *line_prefix = diff_line_prefix(options);\n+\tstruct strbuf out = STRBUF_INIT;\n \n \tif (data->nr == 0)\n \t\treturn;\n \n-\tline_prefix = diff_line_prefix(options);\n \tcount = options->stat_count ? options->stat_count : data->nr;\n \n \treset = diff_get_color_opt(options, DIFF_RESET);\n@@ -1948,26 +1969,32 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t}\n \n \t\tif (file->is_binary) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" %*s\", number_width, \"Bin\");\n+\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\t\tstrbuf_addf(&out, \" %*s\", number_width, \"Bin\");\n \t\t\tif (!added && !deleted) {\n-\t\t\t\tputc('\\n', options->file);\n+\t\t\t\tstrbuf_addch(&out, '\\n');\n+\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\t\tstrbuf_reset(&out);\n \t\t\t\tcontinue;\n \t\t\t}\n-\t\t\tfprintf(options->file, \" %s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addf(&out, \" %s%\"PRIuMAX\"%s\",\n \t\t\t\tdel_c, deleted, reset);\n-\t\t\tfprintf(options->file, \" -> \");\n-\t\t\tfprintf(options->file, \"%s%\"PRIuMAX\"%s\",\n+\t\t\tstrbuf_addstr(&out, \" -> \");\n+\t\t\tstrbuf_addf(&out, \"%s%\"PRIuMAX\"%s\",\n \t\t\t\tadd_c, added, reset);\n-\t\t\tfprintf(options->file, \" bytes\");\n-\t\t\tfprintf(options->file, \"\\n\");\n+\t\t\tstrbuf_addstr(&out, \" bytes\\n\");\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \t\telse if (file->is_unmerged) {\n-\t\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\t\tshow_name(options->file, prefix, name, len);\n-\t\t\tfprintf(options->file, \" Unmerged\\n\");\n+\t\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\t\tstrbuf_addstr(&out, \" Unmerged\\n\");\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t\t out.buf, out.len, 0);\n+\t\t\tstrbuf_reset(&out);\n \t\t\tcontinue;\n \t\t}\n \n@@ -1990,14 +2017,16 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\t\t\tadd = total - del;\n \t\t\t}\n \t\t}\n-\t\tfprintf(options->file, \"%s\", line_prefix);\n-\t\tshow_name(options->file, prefix, name, len);\n-\t\tfprintf(options->file, \" %*\"PRIuMAX\"%s\",\n+\t\tstrbuf_addf(&out, \" %s%-*s |\", prefix, len, name);\n+\t\tstrbuf_addf(&out, \" %*\"PRIuMAX\"%s\",\n \t\t\tnumber_width, added + deleted,\n \t\t\tadded + deleted ? \" \" : \"\");\n-\t\tshow_graph(options->file, '+', add, add_c, reset);\n-\t\tshow_graph(options->file, '-', del, del_c, reset);\n-\t\tfprintf(options->file, \"\\n\");\n+\t\tshow_graph(&out, '+', add, add_c, reset);\n+\t\tshow_graph(&out, '-', del, del_c, reset);\n+\t\tstrbuf_addch(&out, '\\n');\n+\t\temit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,\n+\t\t\t\t out.buf, out.len, 0);\n+\t\tstrbuf_reset(&out);\n \t}\n \n \tfor (i = 0; i < data->nr; i++) {\n@@ -2018,11 +2047,13 @@ static void show_stats(struct diffstat_t *data, struct diff_options *options)\n \t\tif (i < count)\n \t\t\tcontinue;\n \t\tif (!extra_shown)\n-\t\t\tfprintf(options->file, \"%s ...\\n\", line_prefix);\n+\t\t\temit_diff_symbol(options,\n+\t\t\t\t\t DIFF_SYMBOL_STATS_SUMMARY_ABBREV,\n+\t\t\t\t\t NULL, 0, 0);\n \t\textra_shown = 1;\n \t}\n-\tfprintf(options->file, \"%s\", line_prefix);\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\n+\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n }\n \n static void show_shortstats(struct diffstat_t *data, struct diff_options *options)\n@@ -2034,7 +2065,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \n \tfor (i = 0; i < data->nr; i++) {\n \t\tint added = data->files[i]->added;\n-\t\tint deleted= data->files[i]->deleted;\n+\t\tint deleted = data->files[i]->deleted;\n \n \t\tif (data->files[i]->is_unmerged ||\n \t\t (!data->files[i]->is_interesting && (added + deleted == 0))) {\n@@ -2044,8 +2075,7 @@ static void show_shortstats(struct diffstat_t *data, struct diff_options *option\n \t\t\tdels += deleted;\n \t\t}\n \t}\n-\tfprintf(options->file, \"%s\", diff_line_prefix(options));\n-\tprint_stat_summary(options->file, total_files, adds, dels);\n+\tprint_stat_summary_inserts_deletes(options, total_files, adds, dels);\n }\n \n static void show_numstat(struct diffstat_t *data, struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex ae40bac7a9..d6094a1eed 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -405,8 +405,8 @@ extern int parse_rename_score(const char **cp_p);\n \n extern long parse_algorithm_value(const char *value);\n \n-extern int print_stat_summary(FILE *fp, int files,\n-\t\t\t int insertions, int deletions);\n+extern void print_stat_summary(FILE *fp, int files,\n+\t\t\t int insertions, int deletions);\n extern void setup_diff_pager(struct diff_options *);\n \n #endif /* DIFF_H */\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323630","messageId":"20170630205310.7380-16-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 15/25] submodule.c: migrate diff output to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:00Z","receivedAt":"2017-06-30T20:53:54Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"As the submodule process is no longer attached to the same file pointer\n'o->file' as the superprojects process, there is a different result in\ncolor.c::check_auto_color. That is why we need to pass coloring explicitly,\nsuch that the submodule coloring decision will be made by the child process\nprocessing the submodule. Only DIFF_SYMBOL_SUBMODULE_PIPETHROUGH contains\ncolor, the other symbols are for embedding the submodule output into the\nsuperprojects output.\n\nRemove the colors from the function signatures, as all the coloring\ndecisions will be made either inside the child process or the final\nemit_diff_symbol, but not in the functions driving the submodule diff.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++---------\n diff.h | 9 +++++++\n submodule.c | 84 +++++++++++++++++++++++++++----------------------------------\n submodule.h | 13 +++-------\n 4 files changed, 121 insertions(+), 68 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 5c428e02b6..48f719fb07 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,13 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_SUBMODULE_ADD,\n+\tDIFF_SYMBOL_SUBMODULE_DEL,\n+\tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\tDIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\tDIFF_SYMBOL_SUBMODULE_HEADER,\n+\tDIFF_SYMBOL_SUBMODULE_ERROR,\n+\tDIFF_SYMBOL_SUBMODULE_PIPETHROUGH,\n \tDIFF_SYMBOL_REWRITE_DIFF,\n \tDIFF_SYMBOL_BINARY_FILES,\n \tDIFF_SYMBOL_HEADER,\n@@ -625,6 +632,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\temit_line_0(o, context, reset, '\\\\',\n \t\t\t nneof, strlen(nneof));\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n+\tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n+\tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -701,11 +711,68 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line(o, fraginfo, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_ADD:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, set, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_DEL:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line(o, set, reset, line, len);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_UNTRACKED:\n+\t\tfprintf(o->file, \"%sSubmodule %s contains untracked content\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_SUBMODULE_MODIFIED:\n+\t\tfprintf(o->file, \"%sSubmodule %s contains modified content\\n\",\n+\t\t\tdiff_line_prefix(o), line);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n }\n \n+void diff_emit_submodule_del(struct diff_options *o, const char *line)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);\n+}\n+\n+void diff_emit_submodule_add(struct diff_options *o, const char *line)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ADD, line, strlen(line), 0);\n+}\n+\n+void diff_emit_submodule_untracked(struct diff_options *o, const char *path)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_UNTRACKED,\n+\t\t\t path, strlen(path), 0);\n+}\n+\n+void diff_emit_submodule_modified(struct diff_options *o, const char *path)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_MODIFIED,\n+\t\t\t path, strlen(path), 0);\n+}\n+\n+void diff_emit_submodule_header(struct diff_options *o, const char *header)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_HEADER,\n+\t\t\t header, strlen(header), 0);\n+}\n+\n+void diff_emit_submodule_error(struct diff_options *o, const char *err)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ERROR, err, strlen(err), 0);\n+}\n+\n+void diff_emit_submodule_pipethrough(struct diff_options *o,\n+\t\t\t\t const char *line, int len)\n+{\n+\temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_PIPETHROUGH, line, len, 0);\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -2467,24 +2534,16 @@ static void builtin_diff(const char *name_a,\n \tif (o->submodule_format == DIFF_SUBMODULE_LOG &&\n \t (!one->mode || S_ISGITLINK(one->mode)) &&\n \t (!two->mode || S_ISGITLINK(two->mode))) {\n-\t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n-\t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_summary(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_summary(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n-\t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset);\n+\t\t\t\ttwo->dirty_submodule);\n \t\treturn;\n \t} else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&\n \t\t (!one->mode || S_ISGITLINK(one->mode)) &&\n \t\t (!two->mode || S_ISGITLINK(two->mode))) {\n-\t\tconst char *del = diff_get_color_opt(o, DIFF_FILE_OLD);\n-\t\tconst char *add = diff_get_color_opt(o, DIFF_FILE_NEW);\n-\t\tshow_submodule_inline_diff(o->file, one->path ? one->path : two->path,\n-\t\t\t\tline_prefix,\n+\t\tshow_submodule_inline_diff(o, one->path ? one->path : two->path,\n \t\t\t\t&one->oid, &two->oid,\n-\t\t\t\ttwo->dirty_submodule,\n-\t\t\t\tmeta, del, add, reset, o);\n+\t\t\t\ttwo->dirty_submodule);\n \t\treturn;\n \t}\n \ndiff --git a/diff.h b/diff.h\nindex ea66168454..ae40bac7a9 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,15 @@ struct diff_options {\n \tint diff_path_counter;\n };\n \n+void diff_emit_submodule_del(struct diff_options *o, const char *line);\n+void diff_emit_submodule_add(struct diff_options *o, const char *line);\n+void diff_emit_submodule_untracked(struct diff_options *o, const char *path);\n+void diff_emit_submodule_modified(struct diff_options *o, const char *path);\n+void diff_emit_submodule_header(struct diff_options *o, const char *header);\n+void diff_emit_submodule_error(struct diff_options *o, const char *err);\n+void diff_emit_submodule_pipethrough(struct diff_options *o,\n+\t\t\t\t const char *line, int len);\n+\n enum color_diff {\n \tDIFF_RESET = 0,\n \tDIFF_CONTEXT = 1,\ndiff --git a/submodule.c b/submodule.c\nindex da0b805493..07a3183150 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -479,9 +479,7 @@ static int prepare_submodule_summary(struct rev_info *rev, const char *path,\n \treturn prepare_revision_walk(rev);\n }\n \n-static void print_submodule_summary(struct rev_info *rev, FILE *f,\n-\t\tconst char *line_prefix,\n-\t\tconst char *del, const char *add, const char *reset)\n+static void print_submodule_summary(struct rev_info *rev, struct diff_options *o)\n {\n \tstatic const char format[] = \" %m %s\";\n \tstruct strbuf sb = STRBUF_INIT;\n@@ -492,18 +490,12 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \t\tctx.date_mode = rev->date_mode;\n \t\tctx.output_encoding = get_log_output_encoding();\n \t\tstrbuf_setlen(&sb, 0);\n-\t\tstrbuf_addstr(&sb, line_prefix);\n-\t\tif (commit->object.flags & SYMMETRIC_LEFT) {\n-\t\t\tif (del)\n-\t\t\t\tstrbuf_addstr(&sb, del);\n-\t\t}\n-\t\telse if (add)\n-\t\t\tstrbuf_addstr(&sb, add);\n \t\tformat_commit_message(commit, format, &sb, &ctx);\n-\t\tif (reset)\n-\t\t\tstrbuf_addstr(&sb, reset);\n \t\tstrbuf_addch(&sb, '\\n');\n-\t\tfprintf(f, \"%s\", sb.buf);\n+\t\tif (commit->object.flags & SYMMETRIC_LEFT)\n+\t\t\tdiff_emit_submodule_del(o, sb.buf);\n+\t\telse\n+\t\t\tdiff_emit_submodule_add(o, sb.buf);\n \t}\n \tstrbuf_release(&sb);\n }\n@@ -530,11 +522,9 @@ void prepare_submodule_repo_env(struct argv_array *out)\n * attempt to lookup both the left and right commits and put them into the\n * left and right pointers.\n */\n-static void show_submodule_header(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+static void show_submodule_header(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *reset,\n+\t\tunsigned dirty_submodule,\n \t\tstruct commit **left, struct commit **right,\n \t\tstruct commit_list **merge_bases)\n {\n@@ -543,11 +533,10 @@ static void show_submodule_header(FILE *f, const char *path,\n \tint fast_forward = 0, fast_backward = 0;\n \n \tif (dirty_submodule & DIRTY_SUBMODULE_UNTRACKED)\n-\t\tfprintf(f, \"%sSubmodule %s contains untracked content\\n\",\n-\t\t\tline_prefix, path);\n+\t\tdiff_emit_submodule_untracked(o, path);\n+\n \tif (dirty_submodule & DIRTY_SUBMODULE_MODIFIED)\n-\t\tfprintf(f, \"%sSubmodule %s contains modified content\\n\",\n-\t\t\tline_prefix, path);\n+\t\tdiff_emit_submodule_modified(o, path);\n \n \tif (is_null_oid(one))\n \t\tmessage = \"(new submodule)\";\n@@ -589,31 +578,29 @@ static void show_submodule_header(FILE *f, const char *path,\n \t}\n \n output_header:\n-\tstrbuf_addf(&sb, \"%s%sSubmodule %s \", line_prefix, meta, path);\n+\tstrbuf_addf(&sb, \"Submodule %s \", path);\n \tstrbuf_add_unique_abbrev(&sb, one->hash, DEFAULT_ABBREV);\n \tstrbuf_addstr(&sb, (fast_backward || fast_forward) ? \"..\" : \"...\");\n \tstrbuf_add_unique_abbrev(&sb, two->hash, DEFAULT_ABBREV);\n \tif (message)\n-\t\tstrbuf_addf(&sb, \" %s%s\\n\", message, reset);\n+\t\tstrbuf_addf(&sb, \" %s\\n\", message);\n \telse\n-\t\tstrbuf_addf(&sb, \"%s:%s\\n\", fast_backward ? \" (rewind)\" : \"\", reset);\n-\tfwrite(sb.buf, sb.len, 1, f);\n+\t\tstrbuf_addf(&sb, \"%s:\\n\", fast_backward ? \" (rewind)\" : \"\");\n+\tdiff_emit_submodule_header(o, sb.buf);\n \n \tstrbuf_release(&sb);\n }\n \n-void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset)\n+\t\tunsigned dirty_submodule)\n {\n \tstruct rev_info rev;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n-\t\t\t meta, reset, &left, &right, &merge_bases);\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n+\t\t\t &left, &right, &merge_bases);\n \n \t/*\n \t * If we don't have both a left and a right pointer, there is no\n@@ -625,11 +612,11 @@ void show_submodule_summary(FILE *f, const char *path,\n \n \t/* Treat revision walker failure the same as missing commits */\n \tif (prepare_submodule_summary(&rev, path, left, right, merge_bases)) {\n-\t\tfprintf(f, \"%s(revision walker failed)\\n\", line_prefix);\n+\t\tdiff_emit_submodule_error(o, \"(revision walker failed)\\n\");\n \t\tgoto out;\n \t}\n \n-\tprint_submodule_summary(&rev, f, line_prefix, del, add, reset);\n+\tprint_submodule_summary(&rev, o);\n \n out:\n \tif (merge_bases)\n@@ -638,21 +625,18 @@ void show_submodule_summary(FILE *f, const char *path,\n \tclear_commit_marks(right, ~0);\n }\n \n-void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *o)\n+\t\tunsigned dirty_submodule)\n {\n \tconst struct object_id *old = &empty_tree_oid, *new = &empty_tree_oid;\n \tstruct commit *left = NULL, *right = NULL;\n \tstruct commit_list *merge_bases = NULL;\n-\tstruct strbuf submodule_dir = STRBUF_INIT;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n \n-\tshow_submodule_header(f, path, line_prefix, one, two, dirty_submodule,\n-\t\t\t meta, reset, &left, &right, &merge_bases);\n+\tshow_submodule_header(o, path, one, two, dirty_submodule,\n+\t\t\t &left, &right, &merge_bases);\n \n \t/* We need a valid left and right commit to display a difference */\n \tif (!(left || is_null_oid(one)) ||\n@@ -664,16 +648,16 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \tif (right)\n \t\tnew = two;\n \n-\tfflush(f);\n \tcp.git_cmd = 1;\n \tcp.dir = path;\n-\tcp.out = dup(fileno(f));\n+\tcp.out = -1;\n \tcp.no_stdin = 1;\n \n \t/* TODO: other options may need to be passed here. */\n \targv_array_pushl(&cp.args, \"diff\", \"--submodule=diff\", NULL);\n+\targv_array_pushf(&cp.args, \"--color=%s\", want_color(o->use_color) ?\n+\t\t\t \"always\" : \"never\");\n \n-\targv_array_pushf(&cp.args, \"--line-prefix=%s\", line_prefix);\n \tif (DIFF_OPT_TST(o, REVERSE_DIFF)) {\n \t\targv_array_pushf(&cp.args, \"--src-prefix=%s%s/\",\n \t\t\t\t o->b_prefix, path);\n@@ -696,11 +680,17 @@ void show_submodule_inline_diff(FILE *f, const char *path,\n \t\targv_array_push(&cp.args, oid_to_hex(new));\n \n \tprepare_submodule_repo_env(&cp.env_array);\n-\tif (run_command(&cp))\n-\t\tfprintf(f, \"(diff failed)\\n\");\n+\tif (start_command(&cp))\n+\t\tdiff_emit_submodule_error(o, \"(diff failed)\\n\");\n+\n+\twhile (strbuf_getwholeline_fd(&sb, cp.out, '\\n') != EOF)\n+\t\tdiff_emit_submodule_pipethrough(o, sb.buf, sb.len);\n+\n+\tif (finish_command(&cp))\n+\t\tdiff_emit_submodule_error(o, \"(diff failed)\\n\");\n \n done:\n-\tstrbuf_release(&submodule_dir);\n+\tstrbuf_release(&sb);\n \tif (merge_bases)\n \t\tfree_commit_list(merge_bases);\n \tif (left)\ndiff --git a/submodule.h b/submodule.h\nindex cbe5c1726f..829d975585 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -63,17 +63,12 @@ extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n extern void handle_ignore_submodules_arg(struct diff_options *, const char *);\n-extern void show_submodule_summary(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+extern void show_submodule_summary(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset);\n-extern void show_submodule_inline_diff(FILE *f, const char *path,\n-\t\tconst char *line_prefix,\n+\t\tunsigned dirty_submodule);\n+extern void show_submodule_inline_diff(struct diff_options *o, const char *path,\n \t\tstruct object_id *one, struct object_id *two,\n-\t\tunsigned dirty_submodule, const char *meta,\n-\t\tconst char *del, const char *add, const char *reset,\n-\t\tconst struct diff_options *opt);\n+\t\tunsigned dirty_submodule);\n extern void set_config_fetch_recurse_submodules(int value);\n /* Check if we want to update any submodule.*/\n extern int should_update_submodules(void);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323631","messageId":"20170630205310.7380-22-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 21/25] diff.c: buffer all output if asked to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:06Z","receivedAt":"2017-06-30T20:53:56Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Introduce a new option 'emitted_symbols' in the struct diff_options which\ncontrols whether all output is buffered up until all output is available.\nIt is set internally in diff.c when necessary.\n\nWe'll have a new struct 'emitted_string' in diff.c which will be used to\nbuffer each line. The emitted_string will duplicate the memory of the\nline to buffer as that is easiest to reason about for now. In a future\npatch we may want to decrease the memory usage by not duplicating all\noutput for buffering but rather we may want to store offsets into the\nfile or in case of hunk descriptions such as the similarity score, we\ncould just store the relevant number and reproduce the text later on.\n\nThis approach was chosen as a first step because it is quite simple\ncompared to the alternative with less memory footprint.\n\nemit_diff_symbol factors out the emission part and depending on the\ndiff_options->emitted_symbols the emission will be performed directly\nwhen calling emit_diff_symbol or after the whole process is done, i.e.\nby buffering we have add the possibility for a second pass over the\nwhole output before doing the actual output.\n\nIn 6440d34 (2012-03-14, diff: tweak a _copy_ of diff_options with\nword-diff) we introduced a duplicate diff options struct for word\nemissions as we may have different regex settings in there.\nWhen buffering the output, we need to operate on just one buffer,\nso we have to copy back the emissions of the word buffer into the\nmain buffer.\n\nUnconditionally enable output via buffer in this patch as it yields\na great opportunity for testing, i.e. all the diff tests from the\ntest suite pass without having reordering issues (i.e. only parts\nof the output got buffered, and we forgot to buffer other parts).\nThe test suite passes, which gives confidence that we converted all\nfunctions to use emit_string for output.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--\n diff.h | 2 ++\n 2 files changed, 109 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 2db0d7c0f5..31720abf8f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -605,6 +605,47 @@ enum diff_symbol {\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n+/*\n+ * This struct is used when we need to buffer the output of the diff output.\n+ *\n+ * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer\n+ * into the pre/post image file. This pointer could be a union with the\n+ * line pointer. By storing an offset into the file instead of the literal line,\n+ * we can decrease the memory footprint for the buffered output. At first we\n+ * may want to only have indirection for the content lines, but we could also\n+ * enhance the state for emitting prefabricated lines, e.g. the similarity\n+ * score line or hunk/file headers would only need to store a number or path\n+ * and then the output can be constructed later on depending on state.\n+ */\n+struct emitted_diff_symbol {\n+\tconst char *line;\n+\tint len;\n+\tint flags;\n+\tenum diff_symbol s;\n+};\n+#define EMITTED_DIFF_SYMBOL_INIT {NULL}\n+\n+struct emitted_diff_symbols {\n+\tstruct emitted_diff_symbol *buf;\n+\tint nr, alloc;\n+};\n+#define EMITTED_DIFF_SYMBOLS_INIT {NULL, 0, 0}\n+\n+static void append_emitted_diff_symbol(struct diff_options *o,\n+\t\t\t\t struct emitted_diff_symbol *e)\n+{\n+\tstruct emitted_diff_symbol *f;\n+\n+\tALLOC_GROW(o->emitted_symbols->buf,\n+\t\t o->emitted_symbols->nr + 1,\n+\t\t o->emitted_symbols->alloc);\n+\tf = &o->emitted_symbols->buf[o->emitted_symbols->nr++];\n+\n+\tmemcpy(f, e, sizeof(struct emitted_diff_symbol));\n+\tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n+}\n+\n+\n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n \t\t\t\tconst char *line, int len, char sign,\n@@ -631,12 +672,18 @@ static void emit_line_ws_markup(struct diff_options *o,\n \t}\n }\n \n-static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n-\t\t\t const char *line, int len, unsigned flags)\n+static void emit_diff_symbol_from_struct(struct diff_options *o,\n+\t\t\t\t\t struct emitted_diff_symbol *eds)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n \tconst char *context, *reset, *set, *meta, *fraginfo;\n \tstruct strbuf sb = STRBUF_INIT;\n+\n+\tenum diff_symbol s = eds->s;\n+\tconst char *line = eds->line;\n+\tint len = eds->len;\n+\tunsigned flags = eds->flags;\n+\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -778,6 +825,17 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tstrbuf_release(&sb);\n }\n \n+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n+\t\t\t const char *line, int len, unsigned flags)\n+{\n+\tstruct emitted_diff_symbol e = {line, len, flags, s};\n+\n+\tif (o->emitted_symbols)\n+\t\tappend_emitted_diff_symbol(o, &e);\n+\telse\n+\t\temit_diff_symbol_from_struct(o, &e);\n+}\n+\n void diff_emit_submodule_del(struct diff_options *o, const char *line)\n {\n \temit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);\n@@ -1374,9 +1432,29 @@ static void diff_words_show(struct diff_words_data *diff_words)\n /* In \"color-words\" mode, show word-diff of words accumulated in the buffer */\n static void diff_words_flush(struct emit_callback *ecbdata)\n {\n+\tstruct diff_options *wo = ecbdata->diff_words->opt;\n+\n \tif (ecbdata->diff_words->minus.text.size ||\n \t ecbdata->diff_words->plus.text.size)\n \t\tdiff_words_show(ecbdata->diff_words);\n+\n+\tif (wo->emitted_symbols) {\n+\t\tstruct diff_options *o = ecbdata->opt;\n+\t\tstruct emitted_diff_symbols *wol = wo->emitted_symbols;\n+\t\tint i;\n+\n+\t\t/*\n+\t\t * NEEDSWORK:\n+\t\t * Instead of appending each, concat all words to a line?\n+\t\t */\n+\t\tfor (i = 0; i < wol->nr; i++)\n+\t\t\tappend_emitted_diff_symbol(o, &wol->buf[i]);\n+\n+\t\tfor (i = 0; i < wol->nr; i++)\n+\t\t\tfree((void *)wol->buf[i].line);\n+\n+\t\twol->nr = 0;\n+\t}\n }\n \n static void diff_filespec_load_driver(struct diff_filespec *one)\n@@ -1412,6 +1490,11 @@ static void init_diff_words_data(struct emit_callback *ecbdata,\n \t\txcalloc(1, sizeof(struct diff_words_data));\n \tecbdata->diff_words->type = o->word_diff;\n \tecbdata->diff_words->opt = o;\n+\n+\tif (orig_opts->emitted_symbols)\n+\t\to->emitted_symbols =\n+\t\t\txcalloc(1, sizeof(struct emitted_diff_symbols));\n+\n \tif (!o->word_regex)\n \t\to->word_regex = userdiff_word_regex(one);\n \tif (!o->word_regex)\n@@ -1446,6 +1529,7 @@ static void free_diff_words_data(struct emit_callback *ecbdata)\n {\n \tif (ecbdata->diff_words) {\n \t\tdiff_words_flush(ecbdata);\n+\t\tfree (ecbdata->diff_words->opt->emitted_symbols);\n \t\tfree (ecbdata->diff_words->opt);\n \t\tfree (ecbdata->diff_words->minus.text.ptr);\n \t\tfree (ecbdata->diff_words->minus.orig);\n@@ -4996,16 +5080,37 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n+\tstatic struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n \n \tif (WSEH_NEW & WS_RULE_MASK)\n \t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n \n+\t/*\n+\t * For testing purposes we want to make sure the diff machinery\n+\t * works completely with the buffer. If there is anything emitted\n+\t * outside the emit_string, then the order is screwed\n+\t * up and the tests will fail.\n+\t *\n+\t * TODO (later in this series):\n+\t * We'll unset this pointer in a later patch.\n+\t */\n+\to->emitted_symbols = &esm;\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tif (check_pair_status(p))\n \t\t\tdiff_flush_patch(p, o);\n \t}\n+\n+\tif (o->emitted_symbols) {\n+\t\tfor (i = 0; i < esm.nr; i++)\n+\t\t\temit_diff_symbol_from_struct(o, &esm.buf[i]);\n+\n+\t\tfor (i = 0; i < esm.nr; i++)\n+\t\t\tfree((void *)esm.buf[i].line);\n+\t}\n+\tesm.nr = 0;\n }\n \n void diff_flush(struct diff_options *options)\ndiff --git a/diff.h b/diff.h\nindex d6094a1eed..4a3b9bde40 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -186,6 +186,8 @@ struct diff_options {\n \tvoid *output_prefix_data;\n \n \tint diff_path_counter;\n+\n+\tstruct emitted_diff_symbols *emitted_symbols;\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323632","messageId":"20170630205310.7380-26-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 25/25] diff: document the new --color-moved setting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:10Z","receivedAt":"2017-06-30T20:53:59Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n Documentation/config.txt | 15 +++++++++++++--\n Documentation/diff-options.txt | 36 ++++++++++++++++++++++++++++++++++++\n 2 files changed, 49 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 06898a7498..74382e5ff5 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1077,14 +1077,25 @@ This does not affect linkgit:git-format-patch[1] or the\n 'git-diff-{asterisk}' plumbing commands. Can be overridden on the\n command line with the `--color[=]` option.\n \n+diff.colorMoved::\n+\tIf set to either a valid `` or a true value, moved lines\n+\tin a diff are colored differently, for details of valid modes\n+\tsee '--color-moved' in linkgit:git-diff[1]. If simply set to\n+\ttrue the default color mode will be used. When set to false,\n+\tmoved lines are not colored.\n+\n color.diff.::\n \tUse customized color for diff colorization. `` specifies\n \twhich part of the patch to use the specified color, and is one\n \tof `context` (context text - `plain` is a historical synonym),\n \t`meta` (metainformation), `frag`\n \t(hunk header), 'func' (function in hunk header), `old` (removed lines),\n-\t`new` (added lines), `commit` (commit headers), or `whitespace`\n-\t(highlighting whitespace errors).\n+\t`new` (added lines), `commit` (commit headers), `whitespace`\n+\t(highlighting whitespace errors), `oldMoved` (deleted lines),\n+\t`newMoved` (added lines), `oldMovedDimmed`, `oldMovedAlternative`,\n+\t`oldMovedAlternativeDimmed`, `newMovedDimmed`, `newMovedAlternative`\n+\tand `newMovedAlternativeDimmed` (See the ''\n+\tsetting of '--color-moved' in linkgit:git-diff[1] for details).\n \n color.decorate.::\n \tUse customized color for 'git log --decorate' output. `` is one\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex 89cc0f48de..bc52bd0b99 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -231,6 +231,42 @@ ifdef::git-diff[]\n endif::git-diff[]\n \tIt is the same as `--color=never`.\n \n+--color-moved[=]::\n+\tMoved lines of code are colored differently.\n+ifdef::git-diff[]\n+\tIt can be changed by the `diff.colorMoved` configuration setting.\n+endif::git-diff[]\n+\tThe defaults to 'no' if the option is not given\n+\tand to 'zebra' if the option with no mode is given.\n+\tThe mode must be one of:\n++\n+--\n+no::\n+\tMoved lines are not highlighted.\n+default::\n+\tIs a synonym for `zebra`. This may change to a more sensible mode\n+\tin the future.\n+plain::\n+\tAny line that is added in one location and was removed\n+\tin another location will be colored with 'color.diff.newMoved'.\n+\tSimilarly 'color.diff.oldMoved' will be used for removed lines\n+\tthat are added somewhere else in the diff. This mode picks up any\n+\tmoved line, but it is not very useful in a review to determine\n+\tif a block of code was moved without permutation.\n+zebra::\n+\tBlocks of moved code are detected greedily. The detected blocks are\n+\tpainted using either the 'color.diff.{old,new}Moved' color or\n+\t'color.diff.{old,new}MovedAlternative'. The change between\n+\tthe two colors indicates that a new block was detected. If there\n+\tare fewer than 3 adjacent moved lines, they are not marked up\n+\tas moved, but the regular colors 'color.diff.{old,new}' will be\n+\tused.\n+dimmed_zebra::\n+\tSimilar to 'zebra', but additional dimming of uninteresting parts\n+\tof moved code is performed. The bordering lines of two adjacent\n+\tblocks are considered interesting, the rest is uninteresting.\n+--\n+\n --word-diff[=]::\n \tShow a word diff, using the to delimit changed words.\n \tBy default, words are delimited by whitespace; see\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323633","messageId":"20170630205310.7380-24-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 23/25] diff.c: color moved lines differently, plain mode","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:08Z","receivedAt":"2017-06-30T20:54:02Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add the 'plain' mode for move detection of code. This omits the checking\nfor adjacent blocks, so it is not as useful. If you have a lot of the\nsame blocks moved in the same patch, the 'Zebra' would end up slow as it\nis O(n^2) (n is number of same blocks). So this may be useful there and\nis generally easy to add. Instead be very literal at the move detection,\ndo not skip over short blocks here.\n\nSigned-off-by: Stefan Beller \n---\n diff.c | 10 +++++++--\n diff.h | 1 +\n t/t4015-diff-whitespace.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++-\n 3 files changed, 59 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 084ecc5678..79ea592388 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -256,12 +256,14 @@ static int parse_color_moved(const char *arg)\n \n \tif (!strcmp(arg, \"no\"))\n \t\treturn COLOR_MOVED_NO;\n+\telse if (!strcmp(arg, \"plain\"))\n+\t\treturn COLOR_MOVED_PLAIN;\n \telse if (!strcmp(arg, \"zebra\"))\n \t\treturn COLOR_MOVED_ZEBRA;\n \telse if (!strcmp(arg, \"default\"))\n \t\treturn COLOR_MOVED_DEFAULT;\n \telse\n-\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra'\"));\n+\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra', 'plain'\"));\n }\n \n int git_diff_ui_config(const char *var, const char *value, void *cb)\n@@ -879,7 +881,8 @@ static void mark_color_as_moved(struct diff_options *o,\n \t\t}\n \n \t\tif (!match) {\n-\t\t\tif (block_length < COLOR_MOVED_MIN_BLOCK_LENGTH) {\n+\t\t\tif (block_length < COLOR_MOVED_MIN_BLOCK_LENGTH &&\n+\t\t\t o->color_moved != COLOR_MOVED_PLAIN) {\n \t\t\t\tfor (i = 0; i < block_length + 1; i++) {\n \t\t\t\t\tl = &o->emitted_symbols->buf[n - i];\n \t\t\t\t\tl->flags &= ~DIFF_SYMBOL_MOVED_LINE;\n@@ -893,6 +896,9 @@ static void mark_color_as_moved(struct diff_options *o,\n \t\tl->flags |= DIFF_SYMBOL_MOVED_LINE;\n \t\tblock_length++;\n \n+\t\tif (o->color_moved == COLOR_MOVED_PLAIN)\n+\t\t\tcontinue;\n+\n \t\t/* Check any potential block runs, advance each or nullify */\n \t\tfor (i = 0; i < pmb_nr; i++) {\n \t\t\tstruct moved_entry *p = pmb[i];\ndiff --git a/diff.h b/diff.h\nindex 3196802673..4cfd609c54 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -190,6 +190,7 @@ struct diff_options {\n \tstruct emitted_diff_symbols *emitted_symbols;\n \tenum {\n \t\tCOLOR_MOVED_NO = 0,\n+\t\tCOLOR_MOVED_PLAIN = 1,\n \t\tCOLOR_MOVED_ZEBRA = 2,\n \t} color_moved;\n \t#define COLOR_MOVED_DEFAULT COLOR_MOVED_ZEBRA\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 29704ae14e..d1d7b99990 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -986,7 +986,7 @@ test_expect_success 'detect moved code, complete file' '\n \tgit mv test.c main.c &&\n \ttest_config color.diff.oldMoved \"normal red\" &&\n \ttest_config color.diff.newMoved \"normal green\" &&\n-\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tgit diff HEAD --color-moved=zebra --no-renames | test_decode_color >actual &&\n \tcat >expected <<-\\EOF &&\n \tdiff --git a/main.c b/main.c\n \tnew file mode 100644\n@@ -1130,6 +1130,55 @@ test_expect_success 'detect malicious moved code, inside file' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'plain moved code, inside file' '\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\t# needs previous test as setup\n+\tgit diff HEAD --no-renames --color-moved=plain| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'no effect from --color-moved with --word-diff' '\n \tcat <<-\\EOF >text.txt &&\n \tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323634","messageId":"20170630205310.7380-23-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 22/25] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:07Z","receivedAt":"2017-06-30T20:54:04Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a patch consists mostly of moving blocks of code around, it can\nbe quite tedious to ensure that the blocks are moved verbatim, and not\nundesirably modified in the move. To that end, color blocks that are\nmoved within the same patch differently. For example (OM, del, add,\nand NM are different colors):\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OM] - if (!is_authorized_user())\n [OM] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OM] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NM] + sensitive_stuff(spanning,\n [NM] + multiple,\n [NM] + lines);\n [NM] +}\n\nHowever adjacent blocks may be problematic. For example, in this\npotentially malicious patch, the swapping of blocks can be spotted:\n\n [OM] -void sensitive_stuff(void)\n [OM] -{\n [OMA] - if (!is_authorized_user())\n [OMA] - die(\"unauthorized\");\n [OM] - sensitive_stuff(spanning,\n [OM] - multiple,\n [OM] - lines);\n [OMA] -}\n\n void another_function()\n {\n [del] - printf(\"foo\");\n [add] + printf(\"bar\");\n }\n\n [NM] +void sensitive_stuff(void)\n [NM] +{\n [NMA] + sensitive_stuff(spanning,\n [NMA] + multiple,\n [NMA] + lines);\n [NM] + if (!is_authorized_user())\n [NM] + die(\"unauthorized\");\n [NMA] +}\n\nIf the moved code is larger, it is easier to hide some permutation in the\ncode, which is why some alternative coloring is needed.\n\nThis patch implements the first mode:\n* basic alternating 'Zebra' mode\n This conveys all information needed to the user. Defer customization to\n later patches.\n\nFirst I implemented an alternative design, which would try to fingerprint\na line by its neighbors to detect if we are in a block or at the boundary.\nThis idea iss error prone as it inspected each line and its neighboring\nlines to determine if the line was (a) moved and (b) if was deep inside\na hunk by having matching neighboring lines. This is unreliable as the\nwe can construct hunks which have equal neighbors that just exceed the\nnumber of lines inspected. (Think of 'AXYZBXYZCXYZD..' with each letter\nas a line, that is permutated to AXYZCXYZBXYZD..').\n\nInstead this provides a dynamic programming greedy algorithm that finds\nthe largest moved hunk and then has several modes on highlighting bounds.\n\nA note on the options '--submodule=diff' and '--color-words/--word-diff':\nIn the conversion to use emit_line in the prior patches both submodules\nas well as word diff output carefully chose to call emit_line with sign=0.\nAll output with sign=0 is ignored for move detection purposes in this\npatch, such that no weird looking output will be generated for these\ncases. This leads to another thought: We could pass on '--color-moved' to\nsubmodules such that they color up moved lines for themselves. If we'd do\nso only line moves within a repository boundary are marked up.\n\nIt is useful to have moved lines colored, but there are annoying corner\ncases, such as a single line moved, that is very common. For example\nin a typical patch of C code, we have closing braces that end statement\nblocks or functions.\n\nWhile it is technically true that these lines are moved as they show up\nelsewhere, it is harmful for the review as the reviewers attention is\ndrawn to such a minor side annoyance.\n\nFor now let's have a simple solution of hardcoding the number of\nmoved lines to be at least 3 before coloring them. Note, that the\nlength is applied across all blocks to find the 'lonely' blocks\nthat pollute new code, but do not interfere with a permutated\nblock where each permutation has less lines than 3.\n\nHelped-by: Jonathan Tan \nSigned-off-by: Stefan Beller \n---\n diff.c | 344 +++++++++++++++++++++++++++++++++++++++++++--\n diff.h | 12 +-\n t/t4015-diff-whitespace.sh | 261 ++++++++++++++++++++++++++++++++++\n 3 files changed, 602 insertions(+), 15 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 31720abf8f..084ecc5678 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -16,6 +16,7 @@\n #include \"userdiff.h\"\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n+#include \"hashmap.h\"\n #include \"ll-merge.h\"\n #include \"string-list.h\"\n #include \"argv-array.h\"\n@@ -32,6 +33,7 @@ static int diff_indent_heuristic = 1;\n static int diff_rename_limit_default = 400;\n static int diff_suppress_blank_empty;\n static int diff_use_color_default = -1;\n+static int diff_color_moved_default;\n static int diff_context_default = 3;\n static int diff_interhunk_context_default;\n static const char *diff_word_regex_cfg;\n@@ -56,6 +58,10 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n+\tGIT_COLOR_MAGENTA,\t/* OLD_MOVED */\n+\tGIT_COLOR_BLUE,\t\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_CYAN,\t\t/* NEW_MOVED */\n+\tGIT_COLOR_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -81,6 +87,14 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_WHITESPACE;\n \tif (!strcasecmp(var, \"func\"))\n \t\treturn DIFF_FUNCINFO;\n+\tif (!strcasecmp(var, \"oldmoved\"))\n+\t\treturn DIFF_FILE_OLD_MOVED;\n+\tif (!strcasecmp(var, \"oldmovedalternative\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoved\"))\n+\t\treturn DIFF_FILE_NEW_MOVED;\n+\tif (!strcasecmp(var, \"newmovedalternative\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT;\n \treturn -1;\n }\n \n@@ -229,12 +243,40 @@ int git_diff_heuristic_config(const char *var, const char *value, void *cb)\n \treturn 0;\n }\n \n+static int parse_color_moved(const char *arg)\n+{\n+\tswitch (git_parse_maybe_bool(arg)) {\n+\tcase 0:\n+\t\treturn COLOR_MOVED_NO;\n+\tcase 1:\n+\t\treturn COLOR_MOVED_DEFAULT;\n+\tdefault:\n+\t\tbreak;\n+\t}\n+\n+\tif (!strcmp(arg, \"no\"))\n+\t\treturn COLOR_MOVED_NO;\n+\telse if (!strcmp(arg, \"zebra\"))\n+\t\treturn COLOR_MOVED_ZEBRA;\n+\telse if (!strcmp(arg, \"default\"))\n+\t\treturn COLOR_MOVED_DEFAULT;\n+\telse\n+\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra'\"));\n+}\n+\n int git_diff_ui_config(const char *var, const char *value, void *cb)\n {\n \tif (!strcmp(var, \"diff.color\") || !strcmp(var, \"color.diff\")) {\n \t\tdiff_use_color_default = git_config_colorbool(var, value);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"diff.colormoved\")) {\n+\t\tint cm = parse_color_moved(value);\n+\t\tif (cm < 0)\n+\t\t\treturn -1;\n+\t\tdiff_color_moved_default = cm;\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"diff.context\")) {\n \t\tdiff_context_default = git_config_int(var, value);\n \t\tif (diff_context_default < 0)\n@@ -602,7 +644,9 @@ enum diff_symbol {\n * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT\n * 16 is marking if the line is blank at EOF\n */\n-#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n+#define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n+#define DIFF_SYMBOL_MOVED_LINE_ALT\t\t(1<<18)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n /*\n@@ -645,6 +689,243 @@ static void append_emitted_diff_symbol(struct diff_options *o,\n \tf->line = e->line ? xmemdupz(e->line, e->len) : NULL;\n }\n \n+struct moved_entry {\n+\tstruct hashmap_entry ent;\n+\tconst struct emitted_diff_symbol *es;\n+\tstruct moved_entry *next_line;\n+};\n+\n+static int next_byte(const char **cp, const char **endp,\n+\t\t const struct diff_options *diffopt)\n+{\n+\tint retval;\n+\n+\tif (*cp > *endp)\n+\t\treturn -1;\n+\n+\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_CHANGE)) {\n+\t\twhile (*cp < *endp && isspace(**cp))\n+\t\t\t(*cp)++;\n+\t\t/*\n+\t\t * After skipping a couple of whitespaces, we still have to\n+\t\t * account for one space.\n+\t\t */\n+\t\treturn (int)' ';\n+\t}\n+\n+\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE)) {\n+\t\twhile (*cp < *endp && isspace(**cp))\n+\t\t\t(*cp)++;\n+\t\t/* return the first non-ws character via the usual below */\n+\t}\n+\n+\tretval = (unsigned char)(**cp);\n+\t(*cp)++;\n+\treturn retval;\n+}\n+\n+static int moved_entry_cmp(const struct diff_options *diffopt,\n+\t\t\t const struct moved_entry *a,\n+\t\t\t const struct moved_entry *b,\n+\t\t\t const void *keydata)\n+{\n+\tconst char *ap = a->es->line, *ae = a->es->line + a->es->len;\n+\tconst char *bp = b->es->line, *be = b->es->line + b->es->len;\n+\n+\tif (!(diffopt->xdl_opts & XDF_WHITESPACE_FLAGS))\n+\t\treturn a->es->len != b->es->len || memcmp(ap, bp, a->es->len);\n+\n+\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_AT_EOL)) {\n+\t\twhile (ae > ap && isspace(*ae))\n+\t\t\tae--;\n+\t\twhile (be > bp && isspace(*be))\n+\t\t\tbe--;\n+\t}\n+\n+\twhile (1) {\n+\t\tint ca, cb;\n+\t\tca = next_byte(&ap, &ae, diffopt);\n+\t\tcb = next_byte(&bp, &be, diffopt);\n+\t\tif (ca != cb)\n+\t\t\treturn 1;\n+\t\tif (ca < 0)\n+\t\t\treturn 0;\n+\t}\n+}\n+\n+static unsigned get_string_hash(struct emitted_diff_symbol *es, struct diff_options *o)\n+{\n+\tif (o->xdl_opts & XDF_WHITESPACE_FLAGS) {\n+\t\tstatic struct strbuf sb = STRBUF_INIT;\n+\t\tconst char *ap = es->line, *ae = es->line + es->len;\n+\t\tint c;\n+\n+\t\tstrbuf_reset(&sb);\n+\t\twhile (ae > ap && isspace(*ae))\n+\t\t\tae--;\n+\t\twhile ((c = next_byte(&ap, &ae, o)) > 0)\n+\t\t\tstrbuf_addch(&sb, c);\n+\n+\t\treturn memhash(sb.buf, sb.len);\n+\t} else {\n+\t\treturn memhash(es->line, es->len);\n+\t}\n+}\n+\n+static struct moved_entry *prepare_entry(struct diff_options *o,\n+\t\t\t\t\t int line_no)\n+{\n+\tstruct moved_entry *ret = xmalloc(sizeof(*ret));\n+\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];\n+\n+\tret->ent.hash = get_string_hash(l, o);\n+\tret->es = l;\n+\tret->next_line = NULL;\n+\n+\treturn ret;\n+}\n+\n+static void add_lines_to_move_detection(struct diff_options *o,\n+\t\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry *prev_line = NULL;\n+\n+\tint n;\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct hashmap *hm;\n+\t\tstruct moved_entry *key;\n+\n+\t\tswitch (o->emitted_symbols->buf[n].s) {\n+\t\tcase DIFF_SYMBOL_PLUS:\n+\t\t\thm = add_lines;\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MINUS:\n+\t\t\thm = del_lines;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tprev_line = NULL;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tkey = prepare_entry(o, n);\n+\t\tif (prev_line && prev_line->es->s == o->emitted_symbols->buf[n].s)\n+\t\t\tprev_line->next_line = key;\n+\n+\t\thashmap_add(hm, key);\n+\t\tprev_line = key;\n+\t}\n+}\n+\n+static int shrink_potential_moved_blocks(struct moved_entry **pmb,\n+\t\t\t\t\t int pmb_nr)\n+{\n+\tint lp, rp;\n+\n+\t/* Shrink the set of potential block to the remaining running */\n+\tfor (lp = 0, rp = pmb_nr - 1; lp <= rp;) {\n+\t\twhile (lp < pmb_nr && pmb[lp])\n+\t\t\tlp++;\n+\t\t/* lp points at the first NULL now */\n+\n+\t\twhile (rp > -1 && !pmb[rp])\n+\t\t\trp--;\n+\t\t/* rp points at the last non-NULL */\n+\n+\t\tif (lp < pmb_nr && rp > -1 && lp < rp) {\n+\t\t\tpmb[lp] = pmb[rp];\n+\t\t\tpmb[rp] = NULL;\n+\t\t\trp--;\n+\t\t\tlp++;\n+\t\t}\n+\t}\n+\n+\t/* Remember the number of running sets */\n+\treturn rp + 1;\n+}\n+\n+/* Find blocks of moved code, delegate actual coloring decision to helper */\n+static void mark_color_as_moved(struct diff_options *o,\n+\t\t\t\tstruct hashmap *add_lines,\n+\t\t\t\tstruct hashmap *del_lines)\n+{\n+\tstruct moved_entry **pmb = NULL; /* potentially moved blocks */\n+\tint pmb_nr = 0, pmb_alloc = 0;\n+\tint n, flipped_block = 1, block_length = 0;\n+\n+\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct hashmap *hm = NULL;\n+\t\tstruct moved_entry *key;\n+\t\tstruct moved_entry *match = NULL;\n+\t\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];\n+\t\tint i;\n+\n+\t\tswitch (l->s) {\n+\t\tcase DIFF_SYMBOL_PLUS:\n+\t\t\thm = del_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MINUS:\n+\t\t\thm = add_lines;\n+\t\t\tkey = prepare_entry(o, n);\n+\t\t\tmatch = hashmap_get(hm, key, o);\n+\t\t\tfree(key);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tflipped_block = 1;\n+\t\t}\n+\n+\t\tif (!match) {\n+\t\t\tif (block_length < COLOR_MOVED_MIN_BLOCK_LENGTH) {\n+\t\t\t\tfor (i = 0; i < block_length + 1; i++) {\n+\t\t\t\t\tl = &o->emitted_symbols->buf[n - i];\n+\t\t\t\t\tl->flags &= ~DIFF_SYMBOL_MOVED_LINE;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tpmb_nr = 0;\n+\t\t\tblock_length = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE;\n+\t\tblock_length++;\n+\n+\t\t/* Check any potential block runs, advance each or nullify */\n+\t\tfor (i = 0; i < pmb_nr; i++) {\n+\t\t\tstruct moved_entry *p = pmb[i];\n+\t\t\tstruct moved_entry *pnext = (p && p->next_line) ?\n+\t\t\t\t\tp->next_line : NULL;\n+\t\t\tif (pnext && !hm->cmpfn(o, pnext, match, NULL)) {\n+\t\t\t\tpmb[i] = p->next_line;\n+\t\t\t} else {\n+\t\t\t\tpmb[i] = NULL;\n+\t\t\t}\n+\t\t}\n+\n+\t\tpmb_nr = shrink_potential_moved_blocks(pmb, pmb_nr);\n+\n+\t\tif (pmb_nr == 0) {\n+\t\t\t/*\n+\t\t\t * The current line is the start of a new block.\n+\t\t\t * Setup the set of potential blocks.\n+\t\t\t */\n+\t\t\tfor (; match; match = hashmap_get_next(hm, match)) {\n+\t\t\t\tALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);\n+\t\t\t\tpmb[pmb_nr++] = match;\n+\t\t\t}\n+\n+\t\t\tflipped_block = (flipped_block + 1) % 2;\n+\t\t}\n+\n+\t\tif (flipped_block)\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;\n+\t}\n+\n+\tfree(pmb);\n+}\n \n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n@@ -720,14 +1001,24 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n \t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n-\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n+\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);\n+\t\telse\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '+',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n-\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n+\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);\n+\t\telse\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n@@ -3740,6 +4031,8 @@ void diff_setup(struct diff_options *options)\n \t\toptions->a_prefix = \"a/\";\n \t\toptions->b_prefix = \"b/\";\n \t}\n+\n+\toptions->color_moved = diff_color_moved_default;\n }\n \n void diff_setup_done(struct diff_options *options)\n@@ -3849,6 +4142,9 @@ void diff_setup_done(struct diff_options *options)\n \n \tif (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)\n \t\tdie(_(\"--follow requires exactly one pathspec\"));\n+\n+\tif (!options->use_color || external_diff())\n+\t\toptions->color_moved = 0;\n }\n \n static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)\n@@ -4273,7 +4569,19 @@ int diff_opt_parse(struct diff_options *options,\n \t}\n \telse if (!strcmp(arg, \"--no-color\"))\n \t\toptions->use_color = 0;\n-\telse if (!strcmp(arg, \"--color-words\")) {\n+\telse if (!strcmp(arg, \"--color-moved\")) {\n+\t\tif (diff_color_moved_default)\n+\t\t\toptions->color_moved = diff_color_moved_default;\n+\t\tif (options->color_moved == COLOR_MOVED_NO)\n+\t\t\toptions->color_moved = COLOR_MOVED_DEFAULT;\n+\t} else if (!strcmp(arg, \"--no-color-moved\"))\n+\t\toptions->color_moved = COLOR_MOVED_NO;\n+\telse if (skip_prefix(arg, \"--color-moved=\", &arg)) {\n+\t\tint cm = parse_color_moved(arg);\n+\t\tif (cm < 0)\n+\t\t\tdie(\"bad --color-moved argument: %s\", arg);\n+\t\toptions->color_moved = cm;\n+\t} else if (!strcmp(arg, \"--color-words\")) {\n \t\toptions->use_color = 1;\n \t\toptions->word_diff = DIFF_WORDS_COLOR;\n \t}\n@@ -5086,16 +5394,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \tif (WSEH_NEW & WS_RULE_MASK)\n \t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n \n-\t/*\n-\t * For testing purposes we want to make sure the diff machinery\n-\t * works completely with the buffer. If there is anything emitted\n-\t * outside the emit_string, then the order is screwed\n-\t * up and the tests will fail.\n-\t *\n-\t * TODO (later in this series):\n-\t * We'll unset this pointer in a later patch.\n-\t */\n-\to->emitted_symbols = &esm;\n+\tif (o->color_moved)\n+\t\to->emitted_symbols = &esm;\n \n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n@@ -5104,6 +5404,21 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \t}\n \n \tif (o->emitted_symbols) {\n+\t\tif (o->color_moved) {\n+\t\t\tstruct hashmap add_lines, del_lines;\n+\n+\t\t\thashmap_init(&del_lines,\n+\t\t\t\t (hashmap_cmp_fn)moved_entry_cmp, o, 0);\n+\t\t\thashmap_init(&add_lines,\n+\t\t\t\t (hashmap_cmp_fn)moved_entry_cmp, o, 0);\n+\n+\t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n+\t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\n+\t\t\thashmap_free(&add_lines, 0);\n+\t\t\thashmap_free(&del_lines, 0);\n+\t\t}\n+\n \t\tfor (i = 0; i < esm.nr; i++)\n \t\t\temit_diff_symbol_from_struct(o, &esm.buf[i]);\n \n@@ -5185,6 +5500,7 @@ void diff_flush(struct diff_options *options)\n \t\t\tfclose(options->file);\n \t\toptions->file = xfopen(\"/dev/null\", \"w\");\n \t\toptions->close_file = 1;\n+\t\toptions->color_moved = 0;\n \t\tfor (i = 0; i < q->nr; i++) {\n \t\t\tstruct diff_filepair *p = q->queue[i];\n \t\t\tif (check_pair_status(p))\ndiff --git a/diff.h b/diff.h\nindex 4a3b9bde40..3196802673 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -188,6 +188,12 @@ struct diff_options {\n \tint diff_path_counter;\n \n \tstruct emitted_diff_symbols *emitted_symbols;\n+\tenum {\n+\t\tCOLOR_MOVED_NO = 0,\n+\t\tCOLOR_MOVED_ZEBRA = 2,\n+\t} color_moved;\n+\t#define COLOR_MOVED_DEFAULT COLOR_MOVED_ZEBRA\n+\t#define COLOR_MOVED_MIN_BLOCK_LENGTH 3\n };\n \n void diff_emit_submodule_del(struct diff_options *o, const char *line);\n@@ -208,7 +214,11 @@ enum color_diff {\n \tDIFF_FILE_NEW = 5,\n \tDIFF_COMMIT = 6,\n \tDIFF_WHITESPACE = 7,\n-\tDIFF_FUNCINFO = 8\n+\tDIFF_FUNCINFO = 8,\n+\tDIFF_FILE_OLD_MOVED = 9,\n+\tDIFF_FILE_OLD_MOVED_ALT = 10,\n+\tDIFF_FILE_NEW_MOVED = 11,\n+\tDIFF_FILE_NEW_MOVED_ALT = 12\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex 289806d0c7..29704ae14e 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -972,4 +972,265 @@ test_expect_success 'option overrides diff.wsErrorHighlight' '\n \n '\n \n+test_expect_success 'detect moved code, complete file' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >test.c &&\n+\t#include\n+\tmain()\n+\t{\n+\tprintf(\"Hello World\");\n+\t}\n+\tEOF\n+\tgit add test.c &&\n+\tgit commit -m \"add main function\" &&\n+\tgit mv test.c main.c &&\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\tgit diff HEAD --color-moved --no-renames | test_decode_color >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tdiff --git a/main.c b/main.c\n+\tnew file mode 100644\n+\tindex 0000000..a986c57\n+\t--- /dev/null\n+\t+++ b/main.c\n+\t@@ -0,0 +1,5 @@\n+\t+#include\n+\t+main()\n+\t+{\n+\t+printf(\"Hello World\");\n+\t+}\n+\tdiff --git a/test.c b/test.c\n+\tdeleted file mode 100644\n+\tindex a986c57..0000000\n+\t--- a/test.c\n+\t+++ /dev/null\n+\t@@ -1,5 +0,0 @@\n+\t-#include\n+\t-main()\n+\t-{\n+\t-printf(\"Hello World\");\n+\t-}\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'detect malicious moved code, inside file' '\n+\ttest_config color.diff.oldMoved \"normal red\" &&\n+\ttest_config color.diff.newMoved \"normal green\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t\tfoo(u);\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit add main.c test.c &&\n+\tgit commit -m \"add main and test file\" &&\n+\tcat <<-\\EOF >main.c &&\n+\t\t#include\n+\t\tint stuff()\n+\t\t{\n+\t\t\tprintf(\"Hello \");\n+\t\t\tprintf(\"World\\n\");\n+\t\t}\n+\n+\t\tint main()\n+\t\t{\n+\t\t\tfoo();\n+\t\t}\n+\tEOF\n+\tcat <<-\\EOF >test.c &&\n+\t\t#include\n+\t\tint bar()\n+\t\t{\n+\t\t\tprintf(\"Hello World, but different\\n\");\n+\t\t}\n+\n+\t\tint secure_foo(struct user *u)\n+\t\t{\n+\t\t\tfoo(u);\n+\t\t\tif (!u->is_allowed_foo)\n+\t\t\t\treturn;\n+\t\t}\n+\n+\t\tint another_function()\n+\t\t{\n+\t\t\tbar();\n+\t\t}\n+\tEOF\n+\tgit diff HEAD --no-renames --color-moved=zebra| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/main.c b/main.c\n+\tindex 27a619c..7cf9336 100644\n+\t--- a/main.c\n+\t+++ b/main.c\n+\t@@ -5,13 +5,6 @@ printf(\"Hello \");\n+\t printf(\"World\\n\");\n+\t }\n+\t \n+\t-int secure_foo(struct user *u)\n+\t-{\n+\t-if (!u->is_allowed_foo)\n+\t-return;\n+\t-foo(u);\n+\t-}\n+\t-\n+\t int main()\n+\t {\n+\t foo();\n+\tdiff --git a/test.c b/test.c\n+\tindex 1dc1d85..2bedec9 100644\n+\t--- a/test.c\n+\t+++ b/test.c\n+\t@@ -4,6 +4,13 @@ int bar()\n+\t printf(\"Hello World, but different\\n\");\n+\t }\n+\t \n+\t+int secure_foo(struct user *u)\n+\t+{\n+\t+foo(u);\n+\t+if (!u->is_allowed_foo)\n+\t+return;\n+\t+}\n+\t+\n+\t int another_function()\n+\t {\n+\t bar();\n+\tEOF\n+\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'no effect from --color-moved with --word-diff' '\n+\tcat <<-\\EOF >text.txt &&\n+\tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n+\tEOF\n+\tgit add text.txt &&\n+\tgit commit -a -m \"clean state\" &&\n+\tcat <<-\\EOF >text.txt &&\n+\tsimply Lorem Ipsum dummy is text of the typesetting and printing industry.\n+\tEOF\n+\tgit diff --color-moved --word-diff >actual &&\n+\tgit diff --word-diff >expect &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'move detection ignoring whitespace ' '\n+\tgit reset --hard &&\n+\tcat <<\\EOF >lines.txt &&\n+line 1\n+line 2\n+line 3\n+line 4\n+line 5\n+line 6\n+line 7\n+EOF\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<\\EOF >lines.txt &&\n+\tline 5\n+\tline 6\n+\tline 7\n+line 1\n+line 2\n+line 3\n+line 4\n+EOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 734156d..eb89ead 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,7 +1,7 @@\n+\t+\tline 5\n+\t+\tline 6\n+\t+\tline 7\n+\t line 1\n+\t line 2\n+\t line 3\n+\t line 4\n+\t-line 5\n+\t-line 6\n+\t-line 7\n+\tEOF\n+\ttest_cmp expected actual &&\n+\n+\tgit diff HEAD --no-renames -w --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 734156d..eb89ead 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,7 +1,7 @@\n+\t+\tline 5\n+\t+\tline 6\n+\t+\tline 7\n+\t line 1\n+\t line 2\n+\t line 3\n+\t line 4\n+\t-line 5\n+\t-line 6\n+\t-line 7\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'move detection with submodules' '\n+\ttest_create_repo bananas &&\n+\techo ripe >bananas/recipe &&\n+\tgit -C bananas add recipe &&\n+\ttest_commit fruit &&\n+\ttest_commit -C bananas recipe &&\n+\tgit submodule add ./bananas &&\n+\tgit add bananas &&\n+\tgit commit -a -m \"bananas are like a heavy library?\" &&\n+\techo foul >bananas/recipe &&\n+\techo ripe >fruit.t &&\n+\n+\tgit diff --submodule=diff --color-moved >actual &&\n+\n+\t# no move detection as the moved line is across repository boundaries.\n+\ttest_decode_color decoded_actual &&\n+\t! grep BGREEN decoded_actual &&\n+\t! grep BRED decoded_actual &&\n+\n+\t# nor did we mess with it another way\n+\tgit diff --submodule=diff | test_decode_color >expect &&\n+\ttest_cmp expect decoded_actual\n+'\n+\n test_done\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323635","messageId":"20170630205310.7380-25-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 24/25] diff.c: add dimming to moved line detection","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:09Z","receivedAt":"2017-06-30T20:54:08Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Any lines inside a moved block of code are not interesting. Boundaries\nof blocks are only interesting if they are next to another block of moved\ncode.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n color.h | 2 +\n diff.c | 132 +++++++++++++++++++++++++++++++++++++++++----\n diff.h | 9 +++-\n t/t4015-diff-whitespace.sh | 124 ++++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 254 insertions(+), 13 deletions(-)\n\ndiff --git a/color.h b/color.h\nindex 90627650fc..fd2b688dfb 100644\n--- a/color.h\n+++ b/color.h\n@@ -42,6 +42,8 @@ struct strbuf;\n #define GIT_COLOR_BG_BLUE\t\"\\033[44m\"\n #define GIT_COLOR_BG_MAGENTA\t\"\\033[45m\"\n #define GIT_COLOR_BG_CYAN\t\"\\033[46m\"\n+#define GIT_COLOR_FAINT\t\t\"\\033[2m\"\n+#define GIT_COLOR_FAINT_ITALIC\t\"\\033[2;3m\"\n \n /* A special value meaning \"no color selected\" */\n #define GIT_COLOR_NIL \"NIL\"\ndiff --git a/diff.c b/diff.c\nindex 79ea592388..4af73a7e0c 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -58,10 +58,14 @@ static char diff_colors[][COLOR_MAXLEN] = {\n \tGIT_COLOR_YELLOW,\t/* COMMIT */\n \tGIT_COLOR_BG_RED,\t/* WHITESPACE */\n \tGIT_COLOR_NORMAL,\t/* FUNCINFO */\n-\tGIT_COLOR_MAGENTA,\t/* OLD_MOVED */\n-\tGIT_COLOR_BLUE,\t\t/* OLD_MOVED ALTERNATIVE */\n-\tGIT_COLOR_CYAN,\t\t/* NEW_MOVED */\n-\tGIT_COLOR_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n+\tGIT_COLOR_BOLD_MAGENTA,\t/* OLD_MOVED */\n+\tGIT_COLOR_BOLD_BLUE,\t/* OLD_MOVED ALTERNATIVE */\n+\tGIT_COLOR_FAINT,\t/* OLD_MOVED_DIM */\n+\tGIT_COLOR_FAINT_ITALIC,\t/* OLD_MOVED_ALTERNATIVE_DIM */\n+\tGIT_COLOR_BOLD_CYAN,\t/* NEW_MOVED */\n+\tGIT_COLOR_BOLD_YELLOW,\t/* NEW_MOVED ALTERNATIVE */\n+\tGIT_COLOR_FAINT,\t/* NEW_MOVED_DIM */\n+\tGIT_COLOR_FAINT_ITALIC,\t/* NEW_MOVED_ALTERNATIVE_DIM */\n };\n \n static NORETURN void die_want_option(const char *option_name)\n@@ -91,10 +95,18 @@ static int parse_diff_color_slot(const char *var)\n \t\treturn DIFF_FILE_OLD_MOVED;\n \tif (!strcasecmp(var, \"oldmovedalternative\"))\n \t\treturn DIFF_FILE_OLD_MOVED_ALT;\n+\tif (!strcasecmp(var, \"oldmoveddimmed\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_DIM;\n+\tif (!strcasecmp(var, \"oldmovedalternativedimmed\"))\n+\t\treturn DIFF_FILE_OLD_MOVED_ALT_DIM;\n \tif (!strcasecmp(var, \"newmoved\"))\n \t\treturn DIFF_FILE_NEW_MOVED;\n \tif (!strcasecmp(var, \"newmovedalternative\"))\n \t\treturn DIFF_FILE_NEW_MOVED_ALT;\n+\tif (!strcasecmp(var, \"newmoveddimmed\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_DIM;\n+\tif (!strcasecmp(var, \"newmovedalternativedimmed\"))\n+\t\treturn DIFF_FILE_NEW_MOVED_ALT_DIM;\n \treturn -1;\n }\n \n@@ -262,8 +274,10 @@ static int parse_color_moved(const char *arg)\n \t\treturn COLOR_MOVED_ZEBRA;\n \telse if (!strcmp(arg, \"default\"))\n \t\treturn COLOR_MOVED_DEFAULT;\n+\telse if (!strcmp(arg, \"dimmed_zebra\"))\n+\t\treturn COLOR_MOVED_ZEBRA_DIM;\n \telse\n-\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra', 'plain'\"));\n+\t\treturn error(_(\"color moved setting must be one of 'no', 'default', 'zebra', 'dimmed_zebra', 'plain'\"));\n }\n \n int git_diff_ui_config(const char *var, const char *value, void *cb)\n@@ -649,6 +663,7 @@ enum diff_symbol {\n #define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF\t(1<<16)\n #define DIFF_SYMBOL_MOVED_LINE\t\t\t(1<<17)\n #define DIFF_SYMBOL_MOVED_LINE_ALT\t\t(1<<18)\n+#define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING\t(1<<19)\n #define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n \n /*\n@@ -933,6 +948,67 @@ static void mark_color_as_moved(struct diff_options *o,\n \tfree(pmb);\n }\n \n+#define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \\\n+ (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ALT)\n+static void dim_moved_lines(struct diff_options *o)\n+{\n+\tint n;\n+\tfor (n = 0; n < o->emitted_symbols->nr; n++) {\n+\t\tstruct emitted_diff_symbol *prev = (n != 0) ?\n+\t\t\t\t&o->emitted_symbols->buf[n - 1] : NULL;\n+\t\tstruct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];\n+\t\tstruct emitted_diff_symbol *next =\n+\t\t\t\t(n < o->emitted_symbols->nr - 1) ?\n+\t\t\t\t&o->emitted_symbols->buf[n + 1] : NULL;\n+\n+\t\t/* Not a plus or minus line? */\n+\t\tif (l->s != DIFF_SYMBOL_PLUS && l->s != DIFF_SYMBOL_MINUS)\n+\t\t\tcontinue;\n+\n+\t\t/* Not a moved line? */\n+\t\tif (!(l->flags & DIFF_SYMBOL_MOVED_LINE))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * If prev or next are not a plus or minus line,\n+\t\t * pretend they don't exist\n+\t\t */\n+\t\tif (prev && prev->s != DIFF_SYMBOL_PLUS &&\n+\t\t\t prev->s != DIFF_SYMBOL_MINUS)\n+\t\t\tprev = NULL;\n+\t\tif (next && next->s != DIFF_SYMBOL_PLUS &&\n+\t\t\t next->s != DIFF_SYMBOL_MINUS)\n+\t\t\tnext = NULL;\n+\n+\t\t/* Inside a block? */\n+\t\tif ((prev &&\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK)) &&\n+\t\t (next &&\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK))) {\n+\t\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Check if we are at an interesting bound: */\n+\t\tif (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&\n+\t\t (prev->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))\n+\t\t\tcontinue;\n+\t\tif (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&\n+\t\t (next->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=\n+\t\t (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))\n+\t\t\tcontinue;\n+\n+\t\t/*\n+\t\t * The boundary to prev and next are not interesting,\n+\t\t * so this line is not interesting as a whole\n+\t\t */\n+\t\tl->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;\n+\t}\n+}\n+\n static void emit_line_ws_markup(struct diff_options *o,\n \t\t\t\tconst char *set, const char *reset,\n \t\t\t\tconst char *line, int len, char sign,\n@@ -1007,24 +1083,56 @@ static void emit_diff_symbol_from_struct(struct diff_options *o,\n \t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n \t\tbreak;\n \tcase DIFF_SYMBOL_PLUS:\n-\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);\n-\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);\n-\t\telse\n+\t\t\tbreak;\n+\t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\t}\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '+',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n \t\tbreak;\n \tcase DIFF_SYMBOL_MINUS:\n-\t\tif (flags & DIFF_SYMBOL_MOVED_LINE_ALT)\n+\t\tswitch (flags & (DIFF_SYMBOL_MOVED_LINE |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_ALT:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);\n-\t\telse if (flags & DIFF_SYMBOL_MOVED_LINE)\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE |\n+\t\t DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:\n+\t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_DIM);\n+\t\t\tbreak;\n+\t\tcase DIFF_SYMBOL_MOVED_LINE:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);\n-\t\telse\n+\t\t\tbreak;\n+\t\tdefault:\n \t\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\t}\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n \t\temit_line_ws_markup(o, set, reset, line, len, '-',\n \t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n@@ -5420,6 +5528,8 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n \n \t\t\tadd_lines_to_move_detection(o, &add_lines, &del_lines);\n \t\t\tmark_color_as_moved(o, &add_lines, &del_lines);\n+\t\t\tif (o->color_moved == COLOR_MOVED_ZEBRA_DIM)\n+\t\t\t\tdim_moved_lines(o);\n \n \t\t\thashmap_free(&add_lines, 0);\n \t\t\thashmap_free(&del_lines, 0);\ndiff --git a/diff.h b/diff.h\nindex 4cfd609c54..5755f465de 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -192,6 +192,7 @@ struct diff_options {\n \t\tCOLOR_MOVED_NO = 0,\n \t\tCOLOR_MOVED_PLAIN = 1,\n \t\tCOLOR_MOVED_ZEBRA = 2,\n+\t\tCOLOR_MOVED_ZEBRA_DIM = 3,\n \t} color_moved;\n \t#define COLOR_MOVED_DEFAULT COLOR_MOVED_ZEBRA\n \t#define COLOR_MOVED_MIN_BLOCK_LENGTH 3\n@@ -218,8 +219,12 @@ enum color_diff {\n \tDIFF_FUNCINFO = 8,\n \tDIFF_FILE_OLD_MOVED = 9,\n \tDIFF_FILE_OLD_MOVED_ALT = 10,\n-\tDIFF_FILE_NEW_MOVED = 11,\n-\tDIFF_FILE_NEW_MOVED_ALT = 12\n+\tDIFF_FILE_OLD_MOVED_DIM = 11,\n+\tDIFF_FILE_OLD_MOVED_ALT_DIM = 12,\n+\tDIFF_FILE_NEW_MOVED = 13,\n+\tDIFF_FILE_NEW_MOVED_ALT = 14,\n+\tDIFF_FILE_NEW_MOVED_DIM = 15,\n+\tDIFF_FILE_NEW_MOVED_ALT_DIM = 16\n };\n const char *diff_get_color(int diff_use_color, enum color_diff ix);\n #define diff_get_color_opt(o, ix) \\\ndiff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh\nindex d1d7b99990..c3b697411a 100755\n--- a/t/t4015-diff-whitespace.sh\n+++ b/t/t4015-diff-whitespace.sh\n@@ -1179,6 +1179,130 @@ test_expect_success 'plain moved code, inside file' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success 'detect permutations inside moved code -- dimmed_zebra' '\n+\tgit reset --hard &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\tEOF\n+\tgit add lines.txt &&\n+\tgit commit -m \"add poetry\" &&\n+\tcat <<-\\EOF >lines.txt &&\n+\t\tline 4\n+\t\tline 5\n+\t\tline 6\n+\t\tline 7\n+\t\tline 8\n+\t\tline 9\n+\t\tline 1\n+\t\tline 2\n+\t\tline 3\n+\t\tline 14\n+\t\tline 15\n+\t\tline 16\n+\t\tline 10\n+\t\tline 11\n+\t\tline 12\n+\t\tline 13\n+\tEOF\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\ttest_config color.diff.oldMovedDimmed \"normal magenta\" &&\n+\ttest_config color.diff.newMovedDimmed \"normal cyan\" &&\n+\ttest_config color.diff.oldMovedAlternativeDimmed \"normal blue\" &&\n+\ttest_config color.diff.newMovedAlternativeDimmed \"normal yellow\" &&\n+\tgit diff HEAD --no-renames --color-moved=dimmed_zebra| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'cmd option assumes configured colored-moved' '\n+\ttest_config color.diff.oldMoved \"magenta\" &&\n+\ttest_config color.diff.newMoved \"cyan\" &&\n+\ttest_config color.diff.oldMovedAlternative \"blue\" &&\n+\ttest_config color.diff.newMovedAlternative \"yellow\" &&\n+\ttest_config color.diff.oldMovedDimmed \"normal magenta\" &&\n+\ttest_config color.diff.newMovedDimmed \"normal cyan\" &&\n+\ttest_config color.diff.oldMovedAlternativeDimmed \"normal blue\" &&\n+\ttest_config color.diff.newMovedAlternativeDimmed \"normal yellow\" &&\n+\ttest_config diff.colorMoved zebra &&\n+\tgit diff HEAD --no-renames --color-moved| test_decode_color >actual &&\n+\tcat <<-\\EOF >expected &&\n+\tdiff --git a/lines.txt b/lines.txt\n+\tindex 47ea9c3..ba96a38 100644\n+\t--- a/lines.txt\n+\t+++ b/lines.txt\n+\t@@ -1,16 +1,16 @@\n+\t-line 1\n+\t-line 2\n+\t-line 3\n+\t line 4\n+\t line 5\n+\t line 6\n+\t line 7\n+\t line 8\n+\t line 9\n+\t+line 1\n+\t+line 2\n+\t+line 3\n+\t+line 14\n+\t+line 15\n+\t+line 16\n+\t line 10\n+\t line 11\n+\t line 12\n+\t line 13\n+\t-line 14\n+\t-line 15\n+\t-line 16\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'no effect from --color-moved with --word-diff' '\n \tcat <<-\\EOF >text.txt &&\n \tLorem Ipsum is simply dummy text of the printing and typesetting industry.\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323636","messageId":"20170630205310.7380-20-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 19/25] diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:04Z","receivedAt":"2017-06-30T20:54:11Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 10 +++++++---\n 1 file changed, 7 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex e0d39d04da..5a9c55736d 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -571,6 +571,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,\n \tDIFF_SYMBOL_STATS_LINE,\n \tDIFF_SYMBOL_WORD_DIFF,\n+\tDIFF_SYMBOL_STAT_SEP,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -766,6 +767,9 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_WORD_DIFF:\n \t\tfprintf(o->file, \"%.*s\", len, line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_STAT_SEP:\n+\t\tfputs(o->stat_sep, o->file);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -5077,10 +5081,10 @@ void diff_flush(struct diff_options *options)\n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n \t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);\n-\t\t\tif (options->stat_sep) {\n+\t\t\tif (options->stat_sep)\n \t\t\t\t/* attach patch instead of inline */\n-\t\t\t\tfputs(options->stat_sep, options->file);\n-\t\t\t}\n+\t\t\t\temit_diff_symbol(options, DIFF_SYMBOL_STAT_SEP,\n+\t\t\t\t\t\t NULL, 0, 0);\n \t\t}\n \n \t\tdiff_flush_patch_all_file_pairs(options);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323637","messageId":"20170630205310.7380-17-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 16/25] diff.c: convert emit_binary_diff_body to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:53:01Z","receivedAt":"2017-06-30T20:54:16Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++-----------------\n 1 file changed, 46 insertions(+), 17 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 48f719fb07..f5a14359ae 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,11 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n+\tDIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n+\tDIFF_SYMBOL_BINARY_DIFF_BODY,\n+\tDIFF_SYMBOL_BINARY_DIFF_FOOTER,\n \tDIFF_SYMBOL_SUBMODULE_ADD,\n \tDIFF_SYMBOL_SUBMODULE_DEL,\n \tDIFF_SYMBOL_SUBMODULE_UNTRACKED,\n@@ -635,6 +640,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_SUBMODULE_HEADER:\n \tcase DIFF_SYMBOL_SUBMODULE_ERROR:\n \tcase DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:\n+\tcase DIFF_SYMBOL_BINARY_DIFF_BODY:\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n@@ -706,6 +712,19 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_HEADER:\n \t\tfprintf(o->file, \"%s\", line);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER:\n+\t\tfprintf(o->file, \"%sGIT binary patch\\n\", diff_line_prefix(o));\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA:\n+\t\tfprintf(o->file, \"%sdelta %s\\n\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:\n+\t\tfprintf(o->file, \"%sliteral %s\\n\", diff_line_prefix(o), line);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_BINARY_DIFF_FOOTER:\n+\t\tfputs(diff_line_prefix(o), o->file);\n+\t\tfputc('\\n', o->file);\n+\t\tbreak;\n \tcase DIFF_SYMBOL_REWRITE_DIFF:\n \t\tfraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -2390,8 +2409,8 @@ static unsigned char *deflate_it(char *data,\n \treturn deflated;\n }\n \n-static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t\t const char *prefix)\n+static void emit_binary_diff_body(struct diff_options *o,\n+\t\t\t\t mmfile_t *one, mmfile_t *two)\n {\n \tvoid *cp;\n \tvoid *delta;\n@@ -2420,13 +2439,18 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t}\n \n \tif (delta && delta_size < deflate_size) {\n-\t\tfprintf(file, \"%sdelta %lu\\n\", prefix, orig_size);\n+\t\tchar *s = xstrfmt(\"%lu\", orig_size);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,\n+\t\t\t\t s, strlen(s), 0);\n+\t\tfree(s);\n \t\tfree(deflated);\n \t\tdata = delta;\n \t\tdata_size = delta_size;\n-\t}\n-\telse {\n-\t\tfprintf(file, \"%sliteral %lu\\n\", prefix, two->size);\n+\t} else {\n+\t\tchar *s = xstrfmt(\"%lu\", two->size);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,\n+\t\t\t\t s, strlen(s), 0);\n+\t\tfree(s);\n \t\tfree(delta);\n \t\tdata = deflated;\n \t\tdata_size = deflate_size;\n@@ -2435,8 +2459,9 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t/* emit data encoded in base85 */\n \tcp = data;\n \twhile (data_size) {\n+\t\tint len;\n \t\tint bytes = (52 < data_size) ? 52 : data_size;\n-\t\tchar line[70];\n+\t\tchar line[71];\n \t\tdata_size -= bytes;\n \t\tif (bytes <= 26)\n \t\t\tline[0] = bytes + 'A' - 1;\n@@ -2444,20 +2469,24 @@ static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,\n \t\t\tline[0] = bytes - 26 + 'a' - 1;\n \t\tencode_85(line + 1, cp, bytes);\n \t\tcp = (char *) cp + bytes;\n-\t\tfprintf(file, \"%s\", prefix);\n-\t\tfputs(line, file);\n-\t\tfputc('\\n', file);\n+\n+\t\tlen = strlen(line);\n+\t\tline[len++] = '\\n';\n+\t\tline[len] = '\\0';\n+\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_BODY,\n+\t\t\t\t line, len, 0);\n \t}\n-\tfprintf(file, \"%s\\n\", prefix);\n+\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_FOOTER, NULL, 0, 0);\n \tfree(data);\n }\n \n-static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,\n-\t\t\t const char *prefix)\n+static void emit_binary_diff(struct diff_options *o,\n+\t\t\t mmfile_t *one, mmfile_t *two)\n {\n-\tfprintf(file, \"%sGIT binary patch\\n\", prefix);\n-\temit_binary_diff_body(file, one, two, prefix);\n-\temit_binary_diff_body(file, two, one, prefix);\n+\temit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER, NULL, 0, 0);\n+\temit_binary_diff_body(o, one, two);\n+\temit_binary_diff_body(o, two, one);\n }\n \n int diff_filespec_is_binary(struct diff_filespec *one)\n@@ -2643,7 +2672,7 @@ static void builtin_diff(const char *name_a,\n \t\temit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);\n \t\tstrbuf_reset(&header);\n \t\tif (DIFF_OPT_TST(o, BINARY))\n-\t\t\temit_binary_diff(o->file, &mf1, &mf2, line_prefix);\n+\t\t\temit_binary_diff(o, &mf1, &mf2);\n \t\telse {\n \t\t\tstrbuf_addf(&sb, \"%sBinary files %s and %s differ\\n\",\n \t\t\t\t diff_line_prefix(o), lbl[0], lbl[1]);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323638","messageId":"20170630205310.7380-4-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 03/25] diff.c: factor out diff_flush_patch_all_file_pairs","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:48Z","receivedAt":"2017-06-30T20:54:19Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we want to do more things before and after all filepairs\nare flushed. So factor flushing out all file pairs into its own function\nthat the new code can be plugged in easily.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 17 ++++++++++++-----\n 1 file changed, 12 insertions(+), 5 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 94fdb57927..77ef56a6e4 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -4734,6 +4734,17 @@ void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)\n \t\twarning(_(rename_limit_advice), varname, needed);\n }\n \n+static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n+{\n+\tint i;\n+\tstruct diff_queue_struct *q = &diff_queued_diff;\n+\tfor (i = 0; i < q->nr; i++) {\n+\t\tstruct diff_filepair *p = q->queue[i];\n+\t\tif (check_pair_status(p))\n+\t\t\tdiff_flush_patch(p, o);\n+\t}\n+}\n+\n void diff_flush(struct diff_options *options)\n {\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n@@ -4826,11 +4837,7 @@ void diff_flush(struct diff_options *options)\n \t\t\t}\n \t\t}\n \n-\t\tfor (i = 0; i < q->nr; i++) {\n-\t\t\tstruct diff_filepair *p = q->queue[i];\n-\t\t\tif (check_pair_status(p))\n-\t\t\t\tdiff_flush_patch(p, options);\n-\t\t}\n+\t\tdiff_flush_patch_all_file_pairs(options);\n \t}\n \n \tif (output_format & DIFF_FORMAT_CALLBACK)\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323639","messageId":"20170630205310.7380-12-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 11/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:56Z","receivedAt":"2017-06-30T20:54:21Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We have to use fprintf instead of emit_line, because we want to emit the\ntab after the color. This is important for ancient versions of gnu patch\nAFAICT, although we probably do not want to feed colored output to the\npatch utility, such that it would not matter if the trailing tab is\ncolored. Keep the corner case as-is though.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 51 ++++++++++++++++++++++++++++++---------------------\n 1 file changed, 30 insertions(+), 21 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex b2b2a19fcf..49b45fef29 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,6 +561,8 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_FILEPAIR_PLUS,\n+\tDIFF_SYMBOL_FILEPAIR_MINUS,\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n \tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n@@ -611,7 +613,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset, *set;\n+\tconst char *context, *reset, *set, *meta;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -673,6 +675,20 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t}\n \t\temit_line(o, context, reset, line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_FILEPAIR_PLUS:\n+\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\", diff_line_prefix(o), meta,\n+\t\t\tline, reset,\n+\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_FILEPAIR_MINUS:\n+\t\tmeta = diff_get_color_opt(o, DIFF_METAINFO);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\", diff_line_prefix(o), meta,\n+\t\t\tline, reset,\n+\t\t\tstrchr(line, ' ') ? \"\\t\" : \"\");\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -844,8 +860,6 @@ static void emit_rewrite_diff(const char *name_a,\n \t\t\t struct diff_options *o)\n {\n \tint lc_a, lc_b;\n-\tconst char *name_a_tab, *name_b_tab;\n-\tconst char *metainfo = diff_get_color(o->use_color, DIFF_METAINFO);\n \tconst char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);\n \tconst char *reset = diff_get_color(o->use_color, DIFF_RESET);\n \tstatic struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;\n@@ -865,8 +879,6 @@ static void emit_rewrite_diff(const char *name_a,\n \n \tname_a += (*name_a == '/');\n \tname_b += (*name_b == '/');\n-\tname_a_tab = strchr(name_a, ' ') ? \"\\t\" : \"\";\n-\tname_b_tab = strchr(name_b, ' ') ? \"\\t\" : \"\";\n \n \tstrbuf_reset(&a_name);\n \tstrbuf_reset(&b_name);\n@@ -893,11 +905,13 @@ static void emit_rewrite_diff(const char *name_a,\n \n \tlc_a = count_lines(data_one, size_one);\n \tlc_b = count_lines(data_two, size_two);\n-\tfprintf(o->file,\n-\t\t\"%s%s--- %s%s%s\\n%s%s+++ %s%s%s\\n%s%s@@ -\",\n-\t\tline_prefix, metainfo, a_name.buf, name_a_tab, reset,\n-\t\tline_prefix, metainfo, b_name.buf, name_b_tab, reset,\n-\t\tline_prefix, fraginfo);\n+\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,\n+\t\t\t a_name.buf, a_name.len, 0);\n+\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n+\t\t\t b_name.buf, b_name.len, 0);\n+\n+\tfprintf(o->file, \"%s%s@@ -\", line_prefix, fraginfo);\n \tif (!o->irreversible_delete)\n \t\tprint_line_count(o->file, lc_a);\n \telse\n@@ -1365,10 +1379,8 @@ static void find_lno(const char *line, struct emit_callback *ecbdata)\n static void fn_out_consume(void *priv, char *line, unsigned long len)\n {\n \tstruct emit_callback *ecbdata = priv;\n-\tconst char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);\n \tconst char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);\n \tstruct diff_options *o = ecbdata->opt;\n-\tconst char *line_prefix = diff_line_prefix(o);\n \n \to->found_changes = 1;\n \n@@ -1379,15 +1391,12 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \t}\n \n \tif (ecbdata->label_path[0]) {\n-\t\tconst char *name_a_tab, *name_b_tab;\n-\n-\t\tname_a_tab = strchr(ecbdata->label_path[0], ' ') ? \"\\t\" : \"\";\n-\t\tname_b_tab = strchr(ecbdata->label_path[1], ' ') ? \"\\t\" : \"\";\n-\n-\t\tfprintf(o->file, \"%s%s--- %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);\n-\t\tfprintf(o->file, \"%s%s+++ %s%s%s\\n\",\n-\t\t\tline_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,\n+\t\t\t\t ecbdata->label_path[0],\n+\t\t\t\t strlen(ecbdata->label_path[0]), 0);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,\n+\t\t\t\t ecbdata->label_path[1],\n+\t\t\t\t strlen(ecbdata->label_path[1]), 0);\n \t\tecbdata->label_path[0] = ecbdata->label_path[1] = NULL;\n \t}\n \n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323640","messageId":"20170630205310.7380-11-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 10/25] diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:55Z","receivedAt":"2017-06-30T20:54:22Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The context marker use the exact same output pattern, so reuse it.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 85bfd9310d..b2b2a19fcf 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -564,6 +564,7 @@ enum diff_symbol {\n \tDIFF_SYMBOL_WORDS_PORCELAIN,\n \tDIFF_SYMBOL_WORDS,\n \tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_CONTEXT_INCOMPLETE,\n \tDIFF_SYMBOL_PLUS,\n \tDIFF_SYMBOL_MINUS,\n \tDIFF_SYMBOL_NO_LF_EOF,\n@@ -622,6 +623,7 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \tcase DIFF_SYMBOL_CONTEXT_FRAGINFO:\n \t\temit_line(o, \"\", \"\", line, len);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT_INCOMPLETE:\n \tcase DIFF_SYMBOL_CONTEXT_MARKER:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n \t\treset = diff_get_color_opt(o, DIFF_RESET);\n@@ -1448,8 +1450,8 @@ static void fn_out_consume(void *priv, char *line, unsigned long len)\n \tdefault:\n \t\t/* incomplete line at the end */\n \t\tecbdata->lno_in_preimage++;\n-\t\temit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),\n-\t\t\t reset, line, len);\n+\t\temit_diff_symbol(o, DIFF_SYMBOL_CONTEXT_INCOMPLETE,\n+\t\t\t\t line, len, 0);\n \t\tbreak;\n \t}\n }\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323641","messageId":"20170630205310.7380-9-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 08/25] diff.c: migrate emit_line_checked to use emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:53Z","receivedAt":"2017-06-30T20:54:26Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a new flags field to emit_diff_symbol, that will be used by\ncontext lines for:\n* white space rules that are applicable (The first 12 bits)\n Take a note in cahe.c as well, when this ws rules are extended we have\n to fix the bits in the flags field.\n* how the rules are evaluated (actually this double encodes the sign\n of the line, but the code is easier to keep this way, bits 13,14,15)\n* if the line a blank line at EOF (bit 16)\n\nThe check if new lines need to be marked up as extra lines at the end of\nfile, is now done unconditionally. That should be ok, as\n'new_blank_line_at_eof' has a quick early return.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n cache.h | 2 ++\n diff.c | 116 +++++++++++++++++++++++++++++++++++++++++-----------------------\n diff.h | 6 ++--\n 3 files changed, 80 insertions(+), 44 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 96055c2229..18aabacba5 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1996,6 +1996,8 @@ void shift_tree_by(const struct object_id *, const struct object_id *, struct ob\n #define WS_TRAILING_SPACE (WS_BLANK_AT_EOL|WS_BLANK_AT_EOF)\n #define WS_DEFAULT_RULE (WS_TRAILING_SPACE|WS_SPACE_BEFORE_TAB|8)\n #define WS_TAB_WIDTH_MASK 077\n+/* All WS_* -- when extended, adapt diff.c emit_symbol */\n+#define WS_RULE_MASK 07777\n extern unsigned whitespace_rule_cfg;\n extern unsigned whitespace_rule(const char *);\n extern unsigned parse_whitespace_rule(const char *);\ndiff --git a/diff.c b/diff.c\nindex 488096b757..e5430d56da 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -561,17 +561,54 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n }\n \n enum diff_symbol {\n+\tDIFF_SYMBOL_CONTEXT,\n+\tDIFF_SYMBOL_PLUS,\n+\tDIFF_SYMBOL_MINUS,\n \tDIFF_SYMBOL_NO_LF_EOF,\n \tDIFF_SYMBOL_CONTEXT_FRAGINFO,\n \tDIFF_SYMBOL_CONTEXT_MARKER,\n \tDIFF_SYMBOL_SEPARATOR\n };\n+/*\n+ * Flags for content lines:\n+ * 0..12 are whitespace rules\n+ * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT\n+ * 16 is marking if the line is blank at EOF\n+ */\n+#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF (1<<16)\n+#define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)\n+\n+static void emit_line_ws_markup(struct diff_options *o,\n+\t\t\t\tconst char *set, const char *reset,\n+\t\t\t\tconst char *line, int len, char sign,\n+\t\t\t\tunsigned ws_rule, int blank_at_eof)\n+{\n+\tconst char *ws = NULL;\n+\n+\tif (o->ws_error_highlight & ws_rule) {\n+\t\tws = diff_get_color_opt(o, DIFF_WHITESPACE);\n+\t\tif (!*ws)\n+\t\t\tws = NULL;\n+\t}\n+\n+\tif (!ws)\n+\t\temit_line_0(o, set, reset, sign, line, len);\n+\telse if (blank_at_eof)\n+\t\t/* Blank line at EOF - paint '+' as well */\n+\t\temit_line_0(o, ws, reset, sign, line, len);\n+\telse {\n+\t\t/* Emit just the prefix, then the rest. */\n+\t\temit_line_0(o, set, reset, sign, \"\", 0);\n+\t\tws_check_emit(line, len, ws_rule,\n+\t\t\t o->file, set, reset, ws);\n+\t}\n+}\n \n static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n-\t\t\t const char *line, int len)\n+\t\t\t const char *line, int len, unsigned flags)\n {\n \tstatic const char *nneof = \" No newline at end of file\\n\";\n-\tconst char *context, *reset;\n+\tconst char *context, *reset, *set;\n \tswitch (s) {\n \tcase DIFF_SYMBOL_NO_LF_EOF:\n \t\tcontext = diff_get_color_opt(o, DIFF_CONTEXT);\n@@ -593,6 +630,25 @@ static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n \t\t\tdiff_line_prefix(o),\n \t\t\to->line_termination);\n \t\tbreak;\n+\tcase DIFF_SYMBOL_CONTEXT:\n+\t\tset = diff_get_color_opt(o, DIFF_CONTEXT);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, ' ',\n+\t\t\t\t flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_PLUS:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_NEW);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, '+',\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK,\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);\n+\t\tbreak;\n+\tcase DIFF_SYMBOL_MINUS:\n+\t\tset = diff_get_color_opt(o, DIFF_FILE_OLD);\n+\t\treset = diff_get_color_opt(o, DIFF_RESET);\n+\t\temit_line_ws_markup(o, set, reset, line, len, '-',\n+\t\t\t\t flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);\n+\t\tbreak;\n \tdefault:\n \t\tdie(\"BUG: unknown diff symbol\");\n \t}\n@@ -609,57 +665,31 @@ static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line\n \treturn ws_blank_line(line, len, ecbdata->ws_rule);\n }\n \n-static void emit_line_checked(const char *reset,\n-\t\t\t struct emit_callback *ecbdata,\n-\t\t\t const char *line, int len,\n-\t\t\t enum color_diff color,\n-\t\t\t unsigned ws_error_highlight,\n-\t\t\t char sign)\n-{\n-\tconst char *set = diff_get_color(ecbdata->color_diff, color);\n-\tconst char *ws = NULL;\n-\n-\tif (ecbdata->opt->ws_error_highlight & ws_error_highlight) {\n-\t\tws = diff_get_color(ecbdata->color_diff, DIFF_WHITESPACE);\n-\t\tif (!*ws)\n-\t\t\tws = NULL;\n-\t}\n-\n-\tif (!ws)\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, line, len);\n-\telse if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))\n-\t\t/* Blank line at EOF - paint '+' as well */\n-\t\temit_line_0(ecbdata->opt, ws, reset, sign, line, len);\n-\telse {\n-\t\t/* Emit just the prefix, then the rest. */\n-\t\temit_line_0(ecbdata->opt, set, reset, sign, \"\", 0);\n-\t\tws_check_emit(line, len, ecbdata->ws_rule,\n-\t\t\t ecbdata->opt->file, set, reset, ws);\n-\t}\n-}\n-\n static void emit_add_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_FILE_NEW, WSEH_NEW, '+');\n+\tunsigned flags = WSEH_NEW | ecbdata->ws_rule;\n+\tif (new_blank_line_at_eof(ecbdata, line, len))\n+\t\tflags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;\n+\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);\n }\n \n static void emit_del_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_FILE_OLD, WSEH_OLD, '-');\n+\tunsigned flags = WSEH_OLD | ecbdata->ws_rule;\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_MINUS, line, len, flags);\n }\n \n static void emit_context_line(const char *reset,\n \t\t\t struct emit_callback *ecbdata,\n \t\t\t const char *line, int len)\n {\n-\temit_line_checked(reset, ecbdata, line, len,\n-\t\t\t DIFF_CONTEXT, WSEH_CONTEXT, ' ');\n+\tunsigned flags = WSEH_CONTEXT | ecbdata->ws_rule;\n+\temit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_CONTEXT, line, len, flags);\n }\n \n static void emit_hunk_header(struct emit_callback *ecbdata,\n@@ -683,7 +713,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \t memcmp(line, atat, 2) ||\n \t !(ep = memmem(line + 2, len - 2, atat, 2))) {\n \t\temit_diff_symbol(ecbdata->opt,\n-\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len);\n+\t\t\t\t DIFF_SYMBOL_CONTEXT_MARKER, line, len, 0);\n \t\treturn;\n \t}\n \tep += 2; /* skip over @@ */\n@@ -719,7 +749,7 @@ static void emit_hunk_header(struct emit_callback *ecbdata,\n \tstrbuf_add(&msgbuf, line + len, org_len - len);\n \tstrbuf_complete_line(&msgbuf);\n \temit_diff_symbol(ecbdata->opt,\n-\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len);\n+\t\t\t DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len, 0);\n \tstrbuf_release(&msgbuf);\n }\n \n@@ -778,7 +808,7 @@ static void emit_rewrite_lines(struct emit_callback *ecb,\n \t\tdata += len;\n \t}\n \tif (!endp)\n-\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0);\n+\t\temit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0, 0);\n }\n \n static void emit_rewrite_diff(const char *name_a,\n@@ -4771,6 +4801,10 @@ static void diff_flush_patch_all_file_pairs(struct diff_options *o)\n {\n \tint i;\n \tstruct diff_queue_struct *q = &diff_queued_diff;\n+\n+\tif (WSEH_NEW & WS_RULE_MASK)\n+\t\tdie(\"BUG: WS rules bit mask overlaps with diff symbol flags\");\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tif (check_pair_status(p))\n@@ -4861,7 +4895,7 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\ndiff --git a/diff.h b/diff.h\nindex 2d442e296f..ea66168454 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -148,9 +148,9 @@ struct diff_options {\n \tint abbrev;\n \tint ita_invisible_in_index;\n /* white-space error highlighting */\n-#define WSEH_NEW 1\n-#define WSEH_CONTEXT 2\n-#define WSEH_OLD 4\n+#define WSEH_NEW (1<<12)\n+#define WSEH_CONTEXT (1<<13)\n+#define WSEH_OLD (1<<14)\n \tunsigned ws_error_highlight;\n \tconst char *prefix;\n \tint prefix_length;\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323642","messageId":"20170630205310.7380-5-sbeller@google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"[PATCHv2 04/25] diff.c: introduce emit_diff_symbol","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T20:52:49Z","receivedAt":"2017-06-30T20:54:30Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we want to buffer all output before emitting it as a\nnew feature (\"markup moved lines\") conceptually cannot be implemented\nin a single pass over the output.\n\nThere are different approaches to buffer all output such as:\n* Buffering on the char level, i.e. we'd have a char[] which would\n grow at approximately 80 characters a line. This would keep the\n output completely unstructured, but might be very easy to implement,\n such as redirecting all output to a temporary file and working off\n that. The later passes over the buffer are quite complicated though,\n because we have to parse back any output and then decide if it should\n be modified.\n\n* Buffer on a line level. As the output is mostly line oriented already,\n this would make sense, but it still is a bit awkward as we'd have to\n make sense of it again by looking at the first characters of a line\n to decide what part of a diff a line is.\n\n* Buffer semantically. Imagine there is a formal grammar for the diff\n output and we'd keep the symbols of this grammar around. This keeps\n the highest level of structure in the buffered data, such that the\n actual memory requirements are less than say the first option. Instead\n of buffering the characters of the line, we'll buffer what we intend\n to do plus additional information for the specifics. An output of\n\n diff --git a/new.txt b/new.txt\n index fa69b07..412428c 100644\n Binary files a/new.txt and b/new.txt differ\n\n could be buffered as\n DIFF_SYMBOL_DIFF_START + new.txt\n DIFF_SYMBOL_INDEX_MODE + fa69b07 412428c \"non-executable\" flag\n DIFF_SYMBOL_BINARY_FILES + new.txt\n\nThis and the following patches introduce the third option of buffering\nby first moving any output to emit_diff_symbol, and then introducing the\nbuffering in this function.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Junio C Hamano \n---\n diff.c | 22 +++++++++++++++++++---\n 1 file changed, 19 insertions(+), 3 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex 77ef56a6e4..4637368d59 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -560,6 +560,24 @@ static void emit_line(struct diff_options *o, const char *set, const char *reset\n \temit_line_0(o, set, reset, line[0], line+1, len-1);\n }\n \n+enum diff_symbol {\n+\tDIFF_SYMBOL_SEPARATOR\n+};\n+\n+static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,\n+\t\t\t const char *line, int len)\n+{\n+\tswitch (s) {\n+\tcase DIFF_SYMBOL_SEPARATOR:\n+\t\tfprintf(o->file, \"%s%c\",\n+\t\t\tdiff_line_prefix(o),\n+\t\t\to->line_termination);\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: unknown diff symbol\");\n+\t}\n+}\n+\n static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)\n {\n \tif (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&\n@@ -4828,9 +4846,7 @@ void diff_flush(struct diff_options *options)\n \n \tif (output_format & DIFF_FORMAT_PATCH) {\n \t\tif (separator) {\n-\t\t\tfprintf(options->file, \"%s%c\",\n-\t\t\t\tdiff_line_prefix(options),\n-\t\t\t\toptions->line_termination);\n+\t\t\temit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0);\n \t\t\tif (options->stat_sep) {\n \t\t\t\t/* attach patch instead of inline */\n \t\t\t\tfputs(options->stat_sep, options->file);\n-- \n2.13.0.31.g9b732c453e\n\n"},{"id":"323643","messageId":"xmqq1sq1qiyg.fsf@gitster.mtv.corp.google.com","threadId":"46290","inReplyTo":"20170630205310.7380-23-sbeller@google.com","subject":"Re: [PATCHv2 22/25] diff.c: color moved lines differently","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-30T21:11:35Z","receivedAt":"2017-06-30T21:11:43Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> +static int next_byte(const char **cp, const char **endp,\n> +\t\t const struct diff_options *diffopt)\n> +{\n> +\tint retval;\n> +\n> +\tif (*cp > *endp)\n> +\t\treturn -1;\n> +\n> +\tif (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_CHANGE)) {\n> +\t\twhile (*cp < *endp && isspace(**cp))\n> +\t\t\t(*cp)++;\n> +\t\t/*\n> +\t\t * After skipping a couple of whitespaces, we still have to\n> +\t\t * account for one space.\n> +\t\t */\n> +\t\treturn (int)' ';\n\nDo we need a cast here?\n\n> +static unsigned get_string_hash(struct emitted_diff_symbol *es, struct diff_options *o)\n> +{\n> +\tif (o->xdl_opts & XDF_WHITESPACE_FLAGS) {\n> +\t\tstatic struct strbuf sb = STRBUF_INIT;\n> +\t\tconst char *ap = es->line, *ae = es->line + es->len;\n> +\t\tint c;\n> +\n> +\t\tstrbuf_reset(&sb);\n> +\t\twhile (ae > ap && isspace(*ae))\n> +\t\t\tae--;\n\nNot testing for the AT_EOL option here? \n\nIt does not make a difference in correctness; two lines that differ\nonly by their trailing whitespaces will be hashed into the same bin\neven when we are not using ignore-whitespace-at-eol, making the \nhashmap a bit less efficient than necessary.\n\nBy the way, this is an unrelated tangent because I think you\ninherited this pattern by copying and pasting from elsewhere, but I\nthink it would be better if we avoid casting the function pointer\ntype like this:\n\n> +\t\tif (o->color_moved) {\n> +\t\t\tstruct hashmap add_lines, del_lines;\n> +\n> +\t\t\thashmap_init(&del_lines,\n> +\t\t\t\t (hashmap_cmp_fn)moved_entry_cmp, o, 0);\n> +\t\t\thashmap_init(&add_lines,\n> +\t\t\t\t (hashmap_cmp_fn)moved_entry_cmp, o, 0);\n\nWhen hashmap_cmp_fn's definition changes, these two calling sites\nwon't be caught as passing a incorrectly typed callback function by\nthe compiler.\n\nInstead, we can match the actual implementation of the callback\nfunction, e.g.\n\n> +static int moved_entry_cmp(const struct diff_options *diffopt,\n> +\t\t\t const struct moved_entry *a,\n> +\t\t\t const struct moved_entry *b,\n> +\t\t\t const void *keydata)\n> +{\n\nto the expected function type, i.e.\n\nstatic int moved_entry_cmp(const void *fndata,\n\t\t\t const void *entry, const void *entry_or_key,\n\t\t\t const void *keydata)\n{\n\tconst struct diff_options *diffopt = fndata;\n\tconst struct moved_entry *a = entry;\n\tconst struct moved_entry *b = entry_or_key;\n\t...\n\nby casting the parameters.\n"},{"id":"323644","messageId":"CAGZ79kbD13RgcTuiegn4J1piFZrNu6bAxCGEKDYLThnVuYQLrg@mail.gmail.com","threadId":"46290","inReplyTo":"xmqq1sq1qiyg.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCHv2 22/25] diff.c: color moved lines differently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-06-30T21:21:03Z","receivedAt":"2017-06-30T21:21:14Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Jun 30, 2017 at 2:11 PM, Junio C Hamano wrote:\n>> + return (int)' ';\n>\n> Do we need a cast here?\n\nNo, I figured it is good to have it here explicitly, though.\nWe can drop that if you have strong preferences one way or another.\n\n>\n>> +static unsigned get_string_hash(struct emitted_diff_symbol *es, struct diff_options *o)\n>> +{\n>> + if (o->xdl_opts & XDF_WHITESPACE_FLAGS) {\n>> + static struct strbuf sb = STRBUF_INIT;\n>> + const char *ap = es->line, *ae = es->line + es->len;\n>> + int c;\n>> +\n>> + strbuf_reset(&sb);\n>> + while (ae > ap && isspace(*ae))\n>> + ae--;\n>\n> Not testing for the AT_EOL option here?\n\nNo, because the other options are stronger than the AT_EOL,\nsuch that as you note it is still correct.\n\nIf in the future, we'd have another new option e.g.\nIGNORE_TAB_BLANK_CONVERSION_BUT_WARN_ON_LENGTH_DIFF\n(useful for python programmers ;)\nthis would break.\n\n>\n> By the way, this is an unrelated tangent because I think you\n> inherited this pattern by copying and pasting from elsewhere, but I\n> think it would be better if we avoid casting the function pointer\n> type like this:\n>\n>> + if (o->color_moved) {\n>> + struct hashmap add_lines, del_lines;\n>> +\n>> + hashmap_init(&del_lines,\n>> + (hashmap_cmp_fn)moved_entry_cmp, o, 0);\n>> + hashmap_init(&add_lines,\n>> + (hashmap_cmp_fn)moved_entry_cmp, o, 0);\n>\n> When hashmap_cmp_fn's definition changes, these two calling sites\n> won't be caught as passing a incorrectly typed callback function by\n> the compiler.\n>\n> Instead, we can match the actual implementation of the callback\n> function, e.g.\n>\n>> +static int moved_entry_cmp(const struct diff_options *diffopt,\n>> + const struct moved_entry *a,\n>> + const struct moved_entry *b,\n>> + const void *keydata)\n>> +{\n>\n> to the expected function type, i.e.\n>\n> static int moved_entry_cmp(const void *fndata,\n> const void *entry, const void *entry_or_key,\n> const void *keydata)\n> {\n> const struct diff_options *diffopt = fndata;\n> const struct moved_entry *a = entry;\n> const struct moved_entry *b = entry_or_key;\n> ...\n>\n> by casting the parameters.\n\nI agree. I can make a cleanup throughout the whole code base,\nbut I would prefer if that is done in a separate series, as this\nis already slightly lengthy.\n\nThanks,\nStefan\n"},{"id":"323646","messageId":"xmqqshihp37x.fsf@gitster.mtv.corp.google.com","threadId":"46290","inReplyTo":"CAGZ79kbD13RgcTuiegn4J1piFZrNu6bAxCGEKDYLThnVuYQLrg@mail.gmail.com","subject":"Re: [PATCHv2 22/25] diff.c: color moved lines differently","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-30T21:36:50Z","receivedAt":"2017-06-30T21:36:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> On Fri, Jun 30, 2017 at 2:11 PM, Junio C Hamano wrote:\n>>> + return (int)' ';\n>>\n>> Do we need a cast here?\n>\n> No, I figured it is good to have it here explicitly, though.\n> We can drop that if you have strong preferences one way or another.\n\nI was merely wondering if you were protecting against a funny\nplatform where ' ' is a negative number ;-)\n\n>>> +static unsigned get_string_hash(struct emitted_diff_symbol *es, struct diff_options *o)\n>>> +{\n>>> + if (o->xdl_opts & XDF_WHITESPACE_FLAGS) {\n>>> + static struct strbuf sb = STRBUF_INIT;\n>>> + const char *ap = es->line, *ae = es->line + es->len;\n>>> + int c;\n>>> +\n>>> + strbuf_reset(&sb);\n>>> + while (ae > ap && isspace(*ae))\n>>> + ae--;\n>>\n>> Not testing for the AT_EOL option here?\n>\n> No, because the other options are stronger than the AT_EOL,\n> such that as you note it is still correct.\n>\n> If in the future, we'd have another new option e.g.\n> IGNORE_TAB_BLANK_CONVERSION_BUT_WARN_ON_LENGTH_DIFF\n> (useful for python programmers ;)\n> this would break.\n\nThat remark makes it sound as if this is a timebomb ticking, but I\ndo not think that is the case. This is merely stripping whitespaces\nat the end; mixing tabs and spaces without changing the indentation\nlevel matters only when you have something that is !isspace() to\nindent to begin with ;-)\n\nSo, the effect of not checking is only a hashmap that is bit less\nefficient than necessary, but is there an undue cost of actually\nchecking and doing this skipping only when we are ignoring the\nwhitespaces at the end of lines?\n\n>> By the way, this is an unrelated tangent because I think you\n> ...\n> I agree. I can make a cleanup throughout the whole code base,\n> but I would prefer if that is done in a separate series, as this\n> is already slightly lengthy.\n\nOh, I 100% agree that it is an unrelated tangent. \n"},{"id":"323647","messageId":"xmqqo9t5p2nf.fsf@gitster.mtv.corp.google.com","threadId":"46290","inReplyTo":"20170630205310.7380-1-sbeller@google.com","subject":"Re: [PATCHv2 00/25] Reroll of sb/diff-color-moved.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-06-30T21:49:08Z","receivedAt":"2017-06-30T21:49:15Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This (obviously) passes the failing tests.\n\nI should have spotted why the rebase of the previous one did not\nwork myself before bugging you X-<.\n\nThanks. Queued.\n"}]}