{"thread":{"id":"50866","subject":"[PATCH v5 0/6] blame: add the ability to ignore commits","startedAt":"2019-04-03T16:02:18Z","lastAt":"2019-04-04T16:37:15Z","messageCount":8,"participants":["Barret Rhoden","SZEDER Gábor"],"isPatch":true,"patchVersion":5,"patchTotal":6},"messages":[{"id":"373055","messageId":"20190403160207.149174-1-brho@google.com","threadId":"50866","inReplyTo":null,"subject":"[PATCH v5 0/6] blame: add the ability to ignore commits","fromName":"Barret Rhoden","fromEmail":"brho@google.com","sentAt":"2019-04-03T16:02:01Z","receivedAt":"2019-04-03T16:02:18Z","isPatch":true,"sender":{"key":"brho@google.com","avatar":null},"body":"This patch set adds the ability to ignore a set of commits and their\nchanges when blaming.  This can be used to ignore a commit deemed 'not\ninteresting,' such as reformatting.\n\nThe last patch in the series is an RFC, and the others could be merged\nwithout it.  That last patch changes the heuristic by which ignored\nlines are attributed to specific lines in the parent commit.  This\nincreases the likelihood of blaming the 'right' commit, where 'right' is\nsubjective.\n\nThe last patch needs a little work still - there's a TODO section in\nits commit message.  It includes some of Michael's code, so if we are\ngoing to keep it, I'd like to sort out authorship correctly.\n\nv4 -> v5\nv4: https://public-inbox.org/git/20190226170648.211847-1-brho@google.com/\n- Changed the handling of blame_entries from ignored commits so that you\n  can use any algorithm you want to map lines from the diff chunk to\n  different parts of the parent commit.\n- fill_origin_blob() optionally can track the offsets of the start of\n  every line, similar to what we do in the scoreboard for the final\n  file.  This can be used by the matching algorithm.  It has no effect\n  if you are not ignoring commits.\n- RFC of a fuzzy/fingerprinting heuristic, based on Michael Platings RFC\n  at https://public-inbox.org/git/20190324235020.49706-2-michael@platin.gs/\n- Made the tests that detect unblamable entries more resilient to\n  different heuristics.\n- Fixed a few bugs:\n\t- tests were not grepping the line number from --line-porcelain\n\t  correctly.\n\t- In the old version, when I passed the \"upper\" part of the\n\t  blame entry to the target and marked unblamable, the suspect\n\t  was incorrectly marked as the parent.  The s_lno was also in\n\t  the parent's address space.\n\nv3 -> v4\nv3: https://public-inbox.org/git/20190212222722.240676-1-brho@google.com/\n- Cleaned up the tests, especially removing usage of sed -i.\n- Squashed the 'tests' commit into the other blame commits.  Let me know\n  if you'd like further squashing.\n\nv2 -> v3\nv2: https://public-inbox.org/git/20190117202919.157326-1-brho@google.com/\n- SHA-1 -> \"object name\", and fixed other comments\n- Changed error string for oidset_parse_file()\n- Adjusted existing fsck tests to handle those string changes\n- Return hash of all zeros for lines we know we cannot identify\n- Allow repeated options for blame.ignoreRevsFile and\n  --ignore-revs-file.  An empty file name resets the list.  Config\n  options are parsed before the command line options.\n- Rebased to master\n- Added regression tests\n\nv1 -> v2\nv1: https://public-inbox.org/git/20190107213013.231514-1-brho@google.com/\n- extracted the skiplist from fsck to avoid duplicating code\n- overhauled the interface and options\n- split out markIgnoredFiles\n- handled merges\n\nBarret Rhoden (6):\n  Move init_skiplist() outside of fsck\n  blame: use a helper function in blame_chunk()\n  blame: optionally track the line starts during fill_blame_origin()\n  blame: add the ability to ignore commits and their changes\n  blame: add a config option to mark ignored lines\n  RFC blame: use a fingerprint heuristic to match ignored lines\n\n Documentation/blame-options.txt |  16 ++\n Documentation/config/blame.txt  |  11 +\n Documentation/git-blame.txt     |   1 +\n blame.c                         | 384 +++++++++++++++++++++++++++-----\n blame.h                         |   6 +\n builtin/blame.c                 |  51 +++++\n fsck.c                          |  37 +--\n oidset.c                        |  35 +++\n oidset.h                        |   8 +\n t/t5504-fetch-receive-strict.sh |  14 +-\n t/t8013-blame-ignore-revs.sh    | 201 +++++++++++++++++\n 11 files changed, 665 insertions(+), 99 deletions(-)\n create mode 100755 t/t8013-blame-ignore-revs.sh\n\n-- \n2.21.0.392.gf8f6787159e-goog\n\n"},{"id":"373056","messageId":"20190403160207.149174-2-brho@google.com","threadId":"50866","inReplyTo":"20190403160207.149174-1-brho@google.com","subject":"[PATCH v5 1/6] Move init_skiplist() outside of fsck","fromName":"Barret Rhoden","fromEmail":"brho@google.com","sentAt":"2019-04-03T16:02:02Z","receivedAt":"2019-04-03T16:02:20Z","isPatch":true,"sender":{"key":"brho@google.com","avatar":null},"body":"init_skiplist() took a file consisting of SHA-1s and comments and added\nthe objects to an oidset.  This functionality is useful for other\ncommands.\n\nSigned-off-by: Barret Rhoden <brho@google.com>\n---\n fsck.c                          | 37 +--------------------------------\n oidset.c                        | 35 +++++++++++++++++++++++++++++++\n oidset.h                        |  8 +++++++\n t/t5504-fetch-receive-strict.sh | 14 ++++++-------\n 4 files changed, 51 insertions(+), 43 deletions(-)\n\ndiff --git a/fsck.c b/fsck.c\nindex 2260adb71e7a..d45534ad90f5 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -181,41 +181,6 @@ static int fsck_msg_type(enum fsck_msg_id msg_id,\n \treturn msg_type;\n }\n \n-static void init_skiplist(struct fsck_options *options, const char *path)\n-{\n-\tFILE *fp;\n-\tstruct strbuf sb = STRBUF_INIT;\n-\tstruct object_id oid;\n-\n-\tfp = fopen(path, \"r\");\n-\tif (!fp)\n-\t\tdie(\"Could not open skip list: %s\", path);\n-\twhile (!strbuf_getline(&sb, fp)) {\n-\t\tconst char *p;\n-\t\tconst char *hash;\n-\n-\t\t/*\n-\t\t * Allow trailing comments, leading whitespace\n-\t\t * (including before commits), and empty or whitespace\n-\t\t * only lines.\n-\t\t */\n-\t\thash = strchr(sb.buf, '#');\n-\t\tif (hash)\n-\t\t\tstrbuf_setlen(&sb, hash - sb.buf);\n-\t\tstrbuf_trim(&sb);\n-\t\tif (!sb.len)\n-\t\t\tcontinue;\n-\n-\t\tif (parse_oid_hex(sb.buf, &oid, &p) || *p != '\\0')\n-\t\t\tdie(\"Invalid SHA-1: %s\", sb.buf);\n-\t\toidset_insert(&options->skiplist, &oid);\n-\t}\n-\tif (ferror(fp))\n-\t\tdie_errno(\"Could not read '%s'\", path);\n-\tfclose(fp);\n-\tstrbuf_release(&sb);\n-}\n-\n static int parse_msg_type(const char *str)\n {\n \tif (!strcmp(str, \"error\"))\n@@ -284,7 +249,7 @@ void fsck_set_msg_types(struct fsck_options *options, const char *values)\n \t\tif (!strcmp(buf, \"skiplist\")) {\n \t\t\tif (equal == len)\n \t\t\t\tdie(\"skiplist requires a path\");\n-\t\t\tinit_skiplist(options, buf + equal + 1);\n+\t\t\toidset_parse_file(&options->skiplist, buf + equal + 1);\n \t\t\tbuf += len + 1;\n \t\t\tcontinue;\n \t\t}\ndiff --git a/oidset.c b/oidset.c\nindex fe4eb921df81..878a1b56af1c 100644\n--- a/oidset.c\n+++ b/oidset.c\n@@ -35,3 +35,38 @@ void oidset_clear(struct oidset *set)\n \tkh_release_oid(&set->set);\n \toidset_init(set, 0);\n }\n+\n+void oidset_parse_file(struct oidset *set, const char *path)\n+{\n+\tFILE *fp;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\tstruct object_id oid;\n+\n+\tfp = fopen(path, \"r\");\n+\tif (!fp)\n+\t\tdie(\"Could not open object name list: %s\", path);\n+\twhile (!strbuf_getline(&sb, fp)) {\n+\t\tconst char *p;\n+\t\tconst char *name;\n+\n+\t\t/*\n+\t\t * Allow trailing comments, leading whitespace\n+\t\t * (including before commits), and empty or whitespace\n+\t\t * only lines.\n+\t\t */\n+\t\tname = strchr(sb.buf, '#');\n+\t\tif (name)\n+\t\t\tstrbuf_setlen(&sb, name - sb.buf);\n+\t\tstrbuf_trim(&sb);\n+\t\tif (!sb.len)\n+\t\t\tcontinue;\n+\n+\t\tif (parse_oid_hex(sb.buf, &oid, &p) || *p != '\\0')\n+\t\t\tdie(\"Invalid object name: %s\", sb.buf);\n+\t\toidset_insert(set, &oid);\n+\t}\n+\tif (ferror(fp))\n+\t\tdie_errno(\"Could not read '%s'\", path);\n+\tfclose(fp);\n+\tstrbuf_release(&sb);\n+}\ndiff --git a/oidset.h b/oidset.h\nindex c9d0f6d3cc8b..c4807749df8d 100644\n--- a/oidset.h\n+++ b/oidset.h\n@@ -73,6 +73,14 @@ int oidset_remove(struct oidset *set, const struct object_id *oid);\n  */\n void oidset_clear(struct oidset *set);\n \n+/**\n+ * Add the contents of the file 'path' to an initialized oidset.  Each line is\n+ * an unabbreviated object name.  Comments begin with '#', and trailing comments\n+ * are allowed.  Leading whitespace and empty or white-space only lines are\n+ * ignored.\n+ */\n+void oidset_parse_file(struct oidset *set, const char *path);\n+\n struct oidset_iter {\n \tkh_oid_t *set;\n \tkhiter_t iter;\ndiff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh\nindex 7bc706873c5b..7184f1d07f90 100755\n--- a/t/t5504-fetch-receive-strict.sh\n+++ b/t/t5504-fetch-receive-strict.sh\n@@ -164,9 +164,9 @@ test_expect_success 'fsck with unsorted skipList' '\n test_expect_success 'fsck with invalid or bogus skipList input' '\n \tgit -c fsck.skipList=/dev/null -c fsck.missingEmail=ignore fsck &&\n \ttest_must_fail git -c fsck.skipList=does-not-exist -c fsck.missingEmail=ignore fsck 2>err &&\n-\ttest_i18ngrep \"Could not open skip list: does-not-exist\" err &&\n+\ttest_i18ngrep \"Could not open object name list: does-not-exist\" err &&\n \ttest_must_fail git -c fsck.skipList=.git/config -c fsck.missingEmail=ignore fsck 2>err &&\n-\ttest_i18ngrep \"Invalid SHA-1: \\[core\\]\" err\n+\ttest_i18ngrep \"Invalid object name: \\[core\\]\" err\n '\n \n test_expect_success 'fsck with other accepted skipList input (comments & empty lines)' '\n@@ -193,7 +193,7 @@ test_expect_success 'fsck no garbage output from comments & empty lines errors'\n test_expect_success 'fsck with invalid abbreviated skipList input' '\n \techo $commit | test_copy_bytes 20 >SKIP.abbreviated &&\n \ttest_must_fail git -c fsck.skipList=SKIP.abbreviated fsck 2>err-abbreviated &&\n-\ttest_i18ngrep \"^fatal: Invalid SHA-1: \" err-abbreviated\n+\ttest_i18ngrep \"^fatal: Invalid object name: \" err-abbreviated\n '\n \n test_expect_success 'fsck with exhaustive accepted skipList input (various types of comments etc.)' '\n@@ -226,10 +226,10 @@ test_expect_success 'push with receive.fsck.skipList' '\n \ttest_must_fail git push --porcelain dst bogus &&\n \tgit --git-dir=dst/.git config receive.fsck.skipList does-not-exist &&\n \ttest_must_fail git push --porcelain dst bogus 2>err &&\n-\ttest_i18ngrep \"Could not open skip list: does-not-exist\" err &&\n+\ttest_i18ngrep \"Could not open object name list: does-not-exist\" err &&\n \tgit --git-dir=dst/.git config receive.fsck.skipList config &&\n \ttest_must_fail git push --porcelain dst bogus 2>err &&\n-\ttest_i18ngrep \"Invalid SHA-1: \\[core\\]\" err &&\n+\ttest_i18ngrep \"Invalid object name: \\[core\\]\" err &&\n \n \tgit --git-dir=dst/.git config receive.fsck.skipList SKIP &&\n \tgit push --porcelain dst bogus\n@@ -255,10 +255,10 @@ test_expect_success 'fetch with fetch.fsck.skipList' '\n \ttest_must_fail git --git-dir=dst/.git fetch \"file://$(pwd)\" $refspec &&\n \tgit --git-dir=dst/.git config fetch.fsck.skipList does-not-exist &&\n \ttest_must_fail git --git-dir=dst/.git fetch \"file://$(pwd)\" $refspec 2>err &&\n-\ttest_i18ngrep \"Could not open skip list: does-not-exist\" err &&\n+\ttest_i18ngrep \"Could not open object name list: does-not-exist\" err &&\n \tgit --git-dir=dst/.git config fetch.fsck.skipList dst/.git/config &&\n \ttest_must_fail git --git-dir=dst/.git fetch \"file://$(pwd)\" $refspec 2>err &&\n-\ttest_i18ngrep \"Invalid SHA-1: \\[core\\]\" err &&\n+\ttest_i18ngrep \"Invalid object name: \\[core\\]\" err &&\n \n \tgit --git-dir=dst/.git config fetch.fsck.skipList dst/.git/SKIP &&\n \tgit --git-dir=dst/.git fetch \"file://$(pwd)\" $refspec\n-- \n2.21.0.392.gf8f6787159e-goog\n\n"},{"id":"373057","messageId":"20190403160207.149174-3-brho@google.com","threadId":"50866","inReplyTo":"20190403160207.149174-1-brho@google.com","subject":"[PATCH v5 2/6] blame: use a helper function in blame_chunk()","fromName":"Barret Rhoden","fromEmail":"brho@google.com","sentAt":"2019-04-03T16:02:03Z","receivedAt":"2019-04-03T16:02:21Z","isPatch":true,"sender":{"key":"brho@google.com","avatar":null},"body":"The same code for splitting a blame_entry at a particular line was used\ntwice in blame_chunk(), and I'll use the helper again in an upcoming\npatch.\n\nSigned-off-by: Barret Rhoden <brho@google.com>\n---\n blame.c | 44 ++++++++++++++++++++++++++++----------------\n 1 file changed, 28 insertions(+), 16 deletions(-)\n\ndiff --git a/blame.c b/blame.c\nindex da57233cbbd9..31e05c1458d8 100644\n--- a/blame.c\n+++ b/blame.c\n@@ -838,6 +838,27 @@ static struct blame_entry *reverse_blame(struct blame_entry *head,\n \treturn tail;\n }\n \n+/*\n+ * Splits a blame entry into two entries at 'len' lines.  The original 'e'\n+ * consists of len lines, i.e. [e->lno, e->lno + len), and the second part,\n+ * which is returned, consists of the remainder: [e->lno + len, e->lno +\n+ * e->num_lines).  The caller needs to sort out the reference counting for the\n+ * new entry's suspect.\n+ */\n+static struct blame_entry *split_blame_at(struct blame_entry *e, int len,\n+\t\t\t\t\t  struct blame_origin *new_suspect)\n+{\n+\tstruct blame_entry *n = xcalloc(1, sizeof(struct blame_entry));\n+\n+\tn->suspect = new_suspect;\n+\tn->lno = e->lno + len;\n+\tn->s_lno = e->s_lno + len;\n+\tn->num_lines = e->num_lines - len;\n+\te->num_lines = len;\n+\te->score = 0;\n+\treturn n;\n+}\n+\n /*\n  * Process one hunk from the patch between the current suspect for\n  * blame_entry e and its parent.  This first blames any unfinished\n@@ -864,14 +885,9 @@ static void blame_chunk(struct blame_entry ***dstq, struct blame_entry ***srcq,\n \t\t */\n \t\tif (e->s_lno + e->num_lines > tlno) {\n \t\t\t/* Move second half to a new record */\n-\t\t\tint len = tlno - e->s_lno;\n-\t\t\tstruct blame_entry *n = xcalloc(1, sizeof (struct blame_entry));\n-\t\t\tn->suspect = e->suspect;\n-\t\t\tn->lno = e->lno + len;\n-\t\t\tn->s_lno = e->s_lno + len;\n-\t\t\tn->num_lines = e->num_lines - len;\n-\t\t\te->num_lines = len;\n-\t\t\te->score = 0;\n+\t\t\tstruct blame_entry *n;\n+\n+\t\t\tn = split_blame_at(e, tlno - e->s_lno, e->suspect);\n \t\t\t/* Push new record to diffp */\n \t\t\tn->next = diffp;\n \t\t\tdiffp = n;\n@@ -918,14 +934,10 @@ static void blame_chunk(struct blame_entry ***dstq, struct blame_entry ***srcq,\n \t\t\t * Move second half to a new record to be\n \t\t\t * processed by later chunks\n \t\t\t */\n-\t\t\tint len = same - e->s_lno;\n-\t\t\tstruct blame_entry *n = xcalloc(1, sizeof (struct blame_entry));\n-\t\t\tn->suspect = blame_origin_incref(e->suspect);\n-\t\t\tn->lno = e->lno + len;\n-\t\t\tn->s_lno = e->s_lno + len;\n-\t\t\tn->num_lines = e->num_lines - len;\n-\t\t\te->num_lines = len;\n-\t\t\te->score = 0;\n+\t\t\tstruct blame_entry *n;\n+\n+\t\t\tn = split_blame_at(e, same - e->s_lno,\n+\t\t\t\t\t   blame_origin_incref(e->suspect));\n \t\t\t/* Push new record to samep */\n \t\t\tn->next = samep;\n \t\t\tsamep = n;\n-- \n2.21.0.392.gf8f6787159e-goog\n\n"},{"id":"373058","messageId":"20190403160207.149174-6-brho@google.com","threadId":"50866","inReplyTo":"20190403160207.149174-1-brho@google.com","subject":"[PATCH v5 5/6] blame: add a config option to mark ignored lines","fromName":"Barret Rhoden","fromEmail":"brho@google.com","sentAt":"2019-04-03T16:02:06Z","receivedAt":"2019-04-03T16:02:24Z","isPatch":true,"sender":{"key":"brho@google.com","avatar":null},"body":"When ignoring commits, the commit that is blamed might not be\nresponsible for the change.  Users might want to know when a particular\nline has a potentially inaccurate blame.\n\nThis patch adds a config option to identify these lines by specifying\nblame.markIgnoredFiles.  When this option is set, each blame line is\nmarked with an '*'.  For example:\n\n278b6158d6fdb (Barret Rhoden  2016-04-11 13:57:54 -0400 26)\n\nappears as:\n\n*278b6158d6fd (Barret Rhoden  2016-04-11 13:57:54 -0400 26)\n\nwhere the '*' is placed before the commit, and the hash has one fewer\ncharacters.\n\nSigned-off-by: Barret Rhoden <brho@google.com>\n---\n Documentation/config/blame.txt |  4 ++++\n blame.c                        |  4 ++++\n blame.h                        |  1 +\n builtin/blame.c                |  9 +++++++++\n t/t8013-blame-ignore-revs.sh   | 33 +++++++++++++++++++++++++++++++++\n 5 files changed, 51 insertions(+)\n\ndiff --git a/Documentation/config/blame.txt b/Documentation/config/blame.txt\nindex 4da2788f306d..9f7f0fcf42c9 100644\n--- a/Documentation/config/blame.txt\n+++ b/Documentation/config/blame.txt\n@@ -26,3 +26,7 @@ blame.ignoreRevsFile::\n \t`#` are ignored.  This option may be repeated multiple times.  Empty\n \tfile names will reset the list of ignored revisions.  This option will\n \tbe handled before the command line option `--ignore-revs-file`.\n+\n+blame.markIgnoredLines::\n+\tMark lines that were changed by an ignored revision with a '*' in the\n+\toutput of linkgit:git-blame[1].\ndiff --git a/blame.c b/blame.c\nindex e0612ac34ba7..c06cbd906658 100644\n--- a/blame.c\n+++ b/blame.c\n@@ -515,6 +515,7 @@ void blame_coalesce(struct blame_scoreboard *sb)\n \tfor (ent = sb->ent; ent && (next = ent->next); ent = next) {\n \t\tif (ent->suspect == next->suspect &&\n \t\t    ent->s_lno + ent->num_lines == next->s_lno &&\n+\t\t    ent->ignored == next->ignored &&\n \t\t    ent->unblamable == next->unblamable) {\n \t\t\tent->num_lines += next->num_lines;\n \t\t\tent->next = next->next;\n@@ -767,6 +768,7 @@ static void split_overlap(struct blame_entry *split,\n \tint chunk_end_lno;\n \tmemset(split, 0, sizeof(struct blame_entry [3]));\n \n+\tsplit[0].ignored = split[1].ignored = split[2].ignored = e->ignored;\n \tsplit[0].unblamable = e->unblamable;\n \tsplit[1].unblamable = e->unblamable;\n \tsplit[2].unblamable = e->unblamable;\n@@ -891,6 +893,7 @@ static struct blame_entry *split_blame_at(struct blame_entry *e, int len,\n \tstruct blame_entry *n = xcalloc(1, sizeof(struct blame_entry));\n \n \tn->suspect = new_suspect;\n+\tn->ignored = e->ignored;\n \tn->unblamable = e->unblamable;\n \tn->lno = e->lno + len;\n \tn->s_lno = e->s_lno + len;\n@@ -954,6 +957,7 @@ static void ignore_blame_entry(struct blame_entry *e,\n \tstruct blame_line_tracker *line_blames;\n \tint entry_len, nr_lines;\n \n+\te->ignored = 1;\n \tline_blames = xcalloc(sizeof(struct blame_line_tracker),\n \t\t\t      e->num_lines);\n \tguess_line_blames(e, parent, target, offset, delta, line_blames);\ndiff --git a/blame.h b/blame.h\nindex 93780b01504c..f7755920c90d 100644\n--- a/blame.h\n+++ b/blame.h\n@@ -94,6 +94,7 @@ struct blame_entry {\n \t * scanning the lines over and over.\n \t */\n \tunsigned score;\n+\tint ignored;\n \tint unblamable;\n };\n \ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 5f38e9dccddd..46d96905de75 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -53,6 +53,7 @@ static int show_progress;\n static char repeated_meta_color[COLOR_MAXLEN];\n static int coloring_mode;\n static struct string_list ignore_revs_file_list = STRING_LIST_INIT_NODUP;\n+static int mark_ignored_lines;\n \n static struct date_mode blame_date_mode = { DATE_ISO8601 };\n static size_t blame_date_width;\n@@ -482,6 +483,10 @@ static void emit_other(struct blame_scoreboard *sb, struct blame_entry *ent, int\n \t\t\t}\n \t\t}\n \n+\t\tif (mark_ignored_lines && ent->ignored) {\n+\t\t\tlength--;\n+\t\t\tputchar('*');\n+\t\t}\n \t\tif (ent->unblamable)\n \t\t\tmemset(hex, '0', length);\n \t\tprintf(\"%.*s\", length, hex);\n@@ -710,6 +715,10 @@ static int git_blame_config(const char *var, const char *value, void *cb)\n \t\tstring_list_insert(&ignore_revs_file_list, str);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"blame.markignoredlines\")) {\n+\t\tmark_ignored_lines = git_config_bool(var, value);\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"color.blame.repeatedlines\")) {\n \t\tif (color_parse_mem(value, strlen(value), repeated_meta_color))\n \t\t\twarning(_(\"invalid color '%s' in color.blame.repeatedLines\"),\ndiff --git a/t/t8013-blame-ignore-revs.sh b/t/t8013-blame-ignore-revs.sh\nindex df4993f98682..c4cd9a6c54be 100755\n--- a/t/t8013-blame-ignore-revs.sh\n+++ b/t/t8013-blame-ignore-revs.sh\n@@ -123,6 +123,39 @@ test_expect_success bad_files_and_revs '\n \ttest_i18ngrep \"Invalid object name: NOREV\" err\n \t'\n \n+# Commit Z will touch the first two lines.  Y touched all four.\n+# \tA--B--X--Y--Z\n+# The blame output when ignoring Z should be:\n+# ^Y ... 1)\n+# ^Y ... 2)\n+# Y  ... 3)\n+# Y  ... 4)\n+# We're checking only the first character\n+test_expect_success mark_ignored_lines '\n+\tgit config --add blame.markIgnoredLines true &&\n+\n+\ttest_write_lines line-one-Z line-two-Z y3 y4 >file &&\n+\tgit add file &&\n+\ttest_tick &&\n+\tgit commit -m Z &&\n+\tgit tag Z &&\n+\n+\tgit blame --ignore-rev Z file >blame_raw &&\n+\techo \"*\" >expect &&\n+\n+\tsed -n \"1p\" blame_raw | cut -c1 >actual &&\n+\ttest_cmp expect actual &&\n+\n+\tsed -n \"2p\" blame_raw | cut -c1 >actual &&\n+\ttest_cmp expect actual &&\n+\n+\tsed -n \"3p\" blame_raw | cut -c1 >actual &&\n+\t! test_cmp expect actual &&\n+\n+\tsed -n \"4p\" blame_raw | cut -c1 >actual &&\n+\t! test_cmp expect actual\n+\t'\n+\n # Resetting the repo and creating:\n #\n # A--B--M\n-- \n2.21.0.392.gf8f6787159e-goog\n\n"},{"id":"373059","messageId":"20190403160207.149174-4-brho@google.com","threadId":"50866","inReplyTo":"20190403160207.149174-1-brho@google.com","subject":"[PATCH v5 3/6] blame: optionally track the line starts during fill_blame_origin()","fromName":"Barret Rhoden","fromEmail":"brho@google.com","sentAt":"2019-04-03T16:02:04Z","receivedAt":"2019-04-03T16:02:25Z","isPatch":true,"sender":{"key":"brho@google.com","avatar":null},"body":"The line_starts array is an index of line number to file offset, so that\nwe can quickly find where a particular line begins in a file.  Prior to\nthis commit, we only tracked this information for the final file image,\ni.e. the current version of the file.\n\nThis commit adds the ability to track this information for any version\nof the file throughout the file's history.  In particular, we track this\ninfo when we load the file's image into memory: fill_blame_origin().\n\nThis feature will be used when we attempt to pass blame entries to\nparents when we \"ignore\" a commit.  Most uses of fill_blame_origin()\nwill not require this feature, hence the flag parameter.  Multiple calls\nto fill_blame_origin() are idempotent, and any of them can request the\ncreation of the line_starts array.\n\nSuggested-by: Michael Platings <michael@platin.gs>\nSigned-off-by: Barret Rhoden <brho@google.com>\n---\n blame.c | 76 ++++++++++++++++++++++++++++++++++-----------------------\n blame.h |  2 ++\n 2 files changed, 48 insertions(+), 30 deletions(-)\n\ndiff --git a/blame.c b/blame.c\nindex 31e05c1458d8..cb5806f955a6 100644\n--- a/blame.c\n+++ b/blame.c\n@@ -310,12 +310,42 @@ static int diff_hunks(mmfile_t *file_a, mmfile_t *file_b,\n \treturn xdi_diff(file_a, file_b, &xpp, &xecfg, &ecb);\n }\n \n+static const char *get_next_line(const char *start, const char *end)\n+{\n+\tconst char *nl = memchr(start, '\\n', end - start);\n+\n+\treturn nl ? nl + 1 : end;\n+}\n+\n+static int find_line_starts(int **line_starts, const char *buf,\n+\t\t\t    unsigned long len)\n+{\n+\tconst char *end = buf + len;\n+\tconst char *p;\n+\tint *lineno;\n+\tint num = 0;\n+\n+\tfor (p = buf; p < end; p = get_next_line(p, end))\n+\t\tnum++;\n+\n+\tALLOC_ARRAY(*line_starts, num + 1);\n+\tlineno = *line_starts;\n+\n+\tfor (p = buf; p < end; p = get_next_line(p, end))\n+\t\t*lineno++ = p - buf;\n+\n+\t*lineno = len;\n+\n+\treturn num;\n+}\n+\n /*\n  * Given an origin, prepare mmfile_t structure to be used by the\n  * diff machinery\n  */\n static void fill_origin_blob(struct diff_options *opt,\n-\t\t\t     struct blame_origin *o, mmfile_t *file, int *num_read_blob)\n+\t\t\t     struct blame_origin *o, mmfile_t *file,\n+\t\t\t     int *num_read_blob, int fill_line_starts)\n {\n \tif (!o->file.ptr) {\n \t\tenum object_type type;\n@@ -339,11 +369,16 @@ static void fill_origin_blob(struct diff_options *opt,\n \t}\n \telse\n \t\t*file = o->file;\n+\tif (fill_line_starts && !o->line_starts)\n+\t\to->num_lines = find_line_starts(&o->line_starts, o->file.ptr,\n+\t\t\t\t\t\to->file.size);\n }\n \n static void drop_origin_blob(struct blame_origin *o)\n {\n \tFREE_AND_NULL(o->file.ptr);\n+\tFREE_AND_NULL(o->line_starts);\n+\to->num_lines = 0;\n }\n \n /*\n@@ -992,8 +1027,10 @@ static void pass_blame_to_parent(struct blame_scoreboard *sb,\n \td.offset = 0;\n \td.dstq = &newdest; d.srcq = &target->suspects;\n \n-\tfill_origin_blob(&sb->revs->diffopt, parent, &file_p, &sb->num_read_blob);\n-\tfill_origin_blob(&sb->revs->diffopt, target, &file_o, &sb->num_read_blob);\n+\tfill_origin_blob(&sb->revs->diffopt, parent, &file_p,\n+\t\t\t &sb->num_read_blob, 0);\n+\tfill_origin_blob(&sb->revs->diffopt, target, &file_o,\n+\t\t\t &sb->num_read_blob, 0);\n \tsb->num_get_patch++;\n \n \tif (diff_hunks(&file_p, &file_o, blame_chunk_cb, &d, sb->xdl_opts))\n@@ -1199,7 +1236,8 @@ static void find_move_in_parent(struct blame_scoreboard *sb,\n \tif (!unblamed)\n \t\treturn; /* nothing remains for this target */\n \n-\tfill_origin_blob(&sb->revs->diffopt, parent, &file_p, &sb->num_read_blob);\n+\tfill_origin_blob(&sb->revs->diffopt, parent, &file_p,\n+\t\t\t &sb->num_read_blob, 0);\n \tif (!file_p.ptr)\n \t\treturn;\n \n@@ -1328,7 +1366,8 @@ static void find_copy_in_parent(struct blame_scoreboard *sb,\n \t\t\tnorigin = get_origin(parent, p->one->path);\n \t\t\toidcpy(&norigin->blob_oid, &p->one->oid);\n \t\t\tnorigin->mode = p->one->mode;\n-\t\t\tfill_origin_blob(&sb->revs->diffopt, norigin, &file_p, &sb->num_read_blob);\n+\t\t\tfill_origin_blob(&sb->revs->diffopt, norigin, &file_p,\n+\t\t\t\t\t &sb->num_read_blob, 0);\n \t\t\tif (!file_p.ptr)\n \t\t\t\tcontinue;\n \n@@ -1650,37 +1689,14 @@ void assign_blame(struct blame_scoreboard *sb, int opt)\n \t}\n }\n \n-static const char *get_next_line(const char *start, const char *end)\n-{\n-\tconst char *nl = memchr(start, '\\n', end - start);\n-\treturn nl ? nl + 1 : end;\n-}\n-\n /*\n  * To allow quick access to the contents of nth line in the\n  * final image, prepare an index in the scoreboard.\n  */\n static int prepare_lines(struct blame_scoreboard *sb)\n {\n-\tconst char *buf = sb->final_buf;\n-\tunsigned long len = sb->final_buf_size;\n-\tconst char *end = buf + len;\n-\tconst char *p;\n-\tint *lineno;\n-\tint num = 0;\n-\n-\tfor (p = buf; p < end; p = get_next_line(p, end))\n-\t\tnum++;\n-\n-\tALLOC_ARRAY(sb->lineno, num + 1);\n-\tlineno = sb->lineno;\n-\n-\tfor (p = buf; p < end; p = get_next_line(p, end))\n-\t\t*lineno++ = p - buf;\n-\n-\t*lineno = len;\n-\n-\tsb->num_lines = num;\n+\tsb->num_lines = find_line_starts(&sb->lineno, sb->final_buf,\n+\t\t\t\t\t sb->final_buf_size);\n \treturn sb->num_lines;\n }\n \ndiff --git a/blame.h b/blame.h\nindex be3a895043e0..b418bd2e480d 100644\n--- a/blame.h\n+++ b/blame.h\n@@ -51,6 +51,8 @@ struct blame_origin {\n \t */\n \tstruct blame_entry *suspects;\n \tmmfile_t file;\n+\tint num_lines;\n+\tint *line_starts;\n \tstruct object_id blob_oid;\n \tunsigned mode;\n \t/* guilty gets set when shipping any suspects to the final\n-- \n2.21.0.392.gf8f6787159e-goog\n\n"},{"id":"373060","messageId":"20190403160207.149174-7-brho@google.com","threadId":"50866","inReplyTo":"20190403160207.149174-1-brho@google.com","subject":"[PATCH v5 6/6] RFC blame: use a fingerprint heuristic to match ignored lines","fromName":"Barret Rhoden","fromEmail":"brho@google.com","sentAt":"2019-04-03T16:02:07Z","receivedAt":"2019-04-03T16:02:26Z","isPatch":true,"sender":{"key":"brho@google.com","avatar":null},"body":"This replaces the heuristic used to identify lines from ignored commits\nwith one that finds likely candidate lines in the parent's version of\nthe file.\n\nThe old heuristic simply assigned lines in the target to the same line\nnumber (plus offset) in the parent.  The new function uses\nfingerprinting based on the sum of the bitwise-ORed bits in a string.\n\nThe fingerprint code and the idea to use them for blame\ncame from Michael Platings <michael@platin.gs>.\n\nFor each line in the target, guess_line_blames() finds the best line in\nthe parent, above a magic threshold (1 byte pair, currently).  Ties are\nbroken by proximity of the parent line number to the target's line.\n\nHere's an example of the difference between algorithms.  Consider a file\nwith four commits:\n\n\tcommit-a 11) void new_func_1(void *x, void *y);\n\tcommit-b 12) void new_func_2(void *x, void *y);\n\tcommit-c 13) some_line_c\n\tcommit-d 14) some_line_d\n\nAfter a commit 'X', we have:\n\n\tcommit-X 11) void new_func_1(void *x,\n\tcommit-X 12)                 void *y);\n\tcommit-X 13) void new_func_2(void *x,\n\tcommit-X 14)                 void *y);\n\tcommit-c 15) some_line_c\n\tcommit-d 16) some_line_d\n\nWhen we blame-ignored with the old algorithm, we get:\n\n\tcommit-a 11) void new_func_1(void *x,\n\tcommit-b 12)                 void *y);\n\t00000000 13) void new_func_2(void *x,\n\t00000000 14)                 void *y);\n\tcommit-c 15) some_line_c\n\tcommit-d 16) some_line_d\n\nWhere commit-b is blamed for 12 instead of 13.  With the fingerprint\nalgorithm, we get:\n\n\tcommit-a 11) void new_func_1(void *x,\n\tcommit-b 12)                 void *y);\n\tcommit-b 13) void new_func_2(void *x,\n\tcommit-b 14)                 void *y);\n\tcommit-c 15) some_line_c\n\tcommit-d 16) some_line_d\n\nNote both lines 12 and 14 are given to commit b.  Their match is above\nthe FINGERPRINT_THRESHOLD, and they tied.  Specifically, parent lines 11\nand 12 both match these lines.  The algorithm chose parent line 12,\nsince that was closest to the target line numbers of 12 and 14.\n\nIf we increase the threshold, say to 10, those two lines won't match,\nand will be treated as 'unblamable.'\n\nTODO:\n- Is '1' a decent threshold?  Add another user option?\n\n- Can we decrease that FINGERPRINT_LENGTH?  Or do something about the\nTODO about using a set data structure?\n\n- How about matching *outside* the parent's diff hunk?  Right now, this\njust looks in the parent's hunk for a match.  But a better match may be\nelsewhere in the file.  This might happen when the diff has too small of\na -M.  If we do this, then consider caching the fingerprints for a\nparent's entire file, since multiple target blame_entries may check the\nentire parent's space.  Also, if we do this, we probably need to sort the\nparent's blame list (again), since the spliced entries point outside of\nthe diff hunk's range in the parent's address space.\n\n- If we never intend to match outside the parent's diff hunk, then we\nmight be able to short-circuit guess_line_blames() when the parent's\nchunk length is 0.  Doing this somewhat limits the algorithms, and would\nbe a performance optimization, which I didn't want to do without numbers\nsaying we needed it.\n\n- Fix up this commit + message.  I'd be up for splitting it more,\nparticularly if Michael wants his contributions/fingerprinting in his\nown commit.\n\nSuggested-by: Michael Platings <michael@platin.gs>\nSigned-off-by: Barret Rhoden <brho@google.com>\n---\n blame.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++----\n 1 file changed, 87 insertions(+), 5 deletions(-)\n\ndiff --git a/blame.c b/blame.c\nindex c06cbd906658..50511a300059 100644\n--- a/blame.c\n+++ b/blame.c\n@@ -915,27 +915,109 @@ static int are_lines_adjacent(struct blame_line_tracker *first,\n \t       first->s_lno + 1 == second->s_lno;\n }\n \n-/*\n- * This cheap heuristic assigns lines in the chunk to their relative location in\n- * the parent's chunk.  Any additional lines are left with the target.\n+/* https://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel */\n+static int bitcount(uint32_t v)\n+{\n+\tv = v - ((v >> 1) & 0x55555555u);\n+\tv = (v & 0x33333333u) + ((v >> 2) & 0x33333333u);\n+\treturn (((v + (v >> 4)) & 0xf0f0f0fu) * 0x1010101u) >> 24;\n+}\n+\n+#define FINGERPRINT_LENGTH (8 * 256)\n+#define FINGERPRINT_THRESHOLD 1\n+/* This is just a bitset indicating which byte pairs are present.\n+ * e.g. the string \"good goo\" has pairs \"go\", \"oo\", \"od\", \"d \", \" g\"\n+ * String similarity is calculated as a bitwise or and counting the set bits.\n+ *\n+ * TODO for the string lengths we typically deal with, this would probably be\n+ * implemented more efficiently with a set data structure.\n  */\n+struct fingerprint {\n+\tuint32_t bits[FINGERPRINT_LENGTH];\n+};\n+\n+static void get_fingerprint(struct fingerprint *result, const char *line_begin,\n+\t\t\t    const char *line_end)\n+{\n+\tfor (const char *p = line_begin; p + 1 < line_end; ++p) {\n+\t\tunsigned int c = tolower(*p) | (tolower(*(p + 1)) << 8);\n+\n+\t\tresult->bits[c >> 5] |= 1u << (c & 0x1f);\n+\t}\n+}\n+\n+static int fingerprint_similarity(const struct fingerprint *a,\n+\t\t\t\t  const struct fingerprint *b)\n+{\n+\tint intersection = 0;\n+\n+\tfor (int i = 0; i < FINGERPRINT_LENGTH; ++i)\n+\t\tintersection += bitcount(a->bits[i] & b->bits[i]);\n+\treturn intersection;\n+}\n+\n+static void get_chunk_fingerprints(struct fingerprint *fingerprints,\n+\t\t\t\t   const char *content,\n+\t\t\t\t   const int *line_starts,\n+\t\t\t\t   long chunk_start,\n+\t\t\t\t   long chunk_length)\n+{\n+\tline_starts += chunk_start;\n+\tfor (int i = 0; i != chunk_length; ++i) {\n+\t\tconst char *linestart = content + line_starts[i];\n+\t\tconst char *lineend = content + line_starts[i + 1];\n+\n+\t\tget_fingerprint(fingerprints + i, linestart, lineend);\n+\t}\n+}\n+\n static void guess_line_blames(struct blame_entry *e,\n \t\t\t      struct blame_origin *parent,\n \t\t\t      struct blame_origin *target,\n \t\t\t      int offset, int delta,\n \t\t\t      struct blame_line_tracker *line_blames)\n {\n+\tstruct fingerprint *fp_parent, *fp_target;\n \tint nr_parent_lines = e->num_lines - delta;\n \n+\tfp_parent = xcalloc(sizeof(struct fingerprint), nr_parent_lines);\n+\tfp_target = xcalloc(sizeof(struct fingerprint), e->num_lines);\n+\n+\tget_chunk_fingerprints(fp_parent, parent->file.ptr,\n+\t\t\t       parent->line_starts,\n+\t\t\t       e->s_lno + offset, nr_parent_lines);\n+\tget_chunk_fingerprints(fp_target, target->file.ptr,\n+\t\t\t       target->line_starts,\n+\t\t\t       e->s_lno, e->num_lines);\n+\n \tfor (int i = 0; i < e->num_lines; i++) {\n-\t\tif (i < nr_parent_lines) {\n+\t\tint best_sim_val = FINGERPRINT_THRESHOLD;\n+\t\tint best_sim_idx = -1;\n+\t\tint sim;\n+\n+\t\tfor (int j = 0; j < nr_parent_lines; j++) {\n+\t\t\tsim = fingerprint_similarity(&fp_target[i],\n+\t\t\t\t\t\t     &fp_parent[j]);\n+\t\t\tif (sim < best_sim_val)\n+\t\t\t\tcontinue;\n+\t\t\t/* Break ties with the closest-to-target line number */\n+\t\t\tif (sim == best_sim_val && best_sim_idx != -1 &&\n+\t\t\t    abs(best_sim_idx - i) < abs(j - i))\n+\t\t\t\tcontinue;\n+\t\t\tbest_sim_val = sim;\n+\t\t\tbest_sim_idx = j;\n+\t\t}\n+\t\tif (best_sim_idx >= 0) {\n \t\t\tline_blames[i].is_parent = 1;\n-\t\t\tline_blames[i].s_lno = e->s_lno + i + offset;\n+\t\t\tline_blames[i].s_lno = e->s_lno + offset + best_sim_idx;\n \t\t} else {\n \t\t\tline_blames[i].is_parent = 0;\n \t\t\tline_blames[i].s_lno = e->s_lno + i;\n \t\t}\n \t}\n+\n+\tfree(fp_parent);\n+\tfree(fp_target);\n }\n \n /*\n-- \n2.21.0.392.gf8f6787159e-goog\n\n"},{"id":"373061","messageId":"20190403160207.149174-5-brho@google.com","threadId":"50866","inReplyTo":"20190403160207.149174-1-brho@google.com","subject":"[PATCH v5 4/6] blame: add the ability to ignore commits and their changes","fromName":"Barret Rhoden","fromEmail":"brho@google.com","sentAt":"2019-04-03T16:02:05Z","receivedAt":"2019-04-03T16:02:27Z","isPatch":true,"sender":{"key":"brho@google.com","avatar":null},"body":"Commits that make formatting changes or function renames are often not\ninteresting when blaming a file.  A user may deem such a commit as 'not\ninteresting' and want to ignore and its changes it when assigning blame.\n\nFor example, say a file has the following git history / rev-list:\n\n---O---A---X---B---C---D---Y---E---F\n\nCommits X and Y both touch a particular line, and the other commits do\nnot:\n\nX: \"Take a third parameter\"\n-MyFunc(1, 2);\n+MyFunc(1, 2, 3);\n\nY: \"Remove camelcase\"\n-MyFunc(1, 2, 3);\n+my_func(1, 2, 3);\n\ngit-blame will blame Y for the change.  I'd like to be able to ignore Y:\nboth the existence of the commit as well as any changes it made.  This\ndiffers from -S rev-list, which specifies the list of commits to\nprocess for the blame.  We would still process Y, but just don't let the\nblame 'stick.'\n\nThis patch adds the ability for users to ignore a revision with\n--ignore-rev=rev, which may be repeated.  They can specify a set of\nfiles of full object names of revs, e.g. SHA-1 hashes, one per line.  A\nsingle file may be specified with the blame.ignoreRevFile config option\nor with --ignore-rev-file=file.  Both the config option and the command\nline option may be repeated multiple times.  An empty file name \"\" will\nclear the list of revs from previously processed files.  Config options\nare processed before command line options.\n\nFor a typical use case, projects will maintain the file containing\nrevisions for commits that perform mass reformatting, and their users\nhave the optional to ignore all of the commits in that file.\n\nAdditionally, a user can use the --ignore-rev option for one-off\ninvestigation.  To go back to the example above, X was a substantive\nchange to the function, but not the change the user is interested in.\nThe user inspected X, but wanted to find the previous change to that\nline - perhaps a commit that introduced that function call.\n\nTo make this work, we can't simply remove all ignored commits from the\nrev-list.  We need to diff the changes introduced by Y so that we can\nignore them.  We let the blames get passed to Y, just like when\nprocessing normally.  When Y is the target, we make sure that Y does not\n*keep* any blames.  Any changes that Y is responsible for get passed to\nits parent.  Note we make one pass through all of the scapegoats\n(parents) to attempt to pass blame normally; we don't know if we *need*\nto ignore the commit until we've checked all of the parents.\n\nThe blame_entry will get passed up the tree until we find a commit that\nhas a diff chunk that affects those lines.\n\nOne issue is that the ignored commit *did* make some change, and there is\nno general solution to finding the line in the parent commit that\ncorresponds to a given line in the ignored commit.  That makes it hard\nto attribute a particular line within an ignored commit's diff\ncorrectly.\n\nFor example, the parent of an ignored commit has this, say at line 11:\n\ncommit-a 11) #include \"a.h\"\ncommit-b 12) #include \"b.h\"\n\nCommit X, which we will ignore, swaps these lines:\n\ncommit-X 11) #include \"b.h\"\ncommit-X 12) #include \"a.h\"\n\nWe can pass that blame entry to the parent, but line 11 will be\nattributed to commit A, even though \"include b.h\" came from commit B.\nThe blame mechanism will be looking at the parent's view of the file at\nline number 11.\n\nignore_blame_entry() is set up to allow alternative algorithms for\nguessing per-line blames.  Any line that is not attributed to the parent\nis marked as 'unblamable', and we output a hash of all zeros.\n\nThe existing algorithm is simple: blame each line on the corresponding\nline in the parent's diff chunk.  Any lines beyond that stay with the\ntarget.\n\nFor example, the parent of an ignored commit has this, say at line 11:\n\ncommit-a 11) void new_func_1(void *x, void *y);\ncommit-b 12) void new_func_2(void *x, void *y);\ncommit-c 13) some_line_c\ncommit-d 14) some_line_d\n\nAfter a commit 'X', we have:\n\ncommit-X 11) void new_func_1(void *x,\ncommit-X 12)                 void *y);\ncommit-X 13) void new_func_2(void *x,\ncommit-X 14)                 void *y);\ncommit-c 15) some_line_c\ncommit-d 16) some_line_d\n\nCommit X nets two additionally lines: 13 and 14.  The current\nguess_line_blames() algorithm will not attribute these to the parent,\nwhose diff chunk is only two lines - not four.\n\nWhen we ignore with the current algorithm, we get:\n\ncommit-a 11) void new_func_1(void *x,\ncommit-b 12)                 void *y);\n00000000 13) void new_func_2(void *x,\n00000000 14)                 void *y);\ncommit-c 15) some_line_c\ncommit-d 16) some_line_d\n\nNote that line 12 was blamed on B, though B was the commit for\nnew_func_2(), not new_func_1().  Even when guess_line_blames() finds a\nline in the parent, it may still be incorrect.\n\nSigned-off-by: Barret Rhoden <brho@google.com>\n---\n Documentation/blame-options.txt |  16 +++\n Documentation/config/blame.txt  |   7 ++\n Documentation/git-blame.txt     |   1 +\n blame.c                         | 182 +++++++++++++++++++++++++++++---\n blame.h                         |   3 +\n builtin/blame.c                 |  42 ++++++++\n t/t8013-blame-ignore-revs.sh    | 168 +++++++++++++++++++++++++++++\n 7 files changed, 407 insertions(+), 12 deletions(-)\n create mode 100755 t/t8013-blame-ignore-revs.sh\n\ndiff --git a/Documentation/blame-options.txt b/Documentation/blame-options.txt\nindex dc41957afab2..a0a340ef1b73 100644\n--- a/Documentation/blame-options.txt\n+++ b/Documentation/blame-options.txt\n@@ -110,5 +110,21 @@ commit. And the default value is 40. If there are more than one\n `-C` options given, the <num> argument of the last `-C` will\n take effect.\n \n+--ignore-rev <rev>::\n+\tIgnore changes made by the revision when assigning blame, as if the\n+\tchange never happened.  Lines that were changed or added by an ignored\n+\tcommit will be blamed on the previous commit that changed that line or\n+\tnearby lines.  This option may be specified multiple times to ignore\n+\tmore than one revision.  If the `blame.markIgnoredLines` config option\n+\tis set, then lines that were changed by an ignored commit will be\n+\tmarked with a `*` in the blame output.\n+\n+--ignore-revs-file <file>::\n+\tIgnore revisions listed in `file`, one unabbreviated object name per line.\n+\tWhitespace and comments beginning with `#` are ignored.  This option may be\n+\trepeated, and these files will be processed after any files specified with\n+\tthe `blame.ignoreRevsFile` config option.  An empty file name, `\"\"`, will\n+\tclear the list of revs from previously processed files.\n+\n -h::\n \tShow help message.\ndiff --git a/Documentation/config/blame.txt b/Documentation/config/blame.txt\nindex 67b5c1d1e02a..4da2788f306d 100644\n--- a/Documentation/config/blame.txt\n+++ b/Documentation/config/blame.txt\n@@ -19,3 +19,10 @@ blame.showEmail::\n blame.showRoot::\n \tDo not treat root commits as boundaries in linkgit:git-blame[1].\n \tThis option defaults to false.\n+\n+blame.ignoreRevsFile::\n+\tIgnore revisions listed in the file, one unabbreviated object name per\n+\tline, in linkgit:git-blame[1].  Whitespace and comments beginning with\n+\t`#` are ignored.  This option may be repeated multiple times.  Empty\n+\tfile names will reset the list of ignored revisions.  This option will\n+\tbe handled before the command line option `--ignore-revs-file`.\ndiff --git a/Documentation/git-blame.txt b/Documentation/git-blame.txt\nindex 16323eb80e31..7e8154199635 100644\n--- a/Documentation/git-blame.txt\n+++ b/Documentation/git-blame.txt\n@@ -10,6 +10,7 @@ SYNOPSIS\n [verse]\n 'git blame' [-c] [-b] [-l] [--root] [-t] [-f] [-n] [-s] [-e] [-p] [-w] [--incremental]\n \t    [-L <range>] [-S <revs-file>] [-M] [-C] [-C] [-C] [--since=<date>]\n+\t    [--ignore-rev <rev>] [--ignore-revs-file <file>]\n \t    [--progress] [--abbrev=<n>] [<rev> | --contents <file> | --reverse <rev>..<rev>]\n \t    [--] <file>\n \ndiff --git a/blame.c b/blame.c\nindex cb5806f955a6..e0612ac34ba7 100644\n--- a/blame.c\n+++ b/blame.c\n@@ -514,7 +514,8 @@ void blame_coalesce(struct blame_scoreboard *sb)\n \n \tfor (ent = sb->ent; ent && (next = ent->next); ent = next) {\n \t\tif (ent->suspect == next->suspect &&\n-\t\t    ent->s_lno + ent->num_lines == next->s_lno) {\n+\t\t    ent->s_lno + ent->num_lines == next->s_lno &&\n+\t\t    ent->unblamable == next->unblamable) {\n \t\t\tent->num_lines += next->num_lines;\n \t\t\tent->next = next->next;\n \t\t\tblame_origin_decref(next->suspect);\n@@ -766,6 +767,10 @@ static void split_overlap(struct blame_entry *split,\n \tint chunk_end_lno;\n \tmemset(split, 0, sizeof(struct blame_entry [3]));\n \n+\tsplit[0].unblamable = e->unblamable;\n+\tsplit[1].unblamable = e->unblamable;\n+\tsplit[2].unblamable = e->unblamable;\n+\n \tif (e->s_lno < tlno) {\n \t\t/* there is a pre-chunk part not blamed on parent */\n \t\tsplit[0].suspect = blame_origin_incref(e->suspect);\n@@ -886,6 +891,7 @@ static struct blame_entry *split_blame_at(struct blame_entry *e, int len,\n \tstruct blame_entry *n = xcalloc(1, sizeof(struct blame_entry));\n \n \tn->suspect = new_suspect;\n+\tn->unblamable = e->unblamable;\n \tn->lno = e->lno + len;\n \tn->s_lno = e->s_lno + len;\n \tn->num_lines = e->num_lines - len;\n@@ -894,6 +900,106 @@ static struct blame_entry *split_blame_at(struct blame_entry *e, int len,\n \treturn n;\n }\n \n+struct blame_line_tracker {\n+\tint is_parent;\n+\tint s_lno;\n+};\n+\n+static int are_lines_adjacent(struct blame_line_tracker *first,\n+\t\t\t      struct blame_line_tracker *second)\n+{\n+\treturn first->is_parent == second->is_parent &&\n+\t       first->s_lno + 1 == second->s_lno;\n+}\n+\n+/*\n+ * This cheap heuristic assigns lines in the chunk to their relative location in\n+ * the parent's chunk.  Any additional lines are left with the target.\n+ */\n+static void guess_line_blames(struct blame_entry *e,\n+\t\t\t      struct blame_origin *parent,\n+\t\t\t      struct blame_origin *target,\n+\t\t\t      int offset, int delta,\n+\t\t\t      struct blame_line_tracker *line_blames)\n+{\n+\tint nr_parent_lines = e->num_lines - delta;\n+\n+\tfor (int i = 0; i < e->num_lines; i++) {\n+\t\tif (i < nr_parent_lines) {\n+\t\t\tline_blames[i].is_parent = 1;\n+\t\t\tline_blames[i].s_lno = e->s_lno + i + offset;\n+\t\t} else {\n+\t\t\tline_blames[i].is_parent = 0;\n+\t\t\tline_blames[i].s_lno = e->s_lno + i;\n+\t\t}\n+\t}\n+}\n+\n+/*\n+ * This decides which parts of a blame entry go to the parent (added to the\n+ * ignoredp list) and which stay with the target (added to the diffp list).  The\n+ * actual decision is made in a separate heuristic function.  This consumes e,\n+ * essentially putting it on a list.\n+ *\n+ * Note that the blame entries on the ignoredp list are not necessarily sorted\n+ * with respect to the parent's line numbers yet.\n+ */\n+static void ignore_blame_entry(struct blame_entry *e,\n+\t\t\t       struct blame_origin *parent,\n+\t\t\t       struct blame_origin *target,\n+\t\t\t       int offset, int delta,\n+\t\t\t       struct blame_entry **diffp,\n+\t\t\t       struct blame_entry **ignoredp)\n+{\n+\tstruct blame_line_tracker *line_blames;\n+\tint entry_len, nr_lines;\n+\n+\tline_blames = xcalloc(sizeof(struct blame_line_tracker),\n+\t\t\t      e->num_lines);\n+\tguess_line_blames(e, parent, target, offset, delta, line_blames);\n+\t/*\n+\t * We carve new entries off the front of e.  Each entry comes from a\n+\t * contiguous chunk of lines: adjacent lines from the same origin\n+\t * (either the parent or the target).\n+\t */\n+\tentry_len = 1;\n+\tnr_lines = e->num_lines;\t// e changes in the loop\n+\tfor (int i = 0; i < nr_lines; i++) {\n+\t\tstruct blame_entry *next = NULL;\n+\n+\t\t/*\n+\t\t * We are often adjacent to the next line - only split the blame\n+\t\t * entry when we have to.\n+\t\t */\n+\t\tif (i + 1 < nr_lines) {\n+\t\t\tif (are_lines_adjacent(&line_blames[i],\n+\t\t\t\t\t       &line_blames[i + 1])) {\n+\t\t\t\tentry_len++;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t\tnext = split_blame_at(e, entry_len,\n+\t\t\t\t\t      blame_origin_incref(e->suspect));\n+\t\t}\n+\t\tif (line_blames[i].is_parent) {\n+\t\t\tblame_origin_decref(e->suspect);\n+\t\t\te->suspect = blame_origin_incref(parent);\n+\t\t\te->s_lno = line_blames[i - entry_len + 1].s_lno;\n+\t\t\te->next = *ignoredp;\n+\t\t\t*ignoredp = e;\n+\t\t} else {\n+\t\t\te->unblamable = 1;\n+\t\t\t/* e->s_lno is already in the target's address space. */\n+\t\t\te->next = *diffp;\n+\t\t\t*diffp = e;\n+\t\t}\n+\t\tassert(e->num_lines == entry_len);\n+\t\te = next;\n+\t\tentry_len = 1;\n+\t}\n+\tassert(!e);\n+\tfree(line_blames);\n+}\n+\n /*\n  * Process one hunk from the patch between the current suspect for\n  * blame_entry e and its parent.  This first blames any unfinished\n@@ -903,13 +1009,19 @@ static struct blame_entry *split_blame_at(struct blame_entry *e, int len,\n  * -C options may lead to overlapping/duplicate source line number\n  * ranges, all we can rely on from sorting/merging is the order of the\n  * first suspect line number.\n+ *\n+ * tlno: line number in the target where this chunk begins\n+ * same: line number in the target where this chunk ends\n+ * offset: add to tlno to get the chunk starting point in the parent\n+ * delta: net number of lines added by the chunk; target_size - nr_parent_lines\n  */\n static void blame_chunk(struct blame_entry ***dstq, struct blame_entry ***srcq,\n-\t\t\tint tlno, int offset, int same,\n-\t\t\tstruct blame_origin *parent)\n+\t\t\tint tlno, int offset, int same, int delta,\n+\t\t\tstruct blame_origin *parent,\n+\t\t\tstruct blame_origin *target, int ignore_diffs)\n {\n \tstruct blame_entry *e = **srcq;\n-\tstruct blame_entry *samep = NULL, *diffp = NULL;\n+\tstruct blame_entry *samep = NULL, *diffp = NULL, *ignoredp = NULL;\n \n \twhile (e && e->s_lno < tlno) {\n \t\tstruct blame_entry *next = e->next;\n@@ -977,10 +1089,33 @@ static void blame_chunk(struct blame_entry ***dstq, struct blame_entry ***srcq,\n \t\t\tn->next = samep;\n \t\t\tsamep = n;\n \t\t}\n-\t\te->next = diffp;\n-\t\tdiffp = e;\n+\t\tif (ignore_diffs) {\n+\t\t\tignore_blame_entry(e, parent, target, offset, delta,\n+\t\t\t\t\t   &diffp, &ignoredp);\n+\t\t} else {\n+\t\t\te->next = diffp;\n+\t\t\tdiffp = e;\n+\t\t}\n \t\te = next;\n \t}\n+\tif (ignoredp) {\n+\t\tstruct blame_entry *ignoredp_tail;\n+\n+\t\tignoredp = llist_mergesort(ignoredp, get_next_blame,\n+\t\t\t\t\t   set_next_blame,\n+\t\t\t\t\t   compare_blame_suspect);\n+\t\t/*\n+\t\t * We don't use reverse_blame() since the list was already\n+\t\t * reversed when it was sorted.  But we still need to find the\n+\t\t * tail to splice into the dstq list.\n+\t\t */\n+\t\tignoredp_tail = ignoredp;\n+\t\twhile (ignoredp_tail->next)\n+\t\t\tignoredp_tail = ignoredp_tail->next;\n+\t\tignoredp_tail->next = **dstq;\n+\t\t**dstq = ignoredp;\n+\t\t*dstq = &ignoredp_tail->next;\n+\t}\n \t**srcq = reverse_blame(diffp, reverse_blame(samep, e));\n \t/* Move across elements that are in the unblamable portion */\n \tif (diffp)\n@@ -989,7 +1124,9 @@ static void blame_chunk(struct blame_entry ***dstq, struct blame_entry ***srcq,\n \n struct blame_chunk_cb_data {\n \tstruct blame_origin *parent;\n+\tstruct blame_origin *target;\n \tlong offset;\n+\tint ignore_diffs;\n \tstruct blame_entry **dstq;\n \tstruct blame_entry **srcq;\n };\n@@ -1002,7 +1139,8 @@ static int blame_chunk_cb(long start_a, long count_a,\n \tif (start_a - start_b != d->offset)\n \t\tdie(\"internal error in blame::blame_chunk_cb\");\n \tblame_chunk(&d->dstq, &d->srcq, start_b, start_a - start_b,\n-\t\t    start_b + count_b, d->parent);\n+\t\t    start_b + count_b, count_b - count_a, d->parent, d->target,\n+\t\t    d->ignore_diffs);\n \td->offset = start_a + count_a - (start_b + count_b);\n \treturn 0;\n }\n@@ -1014,7 +1152,7 @@ static int blame_chunk_cb(long start_a, long count_a,\n  */\n static void pass_blame_to_parent(struct blame_scoreboard *sb,\n \t\t\t\t struct blame_origin *target,\n-\t\t\t\t struct blame_origin *parent)\n+\t\t\t\t struct blame_origin *parent, int ignore_diffs)\n {\n \tmmfile_t file_p, file_o;\n \tstruct blame_chunk_cb_data d;\n@@ -1024,13 +1162,15 @@ static void pass_blame_to_parent(struct blame_scoreboard *sb,\n \t\treturn; /* nothing remains for this target */\n \n \td.parent = parent;\n+\td.target = target;\n \td.offset = 0;\n+\td.ignore_diffs = ignore_diffs;\n \td.dstq = &newdest; d.srcq = &target->suspects;\n \n \tfill_origin_blob(&sb->revs->diffopt, parent, &file_p,\n-\t\t\t &sb->num_read_blob, 0);\n+\t\t\t &sb->num_read_blob, ignore_diffs);\n \tfill_origin_blob(&sb->revs->diffopt, target, &file_o,\n-\t\t\t &sb->num_read_blob, 0);\n+\t\t\t &sb->num_read_blob, ignore_diffs);\n \tsb->num_get_patch++;\n \n \tif (diff_hunks(&file_p, &file_o, blame_chunk_cb, &d, sb->xdl_opts))\n@@ -1038,7 +1178,8 @@ static void pass_blame_to_parent(struct blame_scoreboard *sb,\n \t\t    oid_to_hex(&parent->commit->object.oid),\n \t\t    oid_to_hex(&target->commit->object.oid));\n \t/* The rest are the same as the parent */\n-\tblame_chunk(&d.dstq, &d.srcq, INT_MAX, d.offset, INT_MAX, parent);\n+\tblame_chunk(&d.dstq, &d.srcq, INT_MAX, d.offset, INT_MAX, INT_MAX,\n+\t\t    parent, target, 0);\n \t*d.dstq = NULL;\n \tqueue_blames(sb, parent, newdest);\n \n@@ -1545,11 +1686,28 @@ static void pass_blame(struct blame_scoreboard *sb, struct blame_origin *origin,\n \t\t\tblame_origin_incref(porigin);\n \t\t\torigin->previous = porigin;\n \t\t}\n-\t\tpass_blame_to_parent(sb, origin, porigin);\n+\t\tpass_blame_to_parent(sb, origin, porigin, 0);\n \t\tif (!origin->suspects)\n \t\t\tgoto finish;\n \t}\n \n+\t/*\n+\t * Pass remaining suspects for ignored commits to their parents.\n+\t */\n+\tif (oidset_contains(&sb->ignore_list, &commit->object.oid)) {\n+\t\tfor (i = 0, sg = first_scapegoat(revs, commit, sb->reverse);\n+\t\t     i < num_sg && sg;\n+\t\t     sg = sg->next, i++) {\n+\t\t\tstruct blame_origin *porigin = sg_origin[i];\n+\n+\t\t\tif (!porigin)\n+\t\t\t\tcontinue;\n+\t\t\tpass_blame_to_parent(sb, origin, porigin, 1);\n+\t\t\tif (!origin->suspects)\n+\t\t\t\tgoto finish;\n+\t\t}\n+\t}\n+\n \t/*\n \t * Optionally find moves in parents' files.\n \t */\ndiff --git a/blame.h b/blame.h\nindex b418bd2e480d..93780b01504c 100644\n--- a/blame.h\n+++ b/blame.h\n@@ -94,6 +94,7 @@ struct blame_entry {\n \t * scanning the lines over and over.\n \t */\n \tunsigned score;\n+\tint unblamable;\n };\n \n /*\n@@ -119,6 +120,8 @@ struct blame_scoreboard {\n \t/* linked list of blames */\n \tstruct blame_entry *ent;\n \n+\tstruct oidset ignore_list;\n+\n \t/* look-up a line in the final buffer */\n \tint num_lines;\n \tint *lineno;\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 581de0d83226..5f38e9dccddd 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -52,6 +52,7 @@ static int no_whole_file_rename;\n static int show_progress;\n static char repeated_meta_color[COLOR_MAXLEN];\n static int coloring_mode;\n+static struct string_list ignore_revs_file_list = STRING_LIST_INIT_NODUP;\n \n static struct date_mode blame_date_mode = { DATE_ISO8601 };\n static size_t blame_date_width;\n@@ -346,6 +347,8 @@ static void emit_porcelain(struct blame_scoreboard *sb, struct blame_entry *ent,\n \tchar hex[GIT_MAX_HEXSZ + 1];\n \n \toid_to_hex_r(hex, &suspect->commit->object.oid);\n+\tif (ent->unblamable)\n+\t\tmemset(hex, '0', strlen(hex));\n \tprintf(\"%s %d %d %d\\n\",\n \t       hex,\n \t       ent->s_lno + 1,\n@@ -479,6 +482,8 @@ static void emit_other(struct blame_scoreboard *sb, struct blame_entry *ent, int\n \t\t\t}\n \t\t}\n \n+\t\tif (ent->unblamable)\n+\t\t\tmemset(hex, '0', length);\n \t\tprintf(\"%.*s\", length, hex);\n \t\tif (opt & OUTPUT_ANNOTATE_COMPAT) {\n \t\t\tconst char *name;\n@@ -695,6 +700,16 @@ static int git_blame_config(const char *var, const char *value, void *cb)\n \t\tparse_date_format(value, &blame_date_mode);\n \t\treturn 0;\n \t}\n+\tif (!strcmp(var, \"blame.ignorerevsfile\")) {\n+\t\tconst char *str;\n+\t\tint ret;\n+\n+\t\tret = git_config_pathname(&str, var, value);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t\tstring_list_insert(&ignore_revs_file_list, str);\n+\t\treturn 0;\n+\t}\n \tif (!strcmp(var, \"color.blame.repeatedlines\")) {\n \t\tif (color_parse_mem(value, strlen(value), repeated_meta_color))\n \t\t\twarning(_(\"invalid color '%s' in color.blame.repeatedLines\"),\n@@ -774,6 +789,27 @@ static int is_a_rev(const char *name)\n \treturn OBJ_NONE < oid_object_info(the_repository, &oid, NULL);\n }\n \n+static void build_ignorelist(struct blame_scoreboard *sb,\n+\t\t\t     struct string_list *ignore_revs_file_list,\n+\t\t\t     struct string_list *ignore_rev_list)\n+{\n+\tstruct string_list_item *i;\n+\tstruct object_id oid;\n+\n+\toidset_init(&sb->ignore_list, 0);\n+\tfor_each_string_list_item(i, ignore_revs_file_list) {\n+\t\tif (!strcmp(i->string, \"\"))\n+\t\t\toidset_clear(&sb->ignore_list);\n+\t\telse\n+\t\t\toidset_parse_file(&sb->ignore_list, i->string);\n+\t}\n+\tfor_each_string_list_item(i, ignore_rev_list) {\n+\t\tif (get_oid_committish(i->string, &oid))\n+\t\t\tdie(_(\"Cannot find revision %s to ignore\"), i->string);\n+\t\toidset_insert(&sb->ignore_list, &oid);\n+\t}\n+}\n+\n int cmd_blame(int argc, const char **argv, const char *prefix)\n {\n \tstruct rev_info revs;\n@@ -785,6 +821,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n \tstruct progress_info pi = { NULL, 0 };\n \n \tstruct string_list range_list = STRING_LIST_INIT_NODUP;\n+\tstruct string_list ignore_rev_list = STRING_LIST_INIT_NODUP;\n \tint output_option = 0, opt = 0;\n \tint show_stats = 0;\n \tconst char *revs_file = NULL;\n@@ -806,6 +843,8 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n \t\tOPT_BIT('s', NULL, &output_option, N_(\"Suppress author name and timestamp (Default: off)\"), OUTPUT_NO_AUTHOR),\n \t\tOPT_BIT('e', \"show-email\", &output_option, N_(\"Show author email instead of name (Default: off)\"), OUTPUT_SHOW_EMAIL),\n \t\tOPT_BIT('w', NULL, &xdl_opts, N_(\"Ignore whitespace differences\"), XDF_IGNORE_WHITESPACE),\n+\t\tOPT_STRING_LIST(0, \"ignore-rev\", &ignore_rev_list, N_(\"rev\"), N_(\"Ignore <rev> when blaming\")),\n+\t\tOPT_STRING_LIST(0, \"ignore-revs-file\", &ignore_revs_file_list, N_(\"file\"), N_(\"Ignore revisions from <file>\")),\n \t\tOPT_BIT(0, \"color-lines\", &output_option, N_(\"color redundant metadata from previous line differently\"), OUTPUT_COLOR_LINE),\n \t\tOPT_BIT(0, \"color-by-age\", &output_option, N_(\"color lines by age\"), OUTPUT_SHOW_AGE_WITH_COLOR),\n \n@@ -999,6 +1038,9 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n \tsb.contents_from = contents_from;\n \tsb.reverse = reverse;\n \tsb.repo = the_repository;\n+\tbuild_ignorelist(&sb, &ignore_revs_file_list, &ignore_rev_list);\n+\tstring_list_clear(&ignore_revs_file_list, 0);\n+\tstring_list_clear(&ignore_rev_list, 0);\n \tsetup_scoreboard(&sb, path, &o);\n \tlno = sb.num_lines;\n \ndiff --git a/t/t8013-blame-ignore-revs.sh b/t/t8013-blame-ignore-revs.sh\nnew file mode 100755\nindex 000000000000..df4993f98682\n--- /dev/null\n+++ b/t/t8013-blame-ignore-revs.sh\n@@ -0,0 +1,168 @@\n+#!/bin/sh\n+\n+test_description='ignore revisions when blaming'\n+. ./test-lib.sh\n+\n+# Creates:\n+# \tA--B--X\n+# A added line 1 and B added line 2.  X makes changes to those lines.  Sanity\n+# check that X is blamed for both lines.\n+test_expect_success setup '\n+\ttest_commit A file line1 &&\n+\n+\techo line2 >>file &&\n+\tgit add file &&\n+\ttest_tick &&\n+\tgit commit -m B &&\n+\tgit tag B &&\n+\n+\ttest_write_lines line-one line-two >file &&\n+\tgit add file &&\n+\ttest_tick &&\n+\tgit commit -m X &&\n+\tgit tag X &&\n+\n+\tgit blame --line-porcelain file >blame_raw &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 1\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse X >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 2\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse X >expect &&\n+\ttest_cmp expect actual\n+\t'\n+\n+# Ignore X, make sure A is blamed for line 1 and B for line 2.\n+test_expect_success ignore_rev_changing_lines '\n+\tgit blame --line-porcelain --ignore-rev X file >blame_raw &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 1\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse A >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 2\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse B >expect &&\n+\ttest_cmp expect actual\n+\t'\n+\n+# For ignored revs that have added 'unblamable' lines, blame those lines on an\n+# all-zeros rev.\n+# \tA--B--X--Y\n+# Where Y changes lines 1 and 2, and adds lines 3 and 4.  The added lines ought\n+# to have nothing in common with \"line-one\" or \"line-two\", to keep any\n+# heuristics from matching them with any lines in the parent.\n+test_expect_success ignore_rev_adding_unblamable_lines '\n+\ttest_write_lines line-one-change line-two-changed y3 y4 >file &&\n+\tgit add file &&\n+\ttest_tick &&\n+\tgit commit -m Y &&\n+\tgit tag Y &&\n+\n+\tgit rev-parse Y >y_rev &&\n+\tsed -e \"s/[0-9a-f]/0/g\" y_rev >expect &&\n+\tgit blame --line-porcelain file --ignore-rev Y >blame_raw &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 3\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\ttest_cmp expect actual &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 4\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\ttest_cmp expect actual\n+\t'\n+\n+# Ignore X and Y, both in separate files.  Lines 1 == A, 2 == B.\n+test_expect_success ignore_revs_from_files '\n+\tgit rev-parse X >ignore_x &&\n+\tgit rev-parse Y >ignore_y &&\n+\tgit blame --line-porcelain file --ignore-revs-file ignore_x --ignore-revs-file ignore_y >blame_raw &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 1\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse A >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 2\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse B >expect &&\n+\ttest_cmp expect actual\n+\t'\n+\n+# Ignore X from the config option, Y from a file.\n+test_expect_success ignore_revs_from_configs_and_files '\n+\tgit config --add blame.ignoreRevsFile ignore_x &&\n+\tgit blame --line-porcelain file --ignore-revs-file ignore_y >blame_raw &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 1\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse A >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 2\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse B >expect &&\n+\ttest_cmp expect actual\n+\t'\n+\n+# Override blame.ignoreRevsFile (ignore_x) with an empty string.  X should be\n+# blamed now for lines 1 and 2, since we are no longer ignoring X.\n+test_expect_success override_ignore_revs_file '\n+\tgit blame --line-porcelain file --ignore-revs-file \"\" --ignore-revs-file ignore_y >blame_raw &&\n+\tgit rev-parse X >expect &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 1\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\ttest_cmp expect actual &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 2\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\ttest_cmp expect actual\n+\t'\n+test_expect_success bad_files_and_revs '\n+\ttest_must_fail git blame file --ignore-rev NOREV 2>err &&\n+\ttest_i18ngrep \"Cannot find revision NOREV to ignore\" err &&\n+\n+\ttest_must_fail git blame file --ignore-revs-file NOFILE 2>err &&\n+\ttest_i18ngrep \"Could not open object name list: NOFILE\" err &&\n+\n+\techo NOREV >ignore_norev &&\n+\ttest_must_fail git blame file --ignore-revs-file ignore_norev 2>err &&\n+\ttest_i18ngrep \"Invalid object name: NOREV\" err\n+\t'\n+\n+# Resetting the repo and creating:\n+#\n+# A--B--M\n+#  \\   /\n+#   C-+\n+#\n+# 'A' creates a file.  B changes line 1, and C changes line 9.  M merges.\n+test_expect_success ignore_merge '\n+\trm -rf .git/ &&\n+\tgit init &&\n+\n+\ttest_write_lines L1 L2 L3 L4 L5 L6 L7 L8 L9 >file &&\n+\tgit add file &&\n+\ttest_tick &&\n+\tgit commit -m A &&\n+\tgit tag A &&\n+\n+\ttest_write_lines BB L2 L3 L4 L5 L6 L7 L8 L9 >file &&\n+\tgit add file &&\n+\ttest_tick &&\n+\tgit commit -m B &&\n+\tgit tag B &&\n+\n+\tgit reset --hard A &&\n+\ttest_write_lines L1 L2 L3 L4 L5 L6 L7 L8 CC >file &&\n+\tgit add file &&\n+\ttest_tick &&\n+\tgit commit -m C &&\n+\tgit tag C &&\n+\n+\ttest_merge M B &&\n+\tgit blame --line-porcelain file --ignore-rev M >blame_raw &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 1\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse B >expect &&\n+\ttest_cmp expect actual &&\n+\n+\tgrep \"^[0-9a-f]\\+ [0-9]\\+ 9\" blame_raw | sed -e \"s/ .*//\" >actual &&\n+\tgit rev-parse C >expect &&\n+\ttest_cmp expect actual\n+\t'\n+\n+test_done\n-- \n2.21.0.392.gf8f6787159e-goog\n\n"},{"id":"373123","messageId":"20190404163707.GP32732@szeder.dev","threadId":"50866","inReplyTo":"20190403160207.149174-7-brho@google.com","subject":"Re: [PATCH v5 6/6] RFC blame: use a fingerprint heuristic to match ignored lines","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-04-04T16:37:07Z","receivedAt":"2019-04-04T16:37:15Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Wed, Apr 03, 2019 at 12:02:07PM -0400, Barret Rhoden wrote:\n> diff --git a/blame.c b/blame.c\n> index c06cbd906658..50511a300059 100644\n> --- a/blame.c\n> +++ b/blame.c\n> @@ -915,27 +915,109 @@ static int are_lines_adjacent(struct blame_line_tracker *first,\n>  \t       first->s_lno + 1 == second->s_lno;\n>  }\n>  \n> -/*\n> - * This cheap heuristic assigns lines in the chunk to their relative location in\n> - * the parent's chunk.  Any additional lines are left with the target.\n> +/* https://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel */\n> +static int bitcount(uint32_t v)\n> +{\n> +\tv = v - ((v >> 1) & 0x55555555u);\n> +\tv = (v & 0x33333333u) + ((v >> 2) & 0x33333333u);\n> +\treturn (((v + (v >> 4)) & 0xf0f0f0fu) * 0x1010101u) >> 24;\n> +}\n> +\n> +#define FINGERPRINT_LENGTH (8 * 256)\n> +#define FINGERPRINT_THRESHOLD 1\n> +/* This is just a bitset indicating which byte pairs are present.\n> + * e.g. the string \"good goo\" has pairs \"go\", \"oo\", \"od\", \"d \", \" g\"\n> + * String similarity is calculated as a bitwise or and counting the set bits.\n> + *\n> + * TODO for the string lengths we typically deal with, this would probably be\n> + * implemented more efficiently with a set data structure.\n>   */\n> +struct fingerprint {\n> +\tuint32_t bits[FINGERPRINT_LENGTH];\n> +};\n> +\n> +static void get_fingerprint(struct fingerprint *result, const char *line_begin,\n> +\t\t\t    const char *line_end)\n> +{\n> +\tfor (const char *p = line_begin; p + 1 < line_end; ++p) {\n\nWe still stick to C89, which doesn't support for loop initial\ndeclarations yet.  Please declare the loop variable as a regular local\nvariable.  This also applies to the several 'for (int i = 0; ...)'\nloops in the functions below.\n\n> +\t\tunsigned int c = tolower(*p) | (tolower(*(p + 1)) << 8);\n> +\n> +\t\tresult->bits[c >> 5] |= 1u << (c & 0x1f);\n> +\t}\n> +}\n> +\n> +static int fingerprint_similarity(const struct fingerprint *a,\n> +\t\t\t\t  const struct fingerprint *b)\n> +{\n> +\tint intersection = 0;\n> +\n> +\tfor (int i = 0; i < FINGERPRINT_LENGTH; ++i)\n> +\t\tintersection += bitcount(a->bits[i] & b->bits[i]);\n> +\treturn intersection;\n> +}\n> +\n> +static void get_chunk_fingerprints(struct fingerprint *fingerprints,\n> +\t\t\t\t   const char *content,\n> +\t\t\t\t   const int *line_starts,\n> +\t\t\t\t   long chunk_start,\n> +\t\t\t\t   long chunk_length)\n> +{\n> +\tline_starts += chunk_start;\n> +\tfor (int i = 0; i != chunk_length; ++i) {\n> +\t\tconst char *linestart = content + line_starts[i];\n> +\t\tconst char *lineend = content + line_starts[i + 1];\n> +\n> +\t\tget_fingerprint(fingerprints + i, linestart, lineend);\n> +\t}\n> +}\n> +\n>  static void guess_line_blames(struct blame_entry *e,\n>  \t\t\t      struct blame_origin *parent,\n>  \t\t\t      struct blame_origin *target,\n>  \t\t\t      int offset, int delta,\n>  \t\t\t      struct blame_line_tracker *line_blames)\n>  {\n> +\tstruct fingerprint *fp_parent, *fp_target;\n>  \tint nr_parent_lines = e->num_lines - delta;\n>  \n> +\tfp_parent = xcalloc(sizeof(struct fingerprint), nr_parent_lines);\n> +\tfp_target = xcalloc(sizeof(struct fingerprint), e->num_lines);\n> +\n> +\tget_chunk_fingerprints(fp_parent, parent->file.ptr,\n> +\t\t\t       parent->line_starts,\n> +\t\t\t       e->s_lno + offset, nr_parent_lines);\n> +\tget_chunk_fingerprints(fp_target, target->file.ptr,\n> +\t\t\t       target->line_starts,\n> +\t\t\t       e->s_lno, e->num_lines);\n> +\n>  \tfor (int i = 0; i < e->num_lines; i++) {\n> -\t\tif (i < nr_parent_lines) {\n> +\t\tint best_sim_val = FINGERPRINT_THRESHOLD;\n> +\t\tint best_sim_idx = -1;\n> +\t\tint sim;\n> +\n> +\t\tfor (int j = 0; j < nr_parent_lines; j++) {\n> +\t\t\tsim = fingerprint_similarity(&fp_target[i],\n> +\t\t\t\t\t\t     &fp_parent[j]);\n> +\t\t\tif (sim < best_sim_val)\n> +\t\t\t\tcontinue;\n> +\t\t\t/* Break ties with the closest-to-target line number */\n> +\t\t\tif (sim == best_sim_val && best_sim_idx != -1 &&\n> +\t\t\t    abs(best_sim_idx - i) < abs(j - i))\n> +\t\t\t\tcontinue;\n> +\t\t\tbest_sim_val = sim;\n> +\t\t\tbest_sim_idx = j;\n> +\t\t}\n> +\t\tif (best_sim_idx >= 0) {\n>  \t\t\tline_blames[i].is_parent = 1;\n> -\t\t\tline_blames[i].s_lno = e->s_lno + i + offset;\n> +\t\t\tline_blames[i].s_lno = e->s_lno + offset + best_sim_idx;\n>  \t\t} else {\n>  \t\t\tline_blames[i].is_parent = 0;\n>  \t\t\tline_blames[i].s_lno = e->s_lno + i;\n>  \t\t}\n>  \t}\n> +\n> +\tfree(fp_parent);\n> +\tfree(fp_target);\n>  }\n>  \n>  /*\n> -- \n> 2.21.0.392.gf8f6787159e-goog\n> \n"}]}