# [PATCH 0/5] pickaxe refactorings and a new mode to find blobs (WAS: diffcore: add a pickaxe option to find a specific blob) 19 messages from 2018-01-03 to 2018-01-04. Participants: Stefan Beller, Jacob Keller, Junio C Hamano. Thread: https://gitlist.dev/t/47525 ## Stefan Beller, 2018-01-03 00:46 Subject: [PATCH 0/5] pickaxe refactorings and a new mode to find blobs (WAS: diffcore: add a pickaxe option to find a specific blob) Message-ID: <20180103004624.222528-1-sbeller@google.com> URL: https://gitlist.dev/e/20180103004624.222528-1-sbeller%40google.com ``` After some discussion [1], we're convinced that the original approach for adding in just another pickaxe mode to find blobs was too hacky. So I went the less hacky way and did some refactoring first (patches 1-3), Then we'll have the new pickaxe mode to find blobs in patch 4. It grew slightly larger as it had issues with the setup (we neither have a regex nor a KWS to init) in this new world, so there are a few more lines in there. The last patch is just the cherry on the cake, helping to keep users sane by warning when they try to use different pickaxe modes at the same time. Thanks, Stefan [1] https://public-inbox.org/git/CAGZ79kaB0G9zetF6QtC45+ZGLM3gOsYWV7e+gkCe2yKOhb0Ssg@mail.gmail.com/ Stefan Beller (5): diff.h: Make pickaxe_opts an unsigned bit field diff: migrate diff_flags.pickaxe_ignore_case to a pickaxe_opts bit diff: introduce DIFF_PICKAXE_KINDS_MASK diffcore: add a pickaxe option to find a specific blob diff: properly error out when combining multiple pickaxe options Documentation/diff-options.txt | 10 +++++++++ builtin/log.c | 4 ++-- combine-diff.c | 2 +- diff.c | 35 +++++++++++++++++++++++++++--- diff.h | 13 ++++++++++-- diffcore-pickaxe.c | 48 ++++++++++++++++++++++++------------------ revision.c | 7 ++++-- 7 files changed, 89 insertions(+), 30 deletions(-) -- 2.15.1.620.gb9897f4670-goog ``` ## Stefan Beller, 2018-01-03 00:46 Subject: [PATCH 1/5] diff.h: Make pickaxe_opts an unsigned bit field Message-ID: <20180103004624.222528-2-sbeller@google.com> URL: https://gitlist.dev/e/20180103004624.222528-2-sbeller%40google.com In-Reply-To: <20180103004624.222528-1-sbeller@google.com> ``` This variable is used as a bit field[1], and as we are about to add more fields, indicate its usage as a bit field by making it unsigned. [1] containing the bits #define DIFF_PICKAXE_ALL 1 #define DIFF_PICKAXE_REGEX 2 #define DIFF_PICKAXE_KIND_S 4 #define DIFF_PICKAXE_KIND_G 8 Signed-off-by: Stefan Beller --- diff.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/diff.h b/diff.h index 0fb18dd735..ea310f76fd 100644 --- a/diff.h +++ b/diff.h @@ -146,7 +146,7 @@ struct diff_options { int skip_stat_unmatch; int line_termination; int output_format; - int pickaxe_opts; + unsigned pickaxe_opts; int rename_score; int rename_limit; int needed_rename_limit; -- 2.15.1.620.gb9897f4670-goog ``` ## Stefan Beller, 2018-01-03 00:46 Subject: [PATCH 2/5] diff: migrate diff_flags.pickaxe_ignore_case to a pickaxe_opts bit Message-ID: <20180103004624.222528-3-sbeller@google.com> URL: https://gitlist.dev/e/20180103004624.222528-3-sbeller%40google.com In-Reply-To: <20180103004624.222528-1-sbeller@google.com> ``` Currently flags for pickaxing are found in different places. Unify the flags into the `pickaxe_opts` field, which will contain any pickaxe related flags. Signed-off-by: Stefan Beller --- diff.h | 3 ++- diffcore-pickaxe.c | 6 +++--- revision.c | 2 +- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/diff.h b/diff.h index ea310f76fd..8af1213684 100644 --- a/diff.h +++ b/diff.h @@ -91,7 +91,6 @@ struct diff_flags { unsigned override_submodule_config:1; unsigned dirstat_by_line:1; unsigned funccontext:1; - unsigned pickaxe_ignore_case:1; unsigned default_follow_renames:1; }; @@ -327,6 +326,8 @@ extern void diff_setup_done(struct diff_options *); #define DIFF_PICKAXE_KIND_S 4 /* traditional plumbing counter */ #define DIFF_PICKAXE_KIND_G 8 /* grep in the patch */ +#define DIFF_PICKAXE_IGNORE_CASE 32 + extern void diffcore_std(struct diff_options *); extern void diffcore_fix_diff_index(struct diff_options *); diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c index 9476bd2108..4b5d88ea46 100644 --- a/diffcore-pickaxe.c +++ b/diffcore-pickaxe.c @@ -222,11 +222,11 @@ void diffcore_pickaxe(struct diff_options *o) if (opts & (DIFF_PICKAXE_REGEX | DIFF_PICKAXE_KIND_G)) { int cflags = REG_EXTENDED | REG_NEWLINE; - if (o->flags.pickaxe_ignore_case) + if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE) cflags |= REG_ICASE; regcomp_or_die(®ex, needle, cflags); regexp = ®ex; - } else if (o->flags.pickaxe_ignore_case && + } else if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE && has_non_ascii(needle)) { struct strbuf sb = STRBUF_INIT; int cflags = REG_NEWLINE | REG_ICASE; @@ -236,7 +236,7 @@ void diffcore_pickaxe(struct diff_options *o) strbuf_release(&sb); regexp = ®ex; } else { - kws = kwsalloc(o->flags.pickaxe_ignore_case + kws = kwsalloc(o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE ? tolower_trans_tbl : NULL); kwsincr(kws, needle, strlen(needle)); kwsprep(kws); diff --git a/revision.c b/revision.c index e2e691dd5a..ccf1d212ce 100644 --- a/revision.c +++ b/revision.c @@ -2076,7 +2076,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg revs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_ERE; } else if (!strcmp(arg, "--regexp-ignore-case") || !strcmp(arg, "-i")) { revs->grep_filter.ignore_case = 1; - revs->diffopt.flags.pickaxe_ignore_case = 1; + revs->diffopt.pickaxe_opts |= DIFF_PICKAXE_IGNORE_CASE; } else if (!strcmp(arg, "--fixed-strings") || !strcmp(arg, "-F")) { revs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_FIXED; } else if (!strcmp(arg, "--perl-regexp") || !strcmp(arg, "-P")) { -- 2.15.1.620.gb9897f4670-goog ``` ## Stefan Beller, 2018-01-03 00:46 Subject: [PATCH 4/5] diffcore: add a pickaxe option to find a specific blob Message-ID: <20180103004624.222528-5-sbeller@google.com> URL: https://gitlist.dev/e/20180103004624.222528-5-sbeller%40google.com In-Reply-To: <20180103004624.222528-1-sbeller@google.com> ``` Sometimes users are given a hash of an object and they want to identify it further (ex.: Use verify-pack to find the largest blobs, but what are these? or [1]) One might be tempted to extend git-describe to also work with blobs, such that `git describe ` gives a description as ':'. This was implemented at [2]; as seen by the sheer number of responses (>110), it turns out this is tricky to get right. The hard part to get right is picking the correct 'commit-ish' as that could be the commit that (re-)introduced the blob or the blob that removed the blob; the blob could exist in different branches. Junio hinted at a different approach of solving this problem, which this patch implements. Teach the diff machinery another flag for restricting the information to what is shown. For example: $ ./git log --oneline --find-object=v2.0.0:Makefile b2feb64309 Revert the whole "ask curl-config" topic for now 47fbfded53 i18n: only extract comments marked with "TRANSLATORS:" we observe that the Makefile as shipped with 2.0 was appeared in v1.9.2-471-g47fbfded53 and in v2.0.0-rc1-5-gb2feb6430b. The reason why these commits both occur prior to v2.0.0 are evil merges that are not found using this new mechanism. [1] https://stackoverflow.com/questions/223678/which-commit-has-this-blob [2] https://public-inbox.org/git/20171028004419.10139-1-sbeller@google.com/ Signed-off-by: Stefan Beller --- Documentation/diff-options.txt | 10 ++++++++++ diff.c | 21 +++++++++++++++++++- diff.h | 8 +++++++- diffcore-pickaxe.c | 45 +++++++++++++++++++++++++----------------- revision.c | 3 +++ 5 files changed, 67 insertions(+), 20 deletions(-) diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt index dd0dba5b1d..f9cf85db05 100644 --- a/Documentation/diff-options.txt +++ b/Documentation/diff-options.txt @@ -492,6 +492,15 @@ occurrences of that string did not change). See the 'pickaxe' entry in linkgit:gitdiffcore[7] for more information. +--find-object=:: + Look for differences that change the number of occurrences of + the specified object. Similar to `-S`, just the argument is different + in that it doesn't search for a specific string but for a specific + object id. ++ +The object can be a blob or a submodule commit. It implies the `-t` option in +`git-log` to also find trees. + --pickaxe-all:: When `-S` or `-G` finds a change, show all the changes in that changeset, not just the files that contain the change @@ -500,6 +509,7 @@ information. --pickaxe-regex:: Treat the given to `-S` as an extended POSIX regular expression to match. + endif::git-format-patch[] -O:: diff --git a/diff.c b/diff.c index 5508745dc8..a872bdcac1 100644 --- a/diff.c +++ b/diff.c @@ -4082,6 +4082,7 @@ void diff_setup(struct diff_options *options) options->interhunkcontext = diff_interhunk_context_default; options->ws_error_highlight = ws_error_highlight_default; options->flags.rename_empty = 1; + options->objfind = NULL; /* pathchange left =NULL by default */ options->change = diff_change; @@ -4487,6 +4488,23 @@ static int parse_ws_error_highlight_opt(struct diff_options *opt, const char *ar return 1; } +static int parse_objfind_opt(struct diff_options *opt, const char *arg) +{ + struct object_id oid; + + if (get_oid(arg, &oid)) + return error("unable to resolve '%s'", arg); + + if (!opt->objfind) + opt->objfind = xcalloc(1, sizeof(*opt->objfind)); + + opt->pickaxe_opts |= DIFF_PICKAXE_KIND_OBJFIND; + opt->flags.recursive = 1; + opt->flags.tree_in_recursive = 1; + oidset_insert(opt->objfind, &oid); + return 1; +} + int diff_opt_parse(struct diff_options *options, const char **av, int ac, const char *prefix) { @@ -4736,7 +4754,8 @@ int diff_opt_parse(struct diff_options *options, else if ((argcount = short_opt('O', av, &optarg))) { options->orderfile = prefix_filename(prefix, optarg); return argcount; - } + } else if (skip_prefix(arg, "--find-object=", &arg)) + return parse_objfind_opt(options, arg); else if ((argcount = parse_long_opt("diff-filter", av, &optarg))) { int offending = parse_diff_filter_opt(optarg, options); if (offending) diff --git a/diff.h b/diff.h index 9ec4f824fe..8a56cac2ad 100644 --- a/diff.h +++ b/diff.h @@ -7,6 +7,7 @@ #include "tree-walk.h" #include "pathspec.h" #include "object.h" +#include "oidset.h" struct rev_info; struct diff_options; @@ -173,6 +174,8 @@ struct diff_options { enum diff_words_type word_diff; enum diff_submodule_format submodule_format; + struct oidset *objfind; + /* this is set by diffcore for DIFF_FORMAT_PATCH */ int found_changes; @@ -325,8 +328,11 @@ extern void diff_setup_done(struct diff_options *); #define DIFF_PICKAXE_KIND_S 4 /* traditional plumbing counter */ #define DIFF_PICKAXE_KIND_G 8 /* grep in the patch */ +#define DIFF_PICKAXE_KIND_OBJFIND 16 /* specific object IDs */ -#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | DIFF_PICKAXE_KIND_G) +#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | \ + DIFF_PICKAXE_KIND_G | \ + DIFF_PICKAXE_KIND_OBJFIND) #define DIFF_PICKAXE_IGNORE_CASE 32 diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c index 4b5d88ea46..72bb5a9514 100644 --- a/diffcore-pickaxe.c +++ b/diffcore-pickaxe.c @@ -124,13 +124,20 @@ static int pickaxe_match(struct diff_filepair *p, struct diff_options *o, mmfile_t mf1, mf2; int ret; - if (!o->pickaxe[0]) - return 0; - /* ignore unmerged */ if (!DIFF_FILE_VALID(p->one) && !DIFF_FILE_VALID(p->two)) return 0; + if (o->objfind) { + return (DIFF_FILE_VALID(p->one) && + oidset_contains(o->objfind, &p->one->oid)) || + (DIFF_FILE_VALID(p->two) && + oidset_contains(o->objfind, &p->two->oid)); + } + + if (!o->pickaxe[0]) + return 0; + if (o->flags.allow_textconv) { textconv_one = get_textconv(p->one); textconv_two = get_textconv(p->two); @@ -226,20 +233,22 @@ void diffcore_pickaxe(struct diff_options *o) cflags |= REG_ICASE; regcomp_or_die(®ex, needle, cflags); regexp = ®ex; - } else if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE && - has_non_ascii(needle)) { - struct strbuf sb = STRBUF_INIT; - int cflags = REG_NEWLINE | REG_ICASE; - - basic_regex_quote_buf(&sb, needle); - regcomp_or_die(®ex, sb.buf, cflags); - strbuf_release(&sb); - regexp = ®ex; - } else { - kws = kwsalloc(o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE - ? tolower_trans_tbl : NULL); - kwsincr(kws, needle, strlen(needle)); - kwsprep(kws); + } else if (opts & DIFF_PICKAXE_KIND_S) { + if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE && + has_non_ascii(needle)) { + struct strbuf sb = STRBUF_INIT; + int cflags = REG_NEWLINE | REG_ICASE; + + basic_regex_quote_buf(&sb, needle); + regcomp_or_die(®ex, sb.buf, cflags); + strbuf_release(&sb); + regexp = ®ex; + } else { + kws = kwsalloc(o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE + ? tolower_trans_tbl : NULL); + kwsincr(kws, needle, strlen(needle)); + kwsprep(kws); + } } /* Might want to warn when both S and G are on; I don't care... */ @@ -248,7 +257,7 @@ void diffcore_pickaxe(struct diff_options *o) if (regexp) regfree(regexp); - else + if (kws) kwsfree(kws); return; } diff --git a/revision.c b/revision.c index 5d11ecaf27..30f65b3bbd 100644 --- a/revision.c +++ b/revision.c @@ -2412,6 +2412,9 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s revs->diffopt.flags.follow_renames) revs->diff = 1; + if (revs->diffopt.objfind) + revs->simplify_history = 0; + if (revs->topo_order) revs->limited = 1; -- 2.15.1.620.gb9897f4670-goog ``` ## Stefan Beller, 2018-01-03 00:46 Subject: [PATCH 5/5] diff: properly error out when combining multiple pickaxe options Message-ID: <20180103004624.222528-6-sbeller@google.com> URL: https://gitlist.dev/e/20180103004624.222528-6-sbeller%40google.com In-Reply-To: <20180103004624.222528-1-sbeller@google.com> ``` In f506b8e8b5 (git log/diff: add -G that greps in the patch text, 2010-08-23) we were hesitant to check if the user requests both -S and -G at the same time. Now that the pickaxe family also offers --find-object, which looks slightly more different than the former two, let's add a check that those are not used at the same time. Signed-off-by: Stefan Beller --- diff.c | 10 ++++++++++ diffcore-pickaxe.c | 1 - 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/diff.c b/diff.c index a872bdcac1..29973e78d6 100644 --- a/diff.c +++ b/diff.c @@ -4122,6 +4122,16 @@ void diff_setup_done(struct diff_options *options) count++; if (count > 1) die(_("--name-only, --name-status, --check and -s are mutually exclusive")); + count = 0; + + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_S) + count++; + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_G) + count++; + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_OBJFIND) + count++; + if (count > 1) + die(_("-G, -S, --find-object are mutually exclusive")); /* * Most of the time we can say "there are changes" diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c index 72bb5a9514..239ce5122b 100644 --- a/diffcore-pickaxe.c +++ b/diffcore-pickaxe.c @@ -251,7 +251,6 @@ void diffcore_pickaxe(struct diff_options *o) } } - /* Might want to warn when both S and G are on; I don't care... */ pickaxe(&diff_queued_diff, o, regexp, kws, (opts & DIFF_PICKAXE_KIND_G) ? diff_grep : has_changes); -- 2.15.1.620.gb9897f4670-goog ``` ## Stefan Beller, 2018-01-03 00:46 Subject: [PATCH 3/5] diff: introduce DIFF_PICKAXE_KINDS_MASK Message-ID: <20180103004624.222528-4-sbeller@google.com> URL: https://gitlist.dev/e/20180103004624.222528-4-sbeller%40google.com In-Reply-To: <20180103004624.222528-1-sbeller@google.com> ``` Currently the check whether to perform pickaxing is done via checking `diffopt->pickaxe`, which contains the command line argument that we want to pickaxe for. Soon we'll introduce a new type of pickaxing, that will not store anything in the `.pickaxe` field, so let's migrate the check to be dependent on pickaxe_opts. It is not enough to just replace the check for pickaxe by pickaxe_opts, because flags might be set, but pickaxing was not requested ('-i'). To cope with that, introduce a mask to check only for the bits indicating the modes of operation. Signed-off-by: Stefan Beller --- builtin/log.c | 4 ++-- combine-diff.c | 2 +- diff.c | 4 ++-- diff.h | 2 ++ revision.c | 2 +- 5 files changed, 8 insertions(+), 6 deletions(-) diff --git a/builtin/log.c b/builtin/log.c index 6c1fa896ad..bd6f2d1efb 100644 --- a/builtin/log.c +++ b/builtin/log.c @@ -180,8 +180,8 @@ static void cmd_log_init_finish(int argc, const char **argv, const char *prefix, if (rev->show_notes) init_display_notes(&rev->notes_opt); - if (rev->diffopt.pickaxe || rev->diffopt.filter || - rev->diffopt.flags.follow_renames) + if ((rev->diffopt.pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || + rev->diffopt.filter || rev->diffopt.flags.follow_renames) rev->always_show_header = 0; if (source) diff --git a/combine-diff.c b/combine-diff.c index 2505de119a..bc08c4c5b1 100644 --- a/combine-diff.c +++ b/combine-diff.c @@ -1438,7 +1438,7 @@ void diff_tree_combined(const struct object_id *oid, opt->flags.follow_renames || opt->break_opt != -1 || opt->detect_rename || - opt->pickaxe || + (opt->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || opt->filter; diff --git a/diff.c b/diff.c index 0763e89263..5508745dc8 100644 --- a/diff.c +++ b/diff.c @@ -4173,7 +4173,7 @@ void diff_setup_done(struct diff_options *options) /* * Also pickaxe would not work very well if you do not say recursive */ - if (options->pickaxe) + if (options->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) options->flags.recursive = 1; /* * When patches are generated, submodules diffed against the work tree @@ -5777,7 +5777,7 @@ void diffcore_std(struct diff_options *options) if (options->break_opt != -1) diffcore_merge_broken(); } - if (options->pickaxe) + if (options->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) diffcore_pickaxe(options); if (options->orderfile) diffcore_order(options->orderfile); diff --git a/diff.h b/diff.h index 8af1213684..9ec4f824fe 100644 --- a/diff.h +++ b/diff.h @@ -326,6 +326,8 @@ extern void diff_setup_done(struct diff_options *); #define DIFF_PICKAXE_KIND_S 4 /* traditional plumbing counter */ #define DIFF_PICKAXE_KIND_G 8 /* grep in the patch */ +#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | DIFF_PICKAXE_KIND_G) + #define DIFF_PICKAXE_IGNORE_CASE 32 extern void diffcore_std(struct diff_options *); diff --git a/revision.c b/revision.c index ccf1d212ce..5d11ecaf27 100644 --- a/revision.c +++ b/revision.c @@ -2407,7 +2407,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s revs->diff = 1; /* Pickaxe, diff-filter and rename following need diffs */ - if (revs->diffopt.pickaxe || + if ((revs->diffopt.pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || revs->diffopt.filter || revs->diffopt.flags.follow_renames) revs->diff = 1; -- 2.15.1.620.gb9897f4670-goog ``` ## Stefan Beller, 2018-01-03 00:56 Subject: [PATCHv2 4/5] diffcore: add a pickaxe option to find a specific blob Message-ID: <20180103005620.202940-1-sbeller@google.com> URL: https://gitlist.dev/e/20180103005620.202940-1-sbeller%40google.com In-Reply-To: <20180103004624.222528-5-sbeller@google.com> ``` Sometimes users are given a hash of an object and they want to identify it further (ex.: Use verify-pack to find the largest blobs, but what are these? or [1]) One might be tempted to extend git-describe to also work with blobs, such that `git describe ` gives a description as ':'. This was implemented at [2]; as seen by the sheer number of responses (>110), it turns out this is tricky to get right. The hard part to get right is picking the correct 'commit-ish' as that could be the commit that (re-)introduced the blob or the blob that removed the blob; the blob could exist in different branches. Junio hinted at a different approach of solving this problem, which this patch implements. Teach the diff machinery another flag for restricting the information to what is shown. For example: $ ./git log --oneline --find-object=v2.0.0:Makefile b2feb64309 Revert the whole "ask curl-config" topic for now 47fbfded53 i18n: only extract comments marked with "TRANSLATORS:" we observe that the Makefile as shipped with 2.0 was appeared in v1.9.2-471-g47fbfded53 and in v2.0.0-rc1-5-gb2feb6430b. The reason why these commits both occur prior to v2.0.0 are evil merges that are not found using this new mechanism. [1] https://stackoverflow.com/questions/223678/which-commit-has-this-blob [2] https://public-inbox.org/git/20171028004419.10139-1-sbeller@google.com/ Signed-off-by: Stefan Beller --- We lost the tests in the first version, this includes the new file containing the tests. Stefan Documentation/diff-options.txt | 10 +++++++ diff.c | 21 ++++++++++++- diff.h | 8 ++++- diffcore-pickaxe.c | 45 +++++++++++++++++----------- revision.c | 3 ++ t/t4064-diff-oidfind.sh | 68 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 135 insertions(+), 20 deletions(-) create mode 100755 t/t4064-diff-oidfind.sh diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt index dd0dba5b1d..f9cf85db05 100644 --- a/Documentation/diff-options.txt +++ b/Documentation/diff-options.txt @@ -492,6 +492,15 @@ occurrences of that string did not change). See the 'pickaxe' entry in linkgit:gitdiffcore[7] for more information. +--find-object=:: + Look for differences that change the number of occurrences of + the specified object. Similar to `-S`, just the argument is different + in that it doesn't search for a specific string but for a specific + object id. ++ +The object can be a blob or a submodule commit. It implies the `-t` option in +`git-log` to also find trees. + --pickaxe-all:: When `-S` or `-G` finds a change, show all the changes in that changeset, not just the files that contain the change @@ -500,6 +509,7 @@ information. --pickaxe-regex:: Treat the given to `-S` as an extended POSIX regular expression to match. + endif::git-format-patch[] -O:: diff --git a/diff.c b/diff.c index 5508745dc8..a872bdcac1 100644 --- a/diff.c +++ b/diff.c @@ -4082,6 +4082,7 @@ void diff_setup(struct diff_options *options) options->interhunkcontext = diff_interhunk_context_default; options->ws_error_highlight = ws_error_highlight_default; options->flags.rename_empty = 1; + options->objfind = NULL; /* pathchange left =NULL by default */ options->change = diff_change; @@ -4487,6 +4488,23 @@ static int parse_ws_error_highlight_opt(struct diff_options *opt, const char *ar return 1; } +static int parse_objfind_opt(struct diff_options *opt, const char *arg) +{ + struct object_id oid; + + if (get_oid(arg, &oid)) + return error("unable to resolve '%s'", arg); + + if (!opt->objfind) + opt->objfind = xcalloc(1, sizeof(*opt->objfind)); + + opt->pickaxe_opts |= DIFF_PICKAXE_KIND_OBJFIND; + opt->flags.recursive = 1; + opt->flags.tree_in_recursive = 1; + oidset_insert(opt->objfind, &oid); + return 1; +} + int diff_opt_parse(struct diff_options *options, const char **av, int ac, const char *prefix) { @@ -4736,7 +4754,8 @@ int diff_opt_parse(struct diff_options *options, else if ((argcount = short_opt('O', av, &optarg))) { options->orderfile = prefix_filename(prefix, optarg); return argcount; - } + } else if (skip_prefix(arg, "--find-object=", &arg)) + return parse_objfind_opt(options, arg); else if ((argcount = parse_long_opt("diff-filter", av, &optarg))) { int offending = parse_diff_filter_opt(optarg, options); if (offending) diff --git a/diff.h b/diff.h index 9ec4f824fe..8a56cac2ad 100644 --- a/diff.h +++ b/diff.h @@ -7,6 +7,7 @@ #include "tree-walk.h" #include "pathspec.h" #include "object.h" +#include "oidset.h" struct rev_info; struct diff_options; @@ -173,6 +174,8 @@ struct diff_options { enum diff_words_type word_diff; enum diff_submodule_format submodule_format; + struct oidset *objfind; + /* this is set by diffcore for DIFF_FORMAT_PATCH */ int found_changes; @@ -325,8 +328,11 @@ extern void diff_setup_done(struct diff_options *); #define DIFF_PICKAXE_KIND_S 4 /* traditional plumbing counter */ #define DIFF_PICKAXE_KIND_G 8 /* grep in the patch */ +#define DIFF_PICKAXE_KIND_OBJFIND 16 /* specific object IDs */ -#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | DIFF_PICKAXE_KIND_G) +#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | \ + DIFF_PICKAXE_KIND_G | \ + DIFF_PICKAXE_KIND_OBJFIND) #define DIFF_PICKAXE_IGNORE_CASE 32 diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c index 4b5d88ea46..72bb5a9514 100644 --- a/diffcore-pickaxe.c +++ b/diffcore-pickaxe.c @@ -124,13 +124,20 @@ static int pickaxe_match(struct diff_filepair *p, struct diff_options *o, mmfile_t mf1, mf2; int ret; - if (!o->pickaxe[0]) - return 0; - /* ignore unmerged */ if (!DIFF_FILE_VALID(p->one) && !DIFF_FILE_VALID(p->two)) return 0; + if (o->objfind) { + return (DIFF_FILE_VALID(p->one) && + oidset_contains(o->objfind, &p->one->oid)) || + (DIFF_FILE_VALID(p->two) && + oidset_contains(o->objfind, &p->two->oid)); + } + + if (!o->pickaxe[0]) + return 0; + if (o->flags.allow_textconv) { textconv_one = get_textconv(p->one); textconv_two = get_textconv(p->two); @@ -226,20 +233,22 @@ void diffcore_pickaxe(struct diff_options *o) cflags |= REG_ICASE; regcomp_or_die(®ex, needle, cflags); regexp = ®ex; - } else if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE && - has_non_ascii(needle)) { - struct strbuf sb = STRBUF_INIT; - int cflags = REG_NEWLINE | REG_ICASE; - - basic_regex_quote_buf(&sb, needle); - regcomp_or_die(®ex, sb.buf, cflags); - strbuf_release(&sb); - regexp = ®ex; - } else { - kws = kwsalloc(o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE - ? tolower_trans_tbl : NULL); - kwsincr(kws, needle, strlen(needle)); - kwsprep(kws); + } else if (opts & DIFF_PICKAXE_KIND_S) { + if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE && + has_non_ascii(needle)) { + struct strbuf sb = STRBUF_INIT; + int cflags = REG_NEWLINE | REG_ICASE; + + basic_regex_quote_buf(&sb, needle); + regcomp_or_die(®ex, sb.buf, cflags); + strbuf_release(&sb); + regexp = ®ex; + } else { + kws = kwsalloc(o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE + ? tolower_trans_tbl : NULL); + kwsincr(kws, needle, strlen(needle)); + kwsprep(kws); + } } /* Might want to warn when both S and G are on; I don't care... */ @@ -248,7 +257,7 @@ void diffcore_pickaxe(struct diff_options *o) if (regexp) regfree(regexp); - else + if (kws) kwsfree(kws); return; } diff --git a/revision.c b/revision.c index 5d11ecaf27..30f65b3bbd 100644 --- a/revision.c +++ b/revision.c @@ -2412,6 +2412,9 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s revs->diffopt.flags.follow_renames) revs->diff = 1; + if (revs->diffopt.objfind) + revs->simplify_history = 0; + if (revs->topo_order) revs->limited = 1; diff --git a/t/t4064-diff-oidfind.sh b/t/t4064-diff-oidfind.sh new file mode 100755 index 0000000000..3bdf317af8 --- /dev/null +++ b/t/t4064-diff-oidfind.sh @@ -0,0 +1,68 @@ +#!/bin/sh + +test_description='test finding specific blobs in the revision walking' +. ./test-lib.sh + +test_expect_success 'setup ' ' + git commit --allow-empty -m "empty initial commit" && + + echo "Hello, world!" >greeting && + git add greeting && + git commit -m "add the greeting blob" && # borrowed from Git from the Bottom Up + git tag -m "the blob" greeting $(git rev-parse HEAD:greeting) && + + echo asdf >unrelated && + git add unrelated && + git commit -m "unrelated history" && + + git revert HEAD^ && + + git commit --allow-empty -m "another unrelated commit" +' + +test_expect_success 'find the greeting blob' ' + cat >expect <<-EOF && + Revert "add the greeting blob" + add the greeting blob + EOF + + git log --format=%s --find-object=greeting^{blob} >actual && + + test_cmp expect actual +' + +test_expect_success 'setup a tree' ' + mkdir a && + echo asdf >a/file && + git add a/file && + git commit -m "add a file in a subdirectory" +' + +test_expect_success 'find a tree' ' + cat >expect <<-EOF && + add a file in a subdirectory + EOF + + git log --format=%s -t --find-object=HEAD:a >actual && + + test_cmp expect actual +' + +test_expect_success 'setup a submodule' ' + test_create_repo sub && + test_commit -C sub sub && + git submodule add ./sub sub && + git commit -a -m "add sub" +' + +test_expect_success 'find a submodule' ' + cat >expect <<-EOF && + add sub + EOF + + git log --format=%s --find-object=HEAD:sub >actual && + + test_cmp expect actual +' + +test_done -- 2.15.1.620.gb9897f4670-goog ``` ## Jacob Keller, 2018-01-03 00:58 Subject: Re: [PATCH 5/5] diff: properly error out when combining multiple pickaxe options Message-ID: URL: https://gitlist.dev/e/CA%2BP7%2Bxr7W2sTtUBAhQDZXOTVypuiCU0fL%2B-W4tbVPd%2B%2BQbUJcA%40mail.gmail.com In-Reply-To: <20180103004624.222528-6-sbeller@google.com> ``` On Tue, Jan 2, 2018 at 4:46 PM, Stefan Beller wrote: > die(_("--name-only, --name-status, --check and -s are mutually exclusive")); > + count = 0; > + > + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_S) > + count++; > + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_G) > + count++; > + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_OBJFIND) > + count++; > + if (count > 1) > + die(_("-G, -S, --find-object are mutually exclusive")); Nit: I'd put an "and" in the sentence. ``` ## Junio C Hamano, 2018-01-03 21:49 Subject: Re: [PATCH 1/5] diff.h: Make pickaxe_opts an unsigned bit field Message-ID: URL: https://gitlist.dev/e/xmqqfu7mhata.fsf%40gitster.mtv.corp.google.com In-Reply-To: <20180103004624.222528-2-sbeller@google.com> ``` Stefan Beller writes: > This variable is used as a bit field[1], and as we are about to add more > fields, indicate its usage as a bit field by making it unsigned. > > [1] containing the bits > > #define DIFF_PICKAXE_ALL 1 > #define DIFF_PICKAXE_REGEX 2 > #define DIFF_PICKAXE_KIND_S 4 > #define DIFF_PICKAXE_KIND_G 8 > > Signed-off-by: Stefan Beller > --- > diff.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Makes perfect sense. > > diff --git a/diff.h b/diff.h > index 0fb18dd735..ea310f76fd 100644 > --- a/diff.h > +++ b/diff.h > @@ -146,7 +146,7 @@ struct diff_options { > int skip_stat_unmatch; > int line_termination; > int output_format; > - int pickaxe_opts; > + unsigned pickaxe_opts; > int rename_score; > int rename_limit; > int needed_rename_limit; ``` ## Junio C Hamano, 2018-01-03 22:01 Subject: Re: [PATCH 3/5] diff: introduce DIFF_PICKAXE_KINDS_MASK Message-ID: URL: https://gitlist.dev/e/xmqqbmiaha9b.fsf%40gitster.mtv.corp.google.com In-Reply-To: <20180103004624.222528-4-sbeller@google.com> ``` Stefan Beller writes: > Currently the check whether to perform pickaxing is done via checking > `diffopt->pickaxe`, which contains the command line argument that we > want to pickaxe for. Soon we'll introduce a new type of pickaxing, that > will not store anything in the `.pickaxe` field, so let's migrate the > check to be dependent on pickaxe_opts. > > It is not enough to just replace the check for pickaxe by pickaxe_opts, > because flags might be set, but pickaxing was not requested ('-i'). > To cope with that, introduce a mask to check only for the bits indicating > the modes of operation. The resulting code after this series would leave a few "huh?" if it were new code, but the series is not making anything worse, so take this as just something noticed, not as something needs further work. Because we do not allow "log -S -G", there is no legitimate reason why they have to be a bit in the pickaxe_opts flag word. A single enum that says "We are doing pickaxe search and _this_ is the kind of pickaxe search we are doing" would suffice, i.e. the NULL-ness check of rev->diffopt.pickaxe string can be replaced with a check of that enum field against PICKAXE_NONE or something that signals us that no pickaxe is in effect. On the other hand, if somebody comes up with a sensible way to combine more than one pickaxe queries in a single traversal (e.g. "log -S -S" might mean "find a change that loses or gains and in the same commit", or it may mean the same with "... or "), then a more sensible data structure to represent the pickaxe request may have been a list of struct, each of which records the kind and the parameter (e.g. "-S" and "" would be in a single struct, and "-S" and "" would be in another, and these two are in the list that is diffopt->pickaxe). The NULL-ness check of rev->diffopt.pickaxe string would be replaced with a check of the length of that list. In any case, this step looks sensible. > > Signed-off-by: Stefan Beller > --- > builtin/log.c | 4 ++-- > combine-diff.c | 2 +- > diff.c | 4 ++-- > diff.h | 2 ++ > revision.c | 2 +- > 5 files changed, 8 insertions(+), 6 deletions(-) > > diff --git a/builtin/log.c b/builtin/log.c > index 6c1fa896ad..bd6f2d1efb 100644 > --- a/builtin/log.c > +++ b/builtin/log.c > @@ -180,8 +180,8 @@ static void cmd_log_init_finish(int argc, const char **argv, const char *prefix, > if (rev->show_notes) > init_display_notes(&rev->notes_opt); > > - if (rev->diffopt.pickaxe || rev->diffopt.filter || > - rev->diffopt.flags.follow_renames) > + if ((rev->diffopt.pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || > + rev->diffopt.filter || rev->diffopt.flags.follow_renames) > rev->always_show_header = 0; > > if (source) > diff --git a/combine-diff.c b/combine-diff.c > index 2505de119a..bc08c4c5b1 100644 > --- a/combine-diff.c > +++ b/combine-diff.c > @@ -1438,7 +1438,7 @@ void diff_tree_combined(const struct object_id *oid, > opt->flags.follow_renames || > opt->break_opt != -1 || > opt->detect_rename || > - opt->pickaxe || > + (opt->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || > opt->filter; > > > diff --git a/diff.c b/diff.c > index 0763e89263..5508745dc8 100644 > --- a/diff.c > +++ b/diff.c > @@ -4173,7 +4173,7 @@ void diff_setup_done(struct diff_options *options) > /* > * Also pickaxe would not work very well if you do not say recursive > */ > - if (options->pickaxe) > + if (options->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) > options->flags.recursive = 1; > /* > * When patches are generated, submodules diffed against the work tree > @@ -5777,7 +5777,7 @@ void diffcore_std(struct diff_options *options) > if (options->break_opt != -1) > diffcore_merge_broken(); > } > - if (options->pickaxe) > + if (options->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) > diffcore_pickaxe(options); > if (options->orderfile) > diffcore_order(options->orderfile); > diff --git a/diff.h b/diff.h > index 8af1213684..9ec4f824fe 100644 > --- a/diff.h > +++ b/diff.h > @@ -326,6 +326,8 @@ extern void diff_setup_done(struct diff_options *); > #define DIFF_PICKAXE_KIND_S 4 /* traditional plumbing counter */ > #define DIFF_PICKAXE_KIND_G 8 /* grep in the patch */ > > +#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | DIFF_PICKAXE_KIND_G) > + > #define DIFF_PICKAXE_IGNORE_CASE 32 > > extern void diffcore_std(struct diff_options *); > diff --git a/revision.c b/revision.c > index ccf1d212ce..5d11ecaf27 100644 > --- a/revision.c > +++ b/revision.c > @@ -2407,7 +2407,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s > revs->diff = 1; > > /* Pickaxe, diff-filter and rename following need diffs */ > - if (revs->diffopt.pickaxe || > + if ((revs->diffopt.pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || > revs->diffopt.filter || > revs->diffopt.flags.follow_renames) > revs->diff = 1; ``` ## Junio C Hamano, 2018-01-03 22:08 Subject: Re: [PATCH 5/5] diff: properly error out when combining multiple pickaxe options Message-ID: URL: https://gitlist.dev/e/xmqq7esyh9ym.fsf%40gitster.mtv.corp.google.com In-Reply-To: <20180103004624.222528-6-sbeller@google.com> ``` Stefan Beller writes: ; > + count = 0; > + > + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_S) > + count++; > + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_G) > + count++; > + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_OBJFIND) > + count++; > + if (count > 1) > + die(_("-G, -S, --find-object are mutually exclusive")); I thought the reason you defined pickaxe-kind bitmask was so that you can mask this field to grab these (and only these) bits. Once you have that mask, you should be able to use HAS_MULTI_BITS() on the masked result without counting, no? ``` ## Stefan Beller, 2018-01-03 22:18 Subject: Re: [PATCH 5/5] diff: properly error out when combining multiple pickaxe options Message-ID: URL: https://gitlist.dev/e/CAGZ79kaef93BVM%2BXaihad7cij-tc6DvWAgwo%2BPs1zt_3Jn84Tg%40mail.gmail.com In-Reply-To: ``` On Wed, Jan 3, 2018 at 2:08 PM, Junio C Hamano wrote: > Stefan Beller writes: > > ; >> + count = 0; >> + >> + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_S) >> + count++; >> + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_G) >> + count++; >> + if (options->pickaxe_opts & DIFF_PICKAXE_KIND_OBJFIND) >> + count++; >> + if (count > 1) >> + die(_("-G, -S, --find-object are mutually exclusive")); > > I thought the reason you defined pickaxe-kind bitmask was so that > you can mask this field to grab these (and only these) bits. Originally I only wanted to mask out the 'case independency' bit and keep it future proof for any similar bits. > Once you have that mask, you should be able to use HAS_MULTI_BITS() > on the masked result without counting, no? Oh, what a nice macro! Thanks for pointing at it. As soon as I figured out the right place where to put this check, I saw the lines above, whose style I imitated. (I guess there is just no mask defined for "--name-only, --name-status, --check and -s", nor would it make sense to do so; though that given macro should work just fine even for non-continuous masks) ``` ## Stefan Beller, 2018-01-04 22:50 Subject: [PATCHv2 0/6] pickaxe refactorings and a new mode to find blobs (WAS: diffcore: add a pickaxe option to find a specific blob) Message-ID: <20180104225044.191220-1-sbeller@google.com> URL: https://gitlist.dev/e/20180104225044.191220-1-sbeller%40google.com In-Reply-To: <20180103004624.222528-1-sbeller@google.com> ``` v2: Thanks Junio and Jacob for review! * fixed up the last patch to rephrase the error message to contain an 'and' * use HAS_MULTI_BITS as well * a bonus patch that uses HAS_MULTI_BITS for the existing code, too. v1 After some discussion [1], we're convinced that the original approach for adding in just another pickaxe mode to find blobs was too hacky. So I went the less hacky way and did some refactoring first (patches 1-3), Then we'll have the new pickaxe mode to find blobs in patch 4. It grew slightly larger as it had issues with the setup (we neither have a regex nor a KWS to init) in this new world, so there are a few more lines in there. The last patch is just the cherry on the cake, helping to keep users sane by warning when they try to use different pickaxe modes at the same time. Thanks, Stefan [1] https://public-inbox.org/git/CAGZ79kaB0G9zetF6QtC45+ZGLM3gOsYWV7e+gkCe2yKOhb0Ssg@mail.gmail.com/ Stefan Beller (6): diff.h: Make pickaxe_opts an unsigned bit field diff: migrate diff_flags.pickaxe_ignore_case to a pickaxe_opts bit diff: introduce DIFF_PICKAXE_KINDS_MASK diffcore: add a pickaxe option to find a specific blob diff: properly error out when combining multiple pickaxe options diff: use HAS_MULTI_BITS instead of counting bits manually Documentation/diff-options.txt | 10 +++++++ builtin/log.c | 4 +-- combine-diff.c | 2 +- diff.c | 43 ++++++++++++++++++-------- diff.h | 13 ++++++-- diffcore-pickaxe.c | 48 ++++++++++++++++------------- revision.c | 7 +++-- t/t4064-diff-oidfind.sh | 68 ++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 155 insertions(+), 40 deletions(-) create mode 100755 t/t4064-diff-oidfind.sh -- 2.16.0.rc0.223.g4a4ac83678-goog ``` ## Stefan Beller, 2018-01-04 22:50 Subject: [PATCHv2 1/6] diff.h: Make pickaxe_opts an unsigned bit field Message-ID: <20180104225044.191220-2-sbeller@google.com> URL: https://gitlist.dev/e/20180104225044.191220-2-sbeller%40google.com In-Reply-To: <20180104225044.191220-1-sbeller@google.com> ``` This variable is used as a bit field[1], and as we are about to add more fields, indicate its usage as a bit field by making it unsigned. [1] containing the bits #define DIFF_PICKAXE_ALL 1 #define DIFF_PICKAXE_REGEX 2 #define DIFF_PICKAXE_KIND_S 4 #define DIFF_PICKAXE_KIND_G 8 Signed-off-by: Stefan Beller --- diff.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/diff.h b/diff.h index 0fb18dd735..ea310f76fd 100644 --- a/diff.h +++ b/diff.h @@ -146,7 +146,7 @@ struct diff_options { int skip_stat_unmatch; int line_termination; int output_format; - int pickaxe_opts; + unsigned pickaxe_opts; int rename_score; int rename_limit; int needed_rename_limit; -- 2.16.0.rc0.223.g4a4ac83678-goog ``` ## Stefan Beller, 2018-01-04 22:50 Subject: [PATCHv2 3/6] diff: introduce DIFF_PICKAXE_KINDS_MASK Message-ID: <20180104225044.191220-4-sbeller@google.com> URL: https://gitlist.dev/e/20180104225044.191220-4-sbeller%40google.com In-Reply-To: <20180104225044.191220-1-sbeller@google.com> ``` Currently the check whether to perform pickaxing is done via checking `diffopt->pickaxe`, which contains the command line argument that we want to pickaxe for. Soon we'll introduce a new type of pickaxing, that will not store anything in the `.pickaxe` field, so let's migrate the check to be dependent on pickaxe_opts. It is not enough to just replace the check for pickaxe by pickaxe_opts, because flags might be set, but pickaxing was not requested ('-i'). To cope with that, introduce a mask to check only for the bits indicating the modes of operation. Signed-off-by: Stefan Beller --- builtin/log.c | 4 ++-- combine-diff.c | 2 +- diff.c | 4 ++-- diff.h | 2 ++ revision.c | 2 +- 5 files changed, 8 insertions(+), 6 deletions(-) diff --git a/builtin/log.c b/builtin/log.c index 6c1fa896ad..bd6f2d1efb 100644 --- a/builtin/log.c +++ b/builtin/log.c @@ -180,8 +180,8 @@ static void cmd_log_init_finish(int argc, const char **argv, const char *prefix, if (rev->show_notes) init_display_notes(&rev->notes_opt); - if (rev->diffopt.pickaxe || rev->diffopt.filter || - rev->diffopt.flags.follow_renames) + if ((rev->diffopt.pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || + rev->diffopt.filter || rev->diffopt.flags.follow_renames) rev->always_show_header = 0; if (source) diff --git a/combine-diff.c b/combine-diff.c index 2505de119a..bc08c4c5b1 100644 --- a/combine-diff.c +++ b/combine-diff.c @@ -1438,7 +1438,7 @@ void diff_tree_combined(const struct object_id *oid, opt->flags.follow_renames || opt->break_opt != -1 || opt->detect_rename || - opt->pickaxe || + (opt->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || opt->filter; diff --git a/diff.c b/diff.c index 0763e89263..5508745dc8 100644 --- a/diff.c +++ b/diff.c @@ -4173,7 +4173,7 @@ void diff_setup_done(struct diff_options *options) /* * Also pickaxe would not work very well if you do not say recursive */ - if (options->pickaxe) + if (options->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) options->flags.recursive = 1; /* * When patches are generated, submodules diffed against the work tree @@ -5777,7 +5777,7 @@ void diffcore_std(struct diff_options *options) if (options->break_opt != -1) diffcore_merge_broken(); } - if (options->pickaxe) + if (options->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) diffcore_pickaxe(options); if (options->orderfile) diffcore_order(options->orderfile); diff --git a/diff.h b/diff.h index 8af1213684..9ec4f824fe 100644 --- a/diff.h +++ b/diff.h @@ -326,6 +326,8 @@ extern void diff_setup_done(struct diff_options *); #define DIFF_PICKAXE_KIND_S 4 /* traditional plumbing counter */ #define DIFF_PICKAXE_KIND_G 8 /* grep in the patch */ +#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | DIFF_PICKAXE_KIND_G) + #define DIFF_PICKAXE_IGNORE_CASE 32 extern void diffcore_std(struct diff_options *); diff --git a/revision.c b/revision.c index ccf1d212ce..5d11ecaf27 100644 --- a/revision.c +++ b/revision.c @@ -2407,7 +2407,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s revs->diff = 1; /* Pickaxe, diff-filter and rename following need diffs */ - if (revs->diffopt.pickaxe || + if ((revs->diffopt.pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || revs->diffopt.filter || revs->diffopt.flags.follow_renames) revs->diff = 1; -- 2.16.0.rc0.223.g4a4ac83678-goog ``` ## Stefan Beller, 2018-01-04 22:50 Subject: [PATCHv2 4/6] diffcore: add a pickaxe option to find a specific blob Message-ID: <20180104225044.191220-5-sbeller@google.com> URL: https://gitlist.dev/e/20180104225044.191220-5-sbeller%40google.com In-Reply-To: <20180104225044.191220-1-sbeller@google.com> ``` Sometimes users are given a hash of an object and they want to identify it further (ex.: Use verify-pack to find the largest blobs, but what are these? or [1]) One might be tempted to extend git-describe to also work with blobs, such that `git describe ` gives a description as ':'. This was implemented at [2]; as seen by the sheer number of responses (>110), it turns out this is tricky to get right. The hard part to get right is picking the correct 'commit-ish' as that could be the commit that (re-)introduced the blob or the blob that removed the blob; the blob could exist in different branches. Junio hinted at a different approach of solving this problem, which this patch implements. Teach the diff machinery another flag for restricting the information to what is shown. For example: $ ./git log --oneline --find-object=v2.0.0:Makefile b2feb64309 Revert the whole "ask curl-config" topic for now 47fbfded53 i18n: only extract comments marked with "TRANSLATORS:" we observe that the Makefile as shipped with 2.0 was appeared in v1.9.2-471-g47fbfded53 and in v2.0.0-rc1-5-gb2feb6430b. The reason why these commits both occur prior to v2.0.0 are evil merges that are not found using this new mechanism. [1] https://stackoverflow.com/questions/223678/which-commit-has-this-blob [2] https://public-inbox.org/git/20171028004419.10139-1-sbeller@google.com/ Signed-off-by: Stefan Beller --- Documentation/diff-options.txt | 10 +++++++ diff.c | 21 ++++++++++++- diff.h | 8 ++++- diffcore-pickaxe.c | 45 +++++++++++++++++----------- revision.c | 3 ++ t/t4064-diff-oidfind.sh | 68 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 135 insertions(+), 20 deletions(-) create mode 100755 t/t4064-diff-oidfind.sh diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt index dd0dba5b1d..f9cf85db05 100644 --- a/Documentation/diff-options.txt +++ b/Documentation/diff-options.txt @@ -492,6 +492,15 @@ occurrences of that string did not change). See the 'pickaxe' entry in linkgit:gitdiffcore[7] for more information. +--find-object=:: + Look for differences that change the number of occurrences of + the specified object. Similar to `-S`, just the argument is different + in that it doesn't search for a specific string but for a specific + object id. ++ +The object can be a blob or a submodule commit. It implies the `-t` option in +`git-log` to also find trees. + --pickaxe-all:: When `-S` or `-G` finds a change, show all the changes in that changeset, not just the files that contain the change @@ -500,6 +509,7 @@ information. --pickaxe-regex:: Treat the given to `-S` as an extended POSIX regular expression to match. + endif::git-format-patch[] -O:: diff --git a/diff.c b/diff.c index 5508745dc8..a872bdcac1 100644 --- a/diff.c +++ b/diff.c @@ -4082,6 +4082,7 @@ void diff_setup(struct diff_options *options) options->interhunkcontext = diff_interhunk_context_default; options->ws_error_highlight = ws_error_highlight_default; options->flags.rename_empty = 1; + options->objfind = NULL; /* pathchange left =NULL by default */ options->change = diff_change; @@ -4487,6 +4488,23 @@ static int parse_ws_error_highlight_opt(struct diff_options *opt, const char *ar return 1; } +static int parse_objfind_opt(struct diff_options *opt, const char *arg) +{ + struct object_id oid; + + if (get_oid(arg, &oid)) + return error("unable to resolve '%s'", arg); + + if (!opt->objfind) + opt->objfind = xcalloc(1, sizeof(*opt->objfind)); + + opt->pickaxe_opts |= DIFF_PICKAXE_KIND_OBJFIND; + opt->flags.recursive = 1; + opt->flags.tree_in_recursive = 1; + oidset_insert(opt->objfind, &oid); + return 1; +} + int diff_opt_parse(struct diff_options *options, const char **av, int ac, const char *prefix) { @@ -4736,7 +4754,8 @@ int diff_opt_parse(struct diff_options *options, else if ((argcount = short_opt('O', av, &optarg))) { options->orderfile = prefix_filename(prefix, optarg); return argcount; - } + } else if (skip_prefix(arg, "--find-object=", &arg)) + return parse_objfind_opt(options, arg); else if ((argcount = parse_long_opt("diff-filter", av, &optarg))) { int offending = parse_diff_filter_opt(optarg, options); if (offending) diff --git a/diff.h b/diff.h index 9ec4f824fe..8a56cac2ad 100644 --- a/diff.h +++ b/diff.h @@ -7,6 +7,7 @@ #include "tree-walk.h" #include "pathspec.h" #include "object.h" +#include "oidset.h" struct rev_info; struct diff_options; @@ -173,6 +174,8 @@ struct diff_options { enum diff_words_type word_diff; enum diff_submodule_format submodule_format; + struct oidset *objfind; + /* this is set by diffcore for DIFF_FORMAT_PATCH */ int found_changes; @@ -325,8 +328,11 @@ extern void diff_setup_done(struct diff_options *); #define DIFF_PICKAXE_KIND_S 4 /* traditional plumbing counter */ #define DIFF_PICKAXE_KIND_G 8 /* grep in the patch */ +#define DIFF_PICKAXE_KIND_OBJFIND 16 /* specific object IDs */ -#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | DIFF_PICKAXE_KIND_G) +#define DIFF_PICKAXE_KINDS_MASK (DIFF_PICKAXE_KIND_S | \ + DIFF_PICKAXE_KIND_G | \ + DIFF_PICKAXE_KIND_OBJFIND) #define DIFF_PICKAXE_IGNORE_CASE 32 diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c index 4b5d88ea46..72bb5a9514 100644 --- a/diffcore-pickaxe.c +++ b/diffcore-pickaxe.c @@ -124,13 +124,20 @@ static int pickaxe_match(struct diff_filepair *p, struct diff_options *o, mmfile_t mf1, mf2; int ret; - if (!o->pickaxe[0]) - return 0; - /* ignore unmerged */ if (!DIFF_FILE_VALID(p->one) && !DIFF_FILE_VALID(p->two)) return 0; + if (o->objfind) { + return (DIFF_FILE_VALID(p->one) && + oidset_contains(o->objfind, &p->one->oid)) || + (DIFF_FILE_VALID(p->two) && + oidset_contains(o->objfind, &p->two->oid)); + } + + if (!o->pickaxe[0]) + return 0; + if (o->flags.allow_textconv) { textconv_one = get_textconv(p->one); textconv_two = get_textconv(p->two); @@ -226,20 +233,22 @@ void diffcore_pickaxe(struct diff_options *o) cflags |= REG_ICASE; regcomp_or_die(®ex, needle, cflags); regexp = ®ex; - } else if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE && - has_non_ascii(needle)) { - struct strbuf sb = STRBUF_INIT; - int cflags = REG_NEWLINE | REG_ICASE; - - basic_regex_quote_buf(&sb, needle); - regcomp_or_die(®ex, sb.buf, cflags); - strbuf_release(&sb); - regexp = ®ex; - } else { - kws = kwsalloc(o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE - ? tolower_trans_tbl : NULL); - kwsincr(kws, needle, strlen(needle)); - kwsprep(kws); + } else if (opts & DIFF_PICKAXE_KIND_S) { + if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE && + has_non_ascii(needle)) { + struct strbuf sb = STRBUF_INIT; + int cflags = REG_NEWLINE | REG_ICASE; + + basic_regex_quote_buf(&sb, needle); + regcomp_or_die(®ex, sb.buf, cflags); + strbuf_release(&sb); + regexp = ®ex; + } else { + kws = kwsalloc(o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE + ? tolower_trans_tbl : NULL); + kwsincr(kws, needle, strlen(needle)); + kwsprep(kws); + } } /* Might want to warn when both S and G are on; I don't care... */ @@ -248,7 +257,7 @@ void diffcore_pickaxe(struct diff_options *o) if (regexp) regfree(regexp); - else + if (kws) kwsfree(kws); return; } diff --git a/revision.c b/revision.c index 5d11ecaf27..30f65b3bbd 100644 --- a/revision.c +++ b/revision.c @@ -2412,6 +2412,9 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s revs->diffopt.flags.follow_renames) revs->diff = 1; + if (revs->diffopt.objfind) + revs->simplify_history = 0; + if (revs->topo_order) revs->limited = 1; diff --git a/t/t4064-diff-oidfind.sh b/t/t4064-diff-oidfind.sh new file mode 100755 index 0000000000..3bdf317af8 --- /dev/null +++ b/t/t4064-diff-oidfind.sh @@ -0,0 +1,68 @@ +#!/bin/sh + +test_description='test finding specific blobs in the revision walking' +. ./test-lib.sh + +test_expect_success 'setup ' ' + git commit --allow-empty -m "empty initial commit" && + + echo "Hello, world!" >greeting && + git add greeting && + git commit -m "add the greeting blob" && # borrowed from Git from the Bottom Up + git tag -m "the blob" greeting $(git rev-parse HEAD:greeting) && + + echo asdf >unrelated && + git add unrelated && + git commit -m "unrelated history" && + + git revert HEAD^ && + + git commit --allow-empty -m "another unrelated commit" +' + +test_expect_success 'find the greeting blob' ' + cat >expect <<-EOF && + Revert "add the greeting blob" + add the greeting blob + EOF + + git log --format=%s --find-object=greeting^{blob} >actual && + + test_cmp expect actual +' + +test_expect_success 'setup a tree' ' + mkdir a && + echo asdf >a/file && + git add a/file && + git commit -m "add a file in a subdirectory" +' + +test_expect_success 'find a tree' ' + cat >expect <<-EOF && + add a file in a subdirectory + EOF + + git log --format=%s -t --find-object=HEAD:a >actual && + + test_cmp expect actual +' + +test_expect_success 'setup a submodule' ' + test_create_repo sub && + test_commit -C sub sub && + git submodule add ./sub sub && + git commit -a -m "add sub" +' + +test_expect_success 'find a submodule' ' + cat >expect <<-EOF && + add sub + EOF + + git log --format=%s --find-object=HEAD:sub >actual && + + test_cmp expect actual +' + +test_done -- 2.16.0.rc0.223.g4a4ac83678-goog ``` ## Stefan Beller, 2018-01-04 22:50 Subject: [PATCHv2 5/6] diff: properly error out when combining multiple pickaxe options Message-ID: <20180104225044.191220-6-sbeller@google.com> URL: https://gitlist.dev/e/20180104225044.191220-6-sbeller%40google.com In-Reply-To: <20180104225044.191220-1-sbeller@google.com> ``` In f506b8e8b5 (git log/diff: add -G that greps in the patch text, 2010-08-23) we were hesitant to check if the user requests both -S and -G at the same time. Now that the pickaxe family also offers --find-object, which looks slightly more different than the former two, let's add a check that those are not used at the same time. Signed-off-by: Stefan Beller --- diff.c | 3 +++ diffcore-pickaxe.c | 1 - 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/diff.c b/diff.c index a872bdcac1..42858d4c7d 100644 --- a/diff.c +++ b/diff.c @@ -4123,6 +4123,9 @@ void diff_setup_done(struct diff_options *options) if (count > 1) die(_("--name-only, --name-status, --check and -s are mutually exclusive")); + if (HAS_MULTI_BITS(options->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK)) + die(_("-G, -S and --find-object are mutually exclusive")); + /* * Most of the time we can say "there are changes" * only by checking if there are changed paths, but diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c index 72bb5a9514..239ce5122b 100644 --- a/diffcore-pickaxe.c +++ b/diffcore-pickaxe.c @@ -251,7 +251,6 @@ void diffcore_pickaxe(struct diff_options *o) } } - /* Might want to warn when both S and G are on; I don't care... */ pickaxe(&diff_queued_diff, o, regexp, kws, (opts & DIFF_PICKAXE_KIND_G) ? diff_grep : has_changes); -- 2.16.0.rc0.223.g4a4ac83678-goog ``` ## Stefan Beller, 2018-01-04 22:50 Subject: [PATCHv2 2/6] diff: migrate diff_flags.pickaxe_ignore_case to a pickaxe_opts bit Message-ID: <20180104225044.191220-3-sbeller@google.com> URL: https://gitlist.dev/e/20180104225044.191220-3-sbeller%40google.com In-Reply-To: <20180104225044.191220-1-sbeller@google.com> ``` Currently flags for pickaxing are found in different places. Unify the flags into the `pickaxe_opts` field, which will contain any pickaxe related flags. Signed-off-by: Stefan Beller --- diff.h | 3 ++- diffcore-pickaxe.c | 6 +++--- revision.c | 2 +- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/diff.h b/diff.h index ea310f76fd..8af1213684 100644 --- a/diff.h +++ b/diff.h @@ -91,7 +91,6 @@ struct diff_flags { unsigned override_submodule_config:1; unsigned dirstat_by_line:1; unsigned funccontext:1; - unsigned pickaxe_ignore_case:1; unsigned default_follow_renames:1; }; @@ -327,6 +326,8 @@ extern void diff_setup_done(struct diff_options *); #define DIFF_PICKAXE_KIND_S 4 /* traditional plumbing counter */ #define DIFF_PICKAXE_KIND_G 8 /* grep in the patch */ +#define DIFF_PICKAXE_IGNORE_CASE 32 + extern void diffcore_std(struct diff_options *); extern void diffcore_fix_diff_index(struct diff_options *); diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c index 9476bd2108..4b5d88ea46 100644 --- a/diffcore-pickaxe.c +++ b/diffcore-pickaxe.c @@ -222,11 +222,11 @@ void diffcore_pickaxe(struct diff_options *o) if (opts & (DIFF_PICKAXE_REGEX | DIFF_PICKAXE_KIND_G)) { int cflags = REG_EXTENDED | REG_NEWLINE; - if (o->flags.pickaxe_ignore_case) + if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE) cflags |= REG_ICASE; regcomp_or_die(®ex, needle, cflags); regexp = ®ex; - } else if (o->flags.pickaxe_ignore_case && + } else if (o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE && has_non_ascii(needle)) { struct strbuf sb = STRBUF_INIT; int cflags = REG_NEWLINE | REG_ICASE; @@ -236,7 +236,7 @@ void diffcore_pickaxe(struct diff_options *o) strbuf_release(&sb); regexp = ®ex; } else { - kws = kwsalloc(o->flags.pickaxe_ignore_case + kws = kwsalloc(o->pickaxe_opts & DIFF_PICKAXE_IGNORE_CASE ? tolower_trans_tbl : NULL); kwsincr(kws, needle, strlen(needle)); kwsprep(kws); diff --git a/revision.c b/revision.c index e2e691dd5a..ccf1d212ce 100644 --- a/revision.c +++ b/revision.c @@ -2076,7 +2076,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg revs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_ERE; } else if (!strcmp(arg, "--regexp-ignore-case") || !strcmp(arg, "-i")) { revs->grep_filter.ignore_case = 1; - revs->diffopt.flags.pickaxe_ignore_case = 1; + revs->diffopt.pickaxe_opts |= DIFF_PICKAXE_IGNORE_CASE; } else if (!strcmp(arg, "--fixed-strings") || !strcmp(arg, "-F")) { revs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_FIXED; } else if (!strcmp(arg, "--perl-regexp") || !strcmp(arg, "-P")) { -- 2.16.0.rc0.223.g4a4ac83678-goog ``` ## Stefan Beller, 2018-01-04 22:50 Subject: [PATCHv2 6/6] diff: use HAS_MULTI_BITS instead of counting bits manually Message-ID: <20180104225044.191220-7-sbeller@google.com> URL: https://gitlist.dev/e/20180104225044.191220-7-sbeller%40google.com In-Reply-To: <20180104225044.191220-1-sbeller@google.com> ``` This aligns the style to the previous patch. Signed-off-by: Stefan Beller --- diff.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/diff.c b/diff.c index 42858d4c7d..c2ee81a1fa 100644 --- a/diff.c +++ b/diff.c @@ -4107,20 +4107,15 @@ void diff_setup(struct diff_options *options) void diff_setup_done(struct diff_options *options) { - int count = 0; + unsigned check_mask = DIFF_FORMAT_NAME | + DIFF_FORMAT_NAME_STATUS | + DIFF_FORMAT_CHECKDIFF | + DIFF_FORMAT_NO_OUTPUT; if (options->set_default) options->set_default(options); - if (options->output_format & DIFF_FORMAT_NAME) - count++; - if (options->output_format & DIFF_FORMAT_NAME_STATUS) - count++; - if (options->output_format & DIFF_FORMAT_CHECKDIFF) - count++; - if (options->output_format & DIFF_FORMAT_NO_OUTPUT) - count++; - if (count > 1) + if (HAS_MULTI_BITS(options->output_format & check_mask)) die(_("--name-only, --name-status, --check and -s are mutually exclusive")); if (HAS_MULTI_BITS(options->pickaxe_opts & DIFF_PICKAXE_KINDS_MASK)) -- 2.16.0.rc0.223.g4a4ac83678-goog ```