{"thread":{"id":"26011","subject":"[PATCH 5/6] Unified diff output format for bulk moves.","startedAt":"2010-12-09T21:38:01Z","lastAt":"2010-12-09T21:38:07Z","messageCount":7,"participants":["Yann Dirson"],"isPatch":true,"patchVersion":1,"patchTotal":6},"messages":[{"id":"157734","messageId":"1291930687-8990-1-git-send-email-ydirson@altern.org","threadId":"26011","inReplyTo":null,"subject":"[PATCH v9] Detection of directory renames","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2010-12-09T21:38:01Z","receivedAt":"2010-12-09T21:38:01Z","isPatch":true,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"This series requires:\n- v6 of \"generalizing sorted-array handling\" series\n- Jonathan Nieder's \"compat: add memrchr()\" patch,\n  aka. mid:<20101015051750.GA21830@burratino>\n\nChanges since v8:\n\n* use of the new sorted-array.h, which gives the main patch a diet\n* split of debug func into its own patch\n\nNext iteration will be merging the bulk-rm detection into this series.\nInterested eyes can get a preview at t/* branches on\nhttp://repo.or.cz/w/git/ydirson.git\n"},{"id":"157736","messageId":"1291930687-8990-2-git-send-email-ydirson@altern.org","threadId":"26011","inReplyTo":"1291930687-8990-1-git-send-email-ydirson@altern.org","subject":"[PATCH 1/6] Introduce debug_bulkmove() in diffcore-rename.","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2010-12-09T21:38:02Z","receivedAt":"2010-12-09T21:38:02Z","isPatch":true,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"This is an optional debug useful while developping new builk-* features.\n\nSigned-off-by: Yann Dirson <ydirson@altern.org>\n---\n diffcore-rename.c |   16 ++++++++++++++++\n 1 files changed, 16 insertions(+), 0 deletions(-)\n\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex f7afdeb..e16fdeb 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -7,6 +7,22 @@\n #include \"hash.h\"\n #include \"sorted-array.h\"\n \n+#define DEBUG_BULKMOVE 0\n+\n+#if DEBUG_BULKMOVE\n+#define debug_bulkmove(args) __debug_bulkmove args\n+void __debug_bulkmove(const char *fmt, ...)\n+{\n+\tva_list ap;\n+\tva_start(ap, fmt);\n+\tfprintf(stderr, \"[DBG] \");\n+\tvfprintf(stderr, fmt, ap);\n+\tva_end(ap);\n+}\n+#else\n+#define debug_bulkmove(args) do { /*nothing */ } while (0)\n+#endif\n+\n /* Table of rename/copy destinations */\n \n struct diff_rename_dst {\n-- \n1.7.2.3\n"},{"id":"157735","messageId":"1291930687-8990-3-git-send-email-ydirson@altern.org","threadId":"26011","inReplyTo":"1291930687-8990-1-git-send-email-ydirson@altern.org","subject":"[PATCH 2/6] Introduce bulk-move detection in diffcore.","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2010-12-09T21:38:03Z","receivedAt":"2010-12-09T21:38:03Z","isPatch":true,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"This feature tries to group together files moving from and to\nidentical directories - a common case being directory renames.\n\nThis only adds the detection logic.  The output of raw diff is displayed\nas \"Rnnn a/ b/\", and unified diff does not display them at all.  Output\nformats will be refined later in the series.\n\nIt is implemented as a new pass in diffcore-rename, occuring after the\nfile renames get detected, grouping those renames looking like a move\nof a full directory into some other place. It is activated by the new\n--detect-bulk-moves diffcore flag.\n\nPossible optimisations to this code include:\n* avoid use of i_am_not_single by using a separate list\n* use a more informative prefixcmp to avoid strcmp calls\n  eg. in discard_if_outside()\n* optimize for bulk insertions (avoid useless successive memmove's)\n\nOther future developements to be made on top of this include:\n* detect bulk removals (well, that one is rather a subset than a layer above),\n  and possibly bulk additions\n* detect bulk copies\n* detect inexact bulk-moves/copies (where some files were not moved, or were\n  moved to a different place) - problem of computing a similarity score\n* display as such the special case of directory move/rename\n* application of such new diffs: issue a conflict, or just a warning ?\n* teach git-svn (and others ?) to make use of that flag\n* handle new conflict type \"bulk-move/add\"\n* detect \"directory splits\" as well\n* use inexact dir renames to bump score of below-threshold renames\n  from/to same locations\n* support other types of bluk-grouping, like prefixes (see eg. kernel\n  5d1e859c), and maybe config-specified patterns\n* add yours here\n\nThis patch has been improved by the following contributions:\n- Jonathan Nieder: better implementation of copy_dirname()\n- Jonathan Nieder: portable implementation of memrchr() in another patch\n- Junio C Hamano: split individual renames hiding under control of another flag\n- Junio C Hamano: coding style issues\n- Junio C Hamano: better examples\n- Ævar Arnfjörð Bjarmason: Don't use C99 comments.\n- Jonathan Nieder: just too many other helpful suggestions to list them all\n\nThanks-to: Jonathan Nieder <jrnieder@gmail.com>\nThanks-to: Junio C Hamano <gitster@pobox.com>\nThanks-to: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\nSigned-off-by: Yann Dirson <ydirson@free.fr>\n---\n Documentation/diff-options.txt |    4 +\n Documentation/gitdiffcore.txt  |   12 ++\n diff-lib.c                     |    6 +-\n diff.c                         |    5 +\n diff.h                         |    3 +\n diffcore-rename.c              |  322 ++++++++++++++++++++++++++++++++++++++--\n diffcore.h                     |    1 +\n tree-diff.c                    |    4 +-\n 8 files changed, 340 insertions(+), 17 deletions(-)\n\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex f3e9538..186cd6f 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -245,6 +245,10 @@ endif::git-log[]\n \tdelete/add pair to be a rename if more than 90% of the file\n \thasn't changed.\n \n+--detect-bulk-moves::\n+\tDetect bulk move of all files of a directory into a\n+\tdifferent one.\n+\n -C[<n>]::\n --detect-copies[=<n>]::\n \tDetect copies as well as renames.  See also `--find-copies-harder`.\ndiff --git a/Documentation/gitdiffcore.txt b/Documentation/gitdiffcore.txt\nindex 6af29a4..93111ac 100644\n--- a/Documentation/gitdiffcore.txt\n+++ b/Documentation/gitdiffcore.txt\n@@ -175,6 +175,18 @@ the expense of making it slower.  Without `\\--find-copies-harder`,\n 'git diff-{asterisk}' commands can detect copies only if the file that was\n copied happened to have been modified in the same changeset.\n \n+Bulk move of all files of a directory into a different one can get\n+detected using the `\\--detect-bulk-moves` option.  This adds an\n+additional pass on top of the results of per-file rename detection.\n+They are reported with NULL SHA1 id, in addition to the file renames:\n+\n+------------------------------------------------\n+:040000 040000 0000000... 0000000... R100 foo/ bar/\n+:100644 100644 0123456... 1234567... R090 foo/file0 bar/file3\n+:100644 100644 2345678... 2345678... R100 foo/file1 bar/file1\n+:100644 100644 3456789... 3456789... R100 foo/file2 bar/file2\n+------------------------------------------------\n+\n \n diffcore-merge-broken: For Putting \"Complete Rewrites\" Back Together\n --------------------------------------------------------------------\ndiff --git a/diff-lib.c b/diff-lib.c\nindex 392ce2b..5ec3ddc 100644\n--- a/diff-lib.c\n+++ b/diff-lib.c\n@@ -208,7 +208,8 @@ int run_diff_files(struct rev_info *revs, unsigned int option)\n \t\t\t\t\t\t    ce_option, &dirty_submodule);\n \t\tif (!changed && !dirty_submodule) {\n \t\t\tce_mark_uptodate(ce);\n-\t\t\tif (!DIFF_OPT_TST(&revs->diffopt, FIND_COPIES_HARDER))\n+\t\t\tif (!DIFF_OPT_TST(&revs->diffopt, FIND_COPIES_HARDER) &&\n+\t\t\t    !DIFF_OPT_TST(&revs->diffopt, DETECT_BULK_MOVES))\n \t\t\t\tcontinue;\n \t\t}\n \t\toldmode = ce->ce_mode;\n@@ -338,7 +339,8 @@ static int show_modified(struct rev_info *revs,\n \n \toldmode = old->ce_mode;\n \tif (mode == oldmode && !hashcmp(sha1, old->sha1) && !dirty_submodule &&\n-\t    !DIFF_OPT_TST(&revs->diffopt, FIND_COPIES_HARDER))\n+\t    !DIFF_OPT_TST(&revs->diffopt, FIND_COPIES_HARDER) &&\n+\t    !DIFF_OPT_TST(&revs->diffopt, DETECT_BULK_MOVES))\n \t\treturn 0;\n \n \tdiff_change(&revs->diffopt, oldmode, mode,\ndiff --git a/diff.c b/diff.c\nindex db5e844..d64ae44 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3200,6 +3200,11 @@ int diff_opt_parse(struct diff_options *options, const char **av, int ac)\n \t\tDIFF_OPT_SET(options, REVERSE_DIFF);\n \telse if (!strcmp(arg, \"--find-copies-harder\"))\n \t\tDIFF_OPT_SET(options, FIND_COPIES_HARDER);\n+\telse if (!strcmp(arg, \"--detect-bulk-moves\")) {\n+\t\tDIFF_OPT_SET(options, DETECT_BULK_MOVES);\n+\t\tif (!options->detect_rename)\n+\t\t\toptions->detect_rename = DIFF_DETECT_RENAME;\n+\t}\n \telse if (!strcmp(arg, \"--follow\"))\n \t\tDIFF_OPT_SET(options, FOLLOW_RENAMES);\n \telse if (!strcmp(arg, \"--color\"))\ndiff --git a/diff.h b/diff.h\nindex 0083d92..1e2506c 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -78,6 +78,7 @@ typedef struct strbuf *(*diff_prefix_fn_t)(struct diff_options *opt, void *data)\n #define DIFF_OPT_IGNORE_UNTRACKED_IN_SUBMODULES (1 << 25)\n #define DIFF_OPT_IGNORE_DIRTY_SUBMODULES (1 << 26)\n #define DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG (1 << 27)\n+#define DIFF_OPT_DETECT_BULK_MOVES  (1 << 28)\n \n #define DIFF_OPT_TST(opts, flag)    ((opts)->flags & DIFF_OPT_##flag)\n #define DIFF_OPT_SET(opts, flag)    ((opts)->flags |= DIFF_OPT_##flag)\n@@ -265,6 +266,8 @@ extern void diffcore_fix_diff_index(struct diff_options *);\n \"  -C            detect copies.\\n\" \\\n \"  --find-copies-harder\\n\" \\\n \"                try unchanged files as candidate for copy detection.\\n\" \\\n+\"  --detect-bulk-moves\\n\" \\\n+\"                detect moves of all files of a single directory.\\n\" \\\n \"  -l<n>         limit rename attempts up to <n> paths.\\n\" \\\n \"  -O<file>      reorder diffs according to the <file>.\\n\" \\\n \"  -S<string>    find filepair whose only one side contains the string.\\n\" \\\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex e16fdeb..44df490 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -28,6 +28,7 @@ void __debug_bulkmove(const char *fmt, ...)\n struct diff_rename_dst {\n \tstruct diff_filespec *two;\n \tstruct diff_filepair *pair;\n+\tunsigned i_am_not_single:1; /* does not look for a match, only here to be looked at */\n };\n \n static int rename_dst_cmp(struct diff_filespec *ref_spec, struct diff_rename_dst *elem)\n@@ -39,14 +40,23 @@ static void rename_dst_init(struct diff_rename_dst *elem, struct diff_filespec *\n \telem->two = alloc_filespec(ref_spec->path);\n \tfill_filespec(elem->two, ref_spec->sha1, ref_spec->mode);\n \telem->pair = NULL;\n+\telem->i_am_not_single = 0;\n }\n declare_sorted_array(static, struct diff_rename_dst, rename_dst);\n declare_sorted_array_search_elem(static, struct diff_rename_dst, locate_rename_dst,\n \t\t\t\t struct diff_filespec *,\n \t\t\t\t rename_dst, rename_dst_cmp);\n-declare_sorted_array_insert_checkbool(static, struct diff_rename_dst, register_rename_dst,\n-\t\t\t\t      struct diff_filespec *, _gen_locate_rename_dst,\n-\t\t\t\t      rename_dst, rename_dst_cmp, rename_dst_init);\n+declare_sorted_array_insert_elem(static, struct diff_rename_dst, register_rename_dst,\n+\t\t\t\t struct diff_filespec *, _gen_locate_rename_dst,\n+\t\t\t\t rename_dst, rename_dst_cmp, rename_dst_init);\n+\n+static int rename_dst_dircmp(const char *ref_dirname, struct diff_rename_dst *elem)\n+{\n+\t// FIXME: calls strlen many times - but maybe the compiler optimizes ?\n+\treturn strncmp(ref_dirname, elem->two->path, strlen(ref_dirname));\n+}\n+declare_sorted_array_search_elem(static, struct diff_rename_dst, locate_rename_dst_dir,\n+\t\t\t\t const char *, rename_dst, rename_dst_dircmp);\n \n /* Table of rename/copy src files */\n \n@@ -361,8 +371,11 @@ static int find_exact_renames(void)\n \tfor (i = 0; i < rename_src_nr; i++)\n \t\tinsert_file_table(&file_table, -1, i, rename_src[i].one);\n \n-\tfor (i = 0; i < rename_dst_nr; i++)\n+\tfor (i = 0; i < rename_dst_nr; i++) {\n+\t\tif (rename_dst[i].i_am_not_single)\n+\t\t\tcontinue;\n \t\tinsert_file_table(&file_table, 1, i, rename_dst[i].two);\n+\t}\n \n \t/* Find the renames */\n \ti = for_each_hash(&file_table, find_same_files);\n@@ -389,6 +402,253 @@ static void record_if_better(struct diff_score m[], struct diff_score *o)\n \t\tm[worst] = *o;\n }\n \n+struct diff_bulk_rename {\n+\tstruct diff_filespec *one;\n+\tstruct diff_filespec *two;\n+\tint discarded;\n+};\n+\n+struct diff_bulk_rename_candidate {\n+\tchar one_path[PATH_MAX];\n+\tchar two_path[PATH_MAX];\n+};\n+static int bulkmove_candidates_cmp(struct diff_bulk_rename_candidate *ref_candidate,\n+\t\t\t\t   struct diff_bulk_rename *elem)\n+{\n+\tint cmp = strcmp(ref_candidate->one_path, elem->one->path);\n+\tif (!cmp)\n+\t\tcmp = strcmp(ref_candidate->two_path, elem->two->path);\n+\treturn cmp;\n+}\n+static void bulkmove_candidates_init(struct diff_bulk_rename *elem,\n+\t\t\t\t     struct diff_bulk_rename_candidate *ref_candidate)\n+{\n+\telem->one = alloc_filespec(ref_candidate->one_path);\n+\tfill_filespec(elem->one, null_sha1, S_IFDIR);\n+\telem->two = alloc_filespec(ref_candidate->two_path);\n+\tfill_filespec(elem->two, null_sha1, S_IFDIR);\n+\telem->discarded = 0;\n+}\n+declare_sorted_array(static, struct diff_bulk_rename, bulkmove_candidates);\n+declare_sorted_array_insertonly_elem(static, struct diff_bulk_rename, register_bulkmove_candidate,\n+\t\t\t\t     struct diff_bulk_rename_candidate *, bulkmove_candidates,\n+\t\t\t\t     bulkmove_candidates_cmp, bulkmove_candidates_init);\n+\n+/*\n+ * Copy dirname of src into dst, suitable to append a filename without\n+ * an additional \"/\".\n+ * Only handles relative paths since there is no absolute path in a git repo.\n+ * Writes \"\" when there is no \"/\" in src.\n+ * May overwrite more chars than really needed, if src ends with a \"/\".\n+ * Supports in-place modification of src by passing dst == src.\n+ */\n+static const char *copy_dirname(char *dst, const char *src)\n+{\n+\tsize_t len = strlen(src);\n+\tconst char *slash;\n+\tchar *end;\n+\n+\tif (len > 0 && src[len - 1] == '/')\n+\t\t/* Trailing slash.  Ignore it. */\n+\t\tlen--;\n+\n+\tslash = memrchr(src, '/', len);\n+\tif (!slash) {\n+\t\t*dst = '\\0';\n+\t\treturn dst;\n+\t}\n+\n+\tif (dst == src)\n+\t\tdst[slash - src + 1] = '\\0';\n+\telse {\n+\t\tend = mempcpy(dst, src, slash - src + 1);\n+\t\t*end = '\\0';\n+\t}\n+\treturn dst;\n+}\n+\n+// FIXME: leaks like hell.\n+/* See if the fact that one_leftover exists under one_parent_path in\n+ * dst tree should disqualify one_parent_path from bulkmove eligibility.\n+ * Return 1 if it disqualifies, 0 if it is OK.\n+ */\n+static int dispatched_to_different_dirs(const char *one_parent_path)\n+{\n+\t/* this might be a dir split, or files added\n+\t * after the bulk move, or just an isolated\n+\t * rename */\n+\tint two_idx, j, onep_len, maybe_dir_rename;\n+\tstruct diff_rename_dst *one_leftover =\n+\t\tone_leftover = locate_rename_dst_dir(one_parent_path);\n+\n+\tif (!one_leftover)\n+\t\treturn 0;\n+\n+\t/* try to see if it is a file added after the bulk move */\n+\ttwo_idx = one_leftover - rename_dst;\n+\tonep_len = strlen(one_parent_path);\n+\tmaybe_dir_rename = 1;\n+\n+\t/* check no leftover file was already here before */\n+\tfor (j = two_idx; j < rename_dst_nr; j++) {\n+\t\tif (strncmp(rename_dst[j].two->path,\n+\t\t\t    one_parent_path, onep_len))\n+\t\t\tbreak; /* exhausted directory in this direction */\n+\t\tdebug_bulkmove((\"leftover file %s in %s\\n\",\n+\t\t\t\trename_dst[j].two->path, one_parent_path));\n+\t\tif (rename_dst[j].i_am_not_single || /* those were already here */\n+\t\t    (rename_dst[j].pair &&\n+\t\t     !strncmp(rename_dst[j].pair->one->path,\n+\t\t\t      one_parent_path, onep_len) && /* renamed from here */\n+\t\t     strncmp(rename_dst[j].two->path,\n+\t\t\t     one_parent_path, onep_len))) { /* not to a subdir */\n+\t\t\tmaybe_dir_rename = 0;\n+\t\t\tdebug_bulkmove((\"... tells not a bulk move\\n\"));\n+\t\t\tbreak;\n+\t\t}\n+\t\tdebug_bulkmove((\"... not believed to prevent bulk move\\n\"));\n+\t}\n+\tif (!maybe_dir_rename)\n+\t\treturn 1;\n+\t/* try the other direction (dup code) */\n+\tfor (j = two_idx-1; j >= 0; j--) {\n+\t\tif (strncmp(rename_dst[j].two->path,\n+\t\t\t    one_parent_path, onep_len))\n+\t\t\tbreak; /* exhausted directory in this direction */\n+\t\tdebug_bulkmove((\"leftover file %s in '%s'\\n\",\n+\t\t\t\trename_dst[j].two->path, one_parent_path));\n+\t\tif (rename_dst[j].i_am_not_single || /* those were already here */\n+\t\t    (rename_dst[j].pair &&\n+\t\t     !strncmp(rename_dst[j].pair->one->path,\n+\t\t\t      one_parent_path, onep_len) && /* renamed from here */\n+\t\t     strncmp(rename_dst[j].two->path,\n+\t\t\t     one_parent_path, onep_len))) { /* not to a subdir */\n+\t\t\tmaybe_dir_rename = 0;\n+\t\t\tdebug_bulkmove((\"... tells not a bulk move\\n\"));\n+\t\t\tbreak;\n+\t\t}\n+\t\tdebug_bulkmove((\"... not believed to prevent bulk move\\n\"));\n+\t}\n+\tif (!maybe_dir_rename)\n+\t\treturn 1;\n+\n+\t/* Here we are in the case where a directory\n+\t * content was completely moved, but files\n+\t * were added to it afterwards.  Proceed as\n+\t * for a simple bulk move. */\n+\treturn 0;\n+}\n+\n+/*\n+ * Assumes candidate->one is a subdir of seen->one, mark 'seen' as\n+ * discarded if candidate->two is outside seen->two.  Also mark\n+ * 'candidate' itself as discarded if the conflict implies so.\n+ *\n+ * Return 1 if 'seen' was discarded\n+ */\n+static int discard_if_outside(struct diff_bulk_rename *candidate,\n+\t\t\t      struct diff_bulk_rename *seen)\n+{\n+\tif (!prefixcmp(candidate->two->path, seen->two->path)) {\n+\t\tdebug_bulkmove((\" 'dstpair' conforts 'seen'\\n\"));\n+\t\treturn 0;\n+\t}\n+\n+\tdebug_bulkmove((\"discarding %s -> %s from bulk moves (split into %s and %s)\\n\",\n+\t\t\tseen->one->path, seen->two->path,\n+\t\t\tcandidate->two->path, seen->two->path));\n+\tseen->discarded = 1;\n+\t/* Need to discard dstpair as well, unless moving from\n+\t * a strict subdir of seen->one or to a strict subdir\n+\t * of seen->two */\n+\tif (!strcmp(seen->one->path, candidate->one->path) &&\n+\t    prefixcmp(seen->two->path, candidate->two->path)) {\n+\t\tdebug_bulkmove((\"... and not adding self\\n\"));\n+\t\tcandidate->discarded = 1;\n+\t}\n+\treturn 1;\n+}\n+\n+/*\n+ * Check if the rename specified by \"dstpair\" could cause a\n+ * bulk move to be detected, record it in bulkmove_candidates if yes.\n+ */\n+static void check_one_bulk_move(struct diff_filepair *dstpair)\n+{\n+\tstruct diff_bulk_rename_candidate rawcandidate;\n+\n+\t/* genuine new files (or believed to be so) */\n+\tif (!dstpair)\n+\t\treturn;\n+\t/* dummy renames used by copy detection */\n+\tif (!strcmp(dstpair->one->path, dstpair->two->path))\n+\t\treturn;\n+\n+\tcopy_dirname(rawcandidate.one_path, dstpair->one->path);\n+\tcopy_dirname(rawcandidate.two_path, dstpair->two->path);\n+\n+\t/* simple rename with no directory change */\n+\tif (!strcmp(rawcandidate.one_path, rawcandidate.two_path))\n+\t\treturn;\n+\n+\tdebug_bulkmove((\"[] %s -> %s ?\\n\", dstpair->one->path, dstpair->two->path));\n+\n+\t/* loop up rawcandidate.one_path over successive parents */\n+\t// FIXME: also loop over rawcandidate.two_path prefixes\n+\tdo {\n+\t\tstruct diff_bulk_rename *seen;\n+\t\tint old_nr = bulkmove_candidates_nr;\n+\t\tstruct diff_bulk_rename *candidate =\n+\t\t\tregister_bulkmove_candidate(&rawcandidate);\n+\t\tdebug_bulkmove((\"[[]] %s ...\\n\", rawcandidate.one_path));\n+\t\tif (old_nr == bulkmove_candidates_nr) {\n+\t\t\tdebug_bulkmove((\" already seen\\n\"));\n+\t\t\treturn;\n+\t\t}\n+\n+\t\t/* After this commit, are there any files still under rawcandidate.one_path ?\n+\t\t * Any file left would disqualifies this dir for bulk move.\n+\t\t */\n+\t\tif (dispatched_to_different_dirs(rawcandidate.one_path)) {\n+\t\t\t// FIXME: check overlap with discard_if_outside()\n+\t\t\tcandidate->discarded = 1;\n+\t\t\treturn;\n+\t\t}\n+\n+\t\t/* walk up for rawcandidate.one_path prefixes */\n+\t\tfor (seen = candidate-1; (seen >= bulkmove_candidates) &&\n+\t\t\t     !prefixcmp(rawcandidate.one_path, seen->one->path); seen--) {\n+\t\t\tdebug_bulkmove((\" ? %s -> %s\\n\", seen->one->path, seen->two->path));\n+\t\t\t/* subdir of \"seen\" dest dir ? */\n+\t\t\tif (discard_if_outside(candidate, seen))\n+\t\t\t\tcontinue;\n+\t\t}\n+\t\t/* look down for other moves from rawcandidate.one_path */\n+\t\tseen = candidate + 1;\n+\t\tif (seen != bulkmove_candidates + bulkmove_candidates_nr &&\n+\t\t    !strcmp(rawcandidate.one_path, seen->one->path)) {\n+\t\t\tdebug_bulkmove((\" ? %s -> %s (2)\\n\", seen->one->path, seen->two->path));\n+\t\t\t/* subdir of \"seen\" dest dir ? */\n+\t\t\tif (discard_if_outside(candidate, seen))\n+\t\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* next parent if any */\n+\t\tcopy_dirname(rawcandidate.one_path, rawcandidate.one_path);\n+\t} while (*rawcandidate.one_path);\n+}\n+\n+/*\n+ * Take all file renames recorded so far and check if they could cause\n+ * a bulk move to be detected.\n+ */\n+static void diffcore_bulk_moves(void)\n+{\n+\tint i;\n+\tfor (i = 0; i < rename_dst_nr; i++)\n+\t\tcheck_one_bulk_move(rename_dst[i].pair);\n+}\n+\n void diffcore_rename(struct diff_options *options)\n {\n \tint detect_rename = options->detect_rename;\n@@ -399,6 +659,7 @@ void diffcore_rename(struct diff_options *options)\n \tstruct diff_score *mx;\n \tint i, j, rename_count;\n \tint num_create, num_src, dst_cnt;\n+\tstruct diff_bulk_rename *candidate;\n \n \tif (!minimum_score)\n \t\tminimum_score = DEFAULT_RENAME_SCORE;\n@@ -413,8 +674,7 @@ void diffcore_rename(struct diff_options *options)\n \t\t\t\tcontinue; /* not interested */\n \t\t\telse\n \t\t\t\tregister_rename_dst(p->two);\n-\t\t}\n-\t\telse if (!DIFF_FILE_VALID(p->two)) {\n+\t\t} else if (!DIFF_FILE_VALID(p->two)) {\n \t\t\t/*\n \t\t\t * If the source is a broken \"delete\", and\n \t\t\t * they did not really want to get broken,\n@@ -425,14 +685,23 @@ void diffcore_rename(struct diff_options *options)\n \t\t\tif (p->broken_pair && !p->score)\n \t\t\t\tp->one->rename_used++;\n \t\t\tregister_rename_src(p);\n-\t\t}\n-\t\telse if (detect_rename == DIFF_DETECT_COPY) {\n-\t\t\t/*\n-\t\t\t * Increment the \"rename_used\" score by\n-\t\t\t * one, to indicate ourselves as a user.\n-\t\t\t */\n-\t\t\tp->one->rename_used++;\n-\t\t\tregister_rename_src(p);\n+\t\t} else {\n+\t\t\tif (detect_rename == DIFF_DETECT_COPY) {\n+\t\t\t\t/*\n+\t\t\t\t * Increment the \"rename_used\" score by\n+\t\t\t\t * one, to indicate ourselves as a user.\n+\t\t\t\t */\n+\t\t\t\tp->one->rename_used++;\n+\t\t\t\tregister_rename_src(p);\n+\t\t\t}\n+\t\t\tif (DIFF_OPT_TST(options, DETECT_BULK_MOVES)) {\n+\t\t\t\t/* similarly, bulk move detection needs to\n+\t\t\t\t * see all files from second tree, but we don't\n+\t\t\t\t * want them to be matched against single sources.\n+\t\t\t\t */\n+\t\t\t\t// FIXME: check interaction with --find-copies-harder\n+\t\t\t\tregister_rename_dst(p->two)->i_am_not_single = 1;\n+\t\t\t}\n \t\t}\n \t}\n \tif (rename_dst_nr == 0 || rename_src_nr == 0)\n@@ -484,6 +753,8 @@ void diffcore_rename(struct diff_options *options)\n \n \t\tif (rename_dst[i].pair)\n \t\t\tcontinue; /* dealt with exact match already. */\n+\t\tif (rename_dst[i].i_am_not_single)\n+\t\t\tcontinue; /* not looking for a match. */\n \n \t\tm = &mx[dst_cnt * NUM_CANDIDATE_PER_DST];\n \t\tfor (j = 0; j < NUM_CANDIDATE_PER_DST; j++)\n@@ -544,7 +815,30 @@ void diffcore_rename(struct diff_options *options)\n \t/* At this point, we have found some renames and copies and they\n \t * are recorded in rename_dst.  The original list is still in *q.\n \t */\n+\n+\t/* Now possibly factorize those renames and copies. */\n+\tif (DIFF_OPT_TST(options, DETECT_BULK_MOVES))\n+\t\tdiffcore_bulk_moves();\n+\n \tDIFF_QUEUE_CLEAR(&outq);\n+\n+\t/* Now turn non-discarded bulkmove_candidates into real renames */\n+\tfor (candidate = bulkmove_candidates;\n+\t     candidate < bulkmove_candidates + bulkmove_candidates_nr; candidate++) {\n+\t\tstruct diff_filepair* pair;\n+\t\tif (candidate->discarded)\n+\t\t\tcontinue;\n+\t\t/* visualize toplevel dir if needed */\n+\t\tif (!*candidate->one->path)\n+\t\t\tcandidate->one->path = \"./\";\n+\t\tif (!*candidate->two->path)\n+\t\t\tcandidate->two->path = \"./\";\n+\t\tpair = diff_queue(&outq, candidate->one, candidate->two);\n+\t\tpair->score = MAX_SCORE;\n+\t\tpair->renamed_pair = 1;\n+\t\tpair->is_bulkmove = 1;\n+\t}\n+\n \tfor (i = 0; i < q->nr; i++) {\n \t\tstruct diff_filepair *p = q->queue[i];\n \t\tstruct diff_filepair *pair_to_free = NULL;\ndiff --git a/diffcore.h b/diffcore.h\nindex b8f1fde..6dab95b 100644\n--- a/diffcore.h\n+++ b/diffcore.h\n@@ -69,6 +69,7 @@ struct diff_filepair {\n \tunsigned broken_pair : 1;\n \tunsigned renamed_pair : 1;\n \tunsigned is_unmerged : 1;\n+\tunsigned is_bulkmove : 1;\n };\n #define DIFF_PAIR_UNMERGED(p) ((p)->is_unmerged)\n \ndiff --git a/tree-diff.c b/tree-diff.c\nindex 12c9a88..89cedd4 100644\n--- a/tree-diff.c\n+++ b/tree-diff.c\n@@ -49,7 +49,9 @@ static int compare_tree_entry(struct tree_desc *t1, struct tree_desc *t2, const\n \t\tshow_entry(opt, \"+\", t2, base, baselen);\n \t\treturn 1;\n \t}\n-\tif (!DIFF_OPT_TST(opt, FIND_COPIES_HARDER) && !hashcmp(sha1, sha2) && mode1 == mode2)\n+\tif (!DIFF_OPT_TST(opt, FIND_COPIES_HARDER) &&\n+\t    !DIFF_OPT_TST(opt, DETECT_BULK_MOVES) &&\n+\t    !hashcmp(sha1, sha2) && mode1 == mode2)\n \t\treturn 0;\n \n \t/*\n-- \n1.7.2.3\n"},{"id":"157738","messageId":"1291930687-8990-4-git-send-email-ydirson@altern.org","threadId":"26011","inReplyTo":"1291930687-8990-1-git-send-email-ydirson@altern.org","subject":"[PATCH 3/6] Raw diff output format for bulk moves.","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2010-12-09T21:38:04Z","receivedAt":"2010-12-09T21:38:04Z","isPatch":true,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"To distinguish the general bulk-move case (where the destination\ndirectory was pre-existing) from the directory-rename case (where it\nwas not), the output of raw diff is displayed as \"Rnnn a/* b/\".  Those\ncannot be confused with renames of files named \"whatever/*\" with a\nliteral star character, from the full-zero SHA1's.\n\nSigned-off-by: Yann Dirson <ydirson@free.fr>\n---\n Documentation/gitdiffcore.txt |    2 +-\n diff.c                        |    9 ++++++++-\n 2 files changed, 9 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/gitdiffcore.txt b/Documentation/gitdiffcore.txt\nindex 93111ac..2538dc0 100644\n--- a/Documentation/gitdiffcore.txt\n+++ b/Documentation/gitdiffcore.txt\n@@ -181,7 +181,7 @@ additional pass on top of the results of per-file rename detection.\n They are reported with NULL SHA1 id, in addition to the file renames:\n \n ------------------------------------------------\n-:040000 040000 0000000... 0000000... R100 foo/ bar/\n+:040000 040000 0000000... 0000000... R100 foo/* bar/\n :100644 100644 0123456... 1234567... R090 foo/file0 bar/file3\n :100644 100644 2345678... 2345678... R100 foo/file1 bar/file1\n :100644 100644 3456789... 3456789... R100 foo/file2 bar/file2\ndiff --git a/diff.c b/diff.c\nindex d64ae44..0694d7f 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3499,7 +3499,14 @@ static void diff_flush_raw(struct diff_filepair *p, struct diff_options *opt)\n \tif (p->status == DIFF_STATUS_COPIED ||\n \t    p->status == DIFF_STATUS_RENAMED) {\n \t\tconst char *name_a, *name_b;\n-\t\tname_a = p->one->path;\n+\t\tif (p->is_bulkmove) {\n+\t\t\t/* append \"*\" to the first dirname */\n+\t\t\tchar buf[PATH_MAX];\n+\t\t\tchar *next = memccpy(buf, p->one->path, '\\0', PATH_MAX);\n+\t\t\tnext[-1] = '*'; *next = '\\0';\n+\t\t\tname_a = buf;\n+\t\t} else\n+\t\t\tname_a = p->one->path;\n \t\tname_b = p->two->path;\n \t\tstrip_prefix(opt->prefix_length, &name_a, &name_b);\n \t\twrite_name_quoted(name_a, opt->file, inter_name_termination);\n-- \n1.7.2.3\n"},{"id":"157733","messageId":"1291930687-8990-5-git-send-email-ydirson@altern.org","threadId":"26011","inReplyTo":"1291930687-8990-1-git-send-email-ydirson@altern.org","subject":"[PATCH 4/6] Add testcases for the --detect-bulk-moves diffcore flag.","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2010-12-09T21:38:05Z","receivedAt":"2010-12-09T21:38:05Z","isPatch":true,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"This notably includes a couple of tests for cases known not to be\nworking correctly yet.\n\nThis patch has been improved by the following contributions:\n- Jonathan Nieder: reworked style of test script\n- Jonathan Nieder: use \"git commit\" in test instead of only plumbing,\n  and use test_tick\n- Sverre Rabbelier: anonymize hashes\n\nThanks-to: Jonathan Nieder <jrnieder@gmail.com>\nThanks-to: Sverre Rabbelier <srabbelier@gmail.com>\nSigned-off-by: Yann Dirson <ydirson@free.fr>\n---\n t/t4046-diff-bulk-move.sh |  296 +++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 296 insertions(+), 0 deletions(-)\n create mode 100755 t/t4046-diff-bulk-move.sh\n\ndiff --git a/t/t4046-diff-bulk-move.sh b/t/t4046-diff-bulk-move.sh\nnew file mode 100755\nindex 0000000..4b1c78e\n--- /dev/null\n+++ b/t/t4046-diff-bulk-move.sh\n@@ -0,0 +1,296 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2008,2010 Yann Dirson\n+# Copyright (c) 2005 Junio C Hamano\n+#\n+\n+# TODO for dir renames:\n+# * two dirs or more moving all their files to a single dir\n+# * simultaneous bulkmove and rename\n+\n+test_description='Test rename factorization in diff engine.\n+\n+'\n+. ./test-lib.sh\n+. \"$TEST_DIRECTORY\"/diff-lib.sh\n+\n+test_expect_success 'setup' '\n+\tgit commit --allow-empty -m \"original empty commit\"\n+\n+\tmkdir a &&\n+\tprintf \"Line %s\\n\" 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 >a/path0 &&\n+\tsed <a/path0 >a/path1 s/Line/Record/ &&\n+\tsed <a/path0 >a/path2 s/Line/Stuff/ &&\n+\tsed <a/path0 >a/path3 s/Line/Blurb/ &&\n+\n+\tgit update-index --add a/path* &&\n+\ttest_tick &&\n+\tgit commit -m \"original set of files\" &&\n+\n+\t: rename the directory &&\n+\tgit mv a b\n+'\n+test_expect_success 'diff-index --detect-bulk-moves after directory move.' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\ta/*\tb/\n+\t:100644 100644 X X R#\ta/path0\tb/path0\n+\t:100644 100644 X X R#\ta/path1\tb/path1\n+\t:100644 100644 X X R#\ta/path2\tb/path2\n+\t:100644 100644 X X R#\ta/path3\tb/path3\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup non-100% rename' '\n+\techo \"Line 16\" >>b/path0 &&\n+\tgit mv b/path2 b/2path &&\n+\tgit rm -f b/path3 &&\n+\techo anything >b/path100 &&\n+\tgit add b/path100\n+'\n+test_expect_success 'diff-index --detect-bulk-moves after content changes.' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\ta/*\tb/\n+\t:100644 000000 X X D#\ta/path3\n+\t:100644 100644 X X R#\ta/path2\tb/2path\n+\t:100644 100644 X X R#\ta/path0\tb/path0\n+\t:100644 100644 X X R#\ta/path1\tb/path1\n+\t:000000 100644 X X A#\tb/path100\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup bulk move that is not directory move' '\n+\tgit reset -q --hard &&\n+\n+\tmkdir c &&\n+\t(\n+\t\tfor i in 0 1 2; do\n+\t\t\tcp a/path$i c/apath$i || exit\n+\t\tdone\n+\t) &&\n+\tgit update-index --add c/apath* &&\n+\ttest_tick &&\n+\tgit commit -m \"first set of changes\" &&\n+\n+\tgit mv c/* a/\n+'\n+test_expect_success 'diff-index --detect-bulk-moves without full-dir rename.' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\tc/*\ta/\n+\t:100644 100644 X X R#\tc/apath0\ta/apath0\n+\t:100644 100644 X X R#\tc/apath1\ta/apath1\n+\t:100644 100644 X X R#\tc/apath2\ta/apath2\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup bulk move with new file in source dir' '\n+\techo > c/anotherpath \"How much wood?\" &&\n+\tgit update-index --add c/another*\n+'\n+test_expect_success 'diff-index --detect-bulk-moves with new file in source dir.' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\tc/*\ta/\n+\t:100644 100644 X X R#\tc/apath0\ta/apath0\n+\t:100644 100644 X X R#\tc/apath1\ta/apath1\n+\t:100644 100644 X X R#\tc/apath2\ta/apath2\n+\t:000000 100644 X X A#\tc/anotherpath\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup bulk move with interfering copy' '\n+\trm c/anotherpath &&\n+\tgit update-index --remove c/anotherpath &&\n+\tmkdir b &&\n+\tcp a/apath0 b/apath9 &&\n+\techo >> a/apath0 \"more\" &&\n+\tgit update-index --add a/apath0 b/apath9\n+'\n+# scores select the \"wrong\" one as \"moved\" (only a suboptimal detection)\n+test_expect_failure 'diff-index --detect-bulk-moves with interfering copy.' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\tc/*\ta/\n+\t:100644 100644 X X R#\tc/apath0\ta/apath0\n+\t:100644 100644 X X R#\tc/apath1\ta/apath1\n+\t:100644 100644 X X R#\tc/apath2\ta/apath2\n+\t:100644 100644 X X C#\tc/apath0\tb/apath9\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup bulk move to toplevel' '\n+\tgit reset -q --hard &&\n+\tgit mv c/* .\n+'\n+test_expect_success 'diff-index --detect-bulk-moves bulk move to toplevel.' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\tc/*\t./\n+\t:100644 100644 X X R#\tc/apath0\tapath0\n+\t:100644 100644 X X R#\tc/apath1\tapath1\n+\t:100644 100644 X X R#\tc/apath2\tapath2\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup move including a subdir, with some content changes' '\n+\tgit reset -q --hard &&\n+\tmv c a/ &&\n+\tgit update-index --add --remove a/c/* c/apath0 c/apath1 c/apath2 &&\n+\ttest_tick &&\n+\tgit commit -m \"move as subdir\" &&\n+\n+\tgit mv a b &&\n+\techo foo >>b/c/apath0 &&\n+\tgit update-index --add b/c/apath*\n+'\n+test_expect_success 'diff-index --detect-bulk-moves on a move including a subdir.' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\ta/*\tb/\n+\t:040000 040000 X X R#\ta/c/*\tb/c/\n+\t:100644 100644 X X R#\ta/c/apath0\tb/c/apath0\n+\t:100644 100644 X X R#\ta/c/apath1\tb/c/apath1\n+\t:100644 100644 X X R#\ta/c/apath2\tb/c/apath2\n+\t:100644 100644 X X R#\ta/path0\tb/path0\n+\t:100644 100644 X X R#\ta/path1\tb/path1\n+\t:100644 100644 X X R#\ta/path2\tb/path2\n+\t:100644 100644 X X R#\ta/path3\tb/path3\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup move of only a subdir' '\n+\tgit reset -q --hard &&\n+\t: rename a subdirectory of a/. &&\n+\tgit mv a/c a/d\n+'\n+test_expect_success 'moving a subdir only' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\ta/c/*\ta/d/\n+\t:100644 100644 X X R#\ta/c/apath0\ta/d/apath0\n+\t:100644 100644 X X R#\ta/c/apath1\ta/d/apath1\n+\t:100644 100644 X X R#\ta/c/apath2\ta/d/apath2\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup move without a subdir' '\n+\tgit reset -q --hard &&\n+\tmkdir b &&\n+\t: rename files in the directory but not subdir. &&\n+\tgit mv a/path* b/\n+'\n+test_expect_success 'moving files but not subdirs is not mistaken for dir move' '\n+\tcat >expected <<-EOF &&\n+\t:100644 100644 X X R#\ta/path0\tb/path0\n+\t:100644 100644 X X R#\ta/path1\tb/path1\n+\t:100644 100644 X X R#\ta/path2\tb/path2\n+\t:100644 100644 X X R#\ta/path3\tb/path3\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup move of files and subdirs to different places' '\n+\tgit reset -q --hard &&\n+\tgit mv a/c b &&\n+\tgit mv a d\n+'\n+test_expect_success 'moving subdirs into one dir and files into another is not mistaken for dir move' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\ta/c/*\tb/\n+\t:100644 100644 X X R#\ta/c/apath0\tb/apath0\n+\t:100644 100644 X X R#\ta/c/apath1\tb/apath1\n+\t:100644 100644 X X R#\ta/c/apath2\tb/apath2\n+\t:100644 100644 X X R#\ta/path0\td/path0\n+\t:100644 100644 X X R#\ta/path1\td/path1\n+\t:100644 100644 X X R#\ta/path2\td/path2\n+\t:100644 100644 X X R#\ta/path3\td/path3\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+# the same with different ordering\n+test_expect_success 'setup move of files and subdirs to different places' '\n+\tgit mv d 0\n+'\n+test_expect_success 'moving subdirs into one dir and files into another is not mistaken for dir move' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\ta/c/*\tb/\n+\t:100644 100644 X X R#\ta/path0\t0/path0\n+\t:100644 100644 X X R#\ta/path1\t0/path1\n+\t:100644 100644 X X R#\ta/path2\t0/path2\n+\t:100644 100644 X X R#\ta/path3\t0/path3\n+\t:100644 100644 X X R#\ta/c/apath0\tb/apath0\n+\t:100644 100644 X X R#\ta/c/apath1\tb/apath1\n+\t:100644 100644 X X R#\ta/c/apath2\tb/apath2\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_expect_success 'setup move of dir with only subdirs' '\n+\tgit reset -q --hard &&\n+\tmkdir a/b &&\n+\tmv a/path* a/b/ &&\n+\tgit update-index --add --remove a/path0 a/path1 a/path2 a/path3 a/b/path* &&\n+\ttest_tick &&\n+\tgit commit -m \"move all toplevel files down one level\" &&\n+\n+\tgit mv a z\n+'\n+# TODO: only a suboptimal non-detection\n+test_expect_failure 'moving a dir with no direct children files' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\ta/*\tz/\n+\t:040000 040000 X X R#\ta/b/*\tz/b/\n+\t:040000 040000 X X R#\ta/c/*\tz/c/\n+\t:100644 100644 X X R#\ta/b/path0\tz/b/path0\n+\t:100644 100644 X X R#\ta/b/path1\tz/b/path1\n+\t:100644 100644 X X R#\ta/b/path2\tz/b/path2\n+\t:100644 100644 X X R#\ta/b/path3\tz/b/path3\n+\t:100644 100644 X X R#\ta/c/apath0\tz/c/apath0\n+\t:100644 100644 X X R#\ta/c/apath1\tz/c/apath1\n+\t:100644 100644 X X R#\ta/c/apath2\tz/c/apath2\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+# now test moving all files from toplevel into subdir (does not hides file moves) (needs consensus on syntax)\n+# Note: this is a special case of move of a dir into one of its own subdirs, which in\n+# turn is a variant of new files/dirs being added into a dir after all its contents\n+# are moved away\n+\n+test_expect_success 'setup move from toplevel to subdir' '\n+\tgit reset -q --hard HEAD~3 &&\n+\tmv a/* . &&\n+\tgit update-index --add --remove a/path0 a/path1 a/path2 a/path3 path* &&\n+\ttest_tick &&\n+\tgit commit -m \"move all files to toplevel\" &&\n+\n+\tmkdir z &&\n+\tgit mv path* z/\n+'\n+test_expect_success '--detect-bulk-moves everything from toplevel.' '\n+\tcat >expected <<-EOF &&\n+\t:040000 040000 X X R#\t./*\tz/\n+\t:100644 100644 X X R#\tpath0\tz/path0\n+\t:100644 100644 X X R#\tpath1\tz/path1\n+\t:100644 100644 X X R#\tpath2\tz/path2\n+\t:100644 100644 X X R#\tpath3\tz/path3\n+\tEOF\n+\tgit diff-index --detect-bulk-moves HEAD >current &&\n+\tcompare_diff_raw expected current\n+'\n+\n+test_done\n-- \n1.7.2.3\n"},{"id":"157732","messageId":"1291930687-8990-6-git-send-email-ydirson@altern.org","threadId":"26011","inReplyTo":"1291930687-8990-1-git-send-email-ydirson@altern.org","subject":"[PATCH 5/6] Unified diff output format for bulk moves.","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2010-12-09T21:38:06Z","receivedAt":"2010-12-09T21:38:06Z","isPatch":true,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"The output produced is as shown below.\n\ndiff --git-detect-bulk-moves mips/nxp/pnx833x/common/ mips/pnx833x/common/\nbulk move with similarity index 100%\nbulk move from mips/nxp/pnx833x/common/\nbulk move to mips/pnx833x/common/\n\nSigned-off-by: Yann Dirson <ydirson@free.fr>\n---\n Documentation/diff-generate-patch.txt |   19 +++++++++++++\n diff.c                                |   48 +++++++++++++++++++++++++-------\n 2 files changed, 56 insertions(+), 11 deletions(-)\n\ndiff --git a/Documentation/diff-generate-patch.txt b/Documentation/diff-generate-patch.txt\nindex 3ac2bea..367e859 100644\n--- a/Documentation/diff-generate-patch.txt\n+++ b/Documentation/diff-generate-patch.txt\n@@ -71,6 +71,25 @@ separate lines indicate the old and the new mode.\n       rename to a\n \n \n+bulk move entries\n+-----------------\n+\n+When a bulk move is detected, a special block is output in addition to\n+the renames that constitute the bulk move, looking like this:\n+\n+       diff --git-detect-bulk-moves a/dir1/ b/dir2/\n+\n+It is essentially similar to the standard diff header, with a special\n+syntax showing we are describing a difference between two directories,\n+and not a change to be applied as-is to a file.\n+\n+It is followed by three specific extended header lines:\n+\n+       bulk move with similarity index <number>\n+       bulk move from <dir1>/\n+       bulk move to <dir2>/\n+\n+\n combined diff format\n --------------------\n \ndiff --git a/diff.c b/diff.c\nindex 0694d7f..551fab7 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2668,6 +2668,7 @@ static void run_diff_cmd(const char *pgm,\n \tconst char *xfrm_msg = NULL;\n \tint complete_rewrite = (p->status == DIFF_STATUS_MODIFIED) && p->score;\n \tint must_show_header = 0;\n+\tint use_color;\n \n \tif (!DIFF_OPT_TST(o, ALLOW_EXTERNAL))\n \t\tpgm = NULL;\n@@ -2677,14 +2678,36 @@ static void run_diff_cmd(const char *pgm,\n \t\t\tpgm = drv->external;\n \t}\n \n+\t/*\n+\t * don't use colors when the header is intended for an\n+\t * external diff driver\n+\t */\n+\tuse_color = DIFF_OPT_TST(o, COLOR_DIFF) && !pgm;\n+\n+\tif (p->is_bulkmove) {\n+\t\tconst char *set = diff_get_color(use_color, DIFF_METAINFO);\n+\t\tconst char *reset = diff_get_color(use_color, DIFF_RESET);\n+\t\tstruct strbuf *msgbuf;\n+\t\tchar *line_prefix = \"\";\n+\n+\t\tif (o->output_prefix) {\n+\t\t\tmsgbuf = o->output_prefix(o, o->output_prefix_data);\n+\t\t\tline_prefix = msgbuf->buf;\n+\t\t}\n+\t\tfprintf(o->file, \"%s%sdiff --git-detect-bulk-moves %s %s%s\\n\",\n+\t\t\tline_prefix, set, one->path, two->path, reset);\n+\t\tfprintf(o->file, \"%s%sbulk move with similarity index %d%%%s\\n\",\n+\t\t\tline_prefix, set, similarity_index(p), reset);\n+\t\tfprintf(o->file, \"%s%sbulk move from %s%s\\n\",\n+\t\t\tline_prefix, set, one->path, reset);\n+\t\tfprintf(o->file, \"%s%sbulk move to %s%s\\n\",\n+\t\t\tline_prefix, set, two->path, reset);\n+\t\treturn;\n+\t}\n+\n \tif (msg) {\n-\t\t/*\n-\t\t * don't use colors when the header is intended for an\n-\t\t * external diff driver\n-\t\t */\n \t\tfill_metainfo(msg, name, other, one, two, o, p,\n-\t\t\t      &must_show_header,\n-\t\t\t      DIFF_OPT_TST(o, COLOR_DIFF) && !pgm);\n+\t\t\t      &must_show_header, use_color);\n \t\txfrm_msg = msg->len ? msg->buf : NULL;\n \t}\n \n@@ -2757,8 +2780,10 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \t\treturn;\n \t}\n \n-\tdiff_fill_sha1_info(one);\n-\tdiff_fill_sha1_info(two);\n+\tif (!p->is_bulkmove) {\n+\t\tdiff_fill_sha1_info(one);\n+\t\tdiff_fill_sha1_info(two);\n+\t}\n \n \tif (!pgm &&\n \t    DIFF_FILE_VALID(one) && DIFF_FILE_VALID(two) &&\n@@ -3557,9 +3582,10 @@ static void diff_flush_patch(struct diff_filepair *p, struct diff_options *o)\n \tif (diff_unmodified_pair(p))\n \t\treturn;\n \n-\tif ((DIFF_FILE_VALID(p->one) && S_ISDIR(p->one->mode)) ||\n-\t    (DIFF_FILE_VALID(p->two) && S_ISDIR(p->two->mode)))\n-\t\treturn; /* no tree diffs in patch format */\n+\tif (!p->is_bulkmove &&\n+\t    ((DIFF_FILE_VALID(p->one) && S_ISDIR(p->one->mode)) ||\n+\t     (DIFF_FILE_VALID(p->two) && S_ISDIR(p->two->mode))))\n+\t\treturn; /* no tree diffs in patch format, except for bulk moves */\n \n \trun_diff(p, o);\n }\n-- \n1.7.2.3\n"},{"id":"157737","messageId":"1291930687-8990-7-git-send-email-ydirson@altern.org","threadId":"26011","inReplyTo":"1291930687-8990-1-git-send-email-ydirson@altern.org","subject":"[PATCH 6/6] [WIP] Allow hiding renames of individual files involved in a directory rename.","fromName":"Yann Dirson","fromEmail":"ydirson@altern.org","sentAt":"2010-12-09T21:38:07Z","receivedAt":"2010-12-09T21:38:07Z","isPatch":true,"sender":{"key":"ydirson@altern.org","avatar":"https://avatars.githubusercontent.com/u/1190950?v=4"},"body":"Once has identified groups of bulk-moved files, and then\nthe --hide-bulk-move-details flag hides those of the individual renames\nwhich carry no other information (further name change, or content changes).\n\nThis makes it much easier to a human reader to spot content changes\nin a commit that also moves a whole subtree.\n\nSigned-off-by: Yann Dirson <ydirson@free.fr>\n---\n Documentation/diff-options.txt |    5 +++\n diff.c                         |    7 ++++\n diff.h                         |    3 ++\n diffcore-rename.c              |   68 ++++++++++++++++++++++++++++++++++++++--\n diffcore.h                     |    1 +\n 5 files changed, 81 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt\nindex 186cd6f..cc4fe9c 100644\n--- a/Documentation/diff-options.txt\n+++ b/Documentation/diff-options.txt\n@@ -249,6 +249,11 @@ endif::git-log[]\n \tDetect bulk move of all files of a directory into a\n \tdifferent one.\n \n+--hide-bulk-move-details::\n+\tHide the individual files moves that make up a bulk move,\n+\twithout hinding other changes to the involved files (contents\n+\tchange, name change relative to the bulk move's destination).\n+\n -C[<n>]::\n --detect-copies[=<n>]::\n \tDetect copies as well as renames.  See also `--find-copies-harder`.\ndiff --git a/diff.c b/diff.c\nindex 551fab7..7e21e30 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3230,6 +3230,13 @@ int diff_opt_parse(struct diff_options *options, const char **av, int ac)\n \t\tif (!options->detect_rename)\n \t\t\toptions->detect_rename = DIFF_DETECT_RENAME;\n \t}\n+\telse if (!strcmp(arg, \"--hide-bulk-move-details\")) {\n+\t\tDIFF_OPT_SET(options, HIDE_DIR_RENAME_DETAILS);\n+\t\tif (!DIFF_OPT_TST(options, DETECT_BULK_MOVES))\n+\t\t\tDIFF_OPT_SET(options, DETECT_BULK_MOVES);\n+\t\tif (!options->detect_rename)\n+\t\t\toptions->detect_rename = DIFF_DETECT_RENAME;\n+\t}\n \telse if (!strcmp(arg, \"--follow\"))\n \t\tDIFF_OPT_SET(options, FOLLOW_RENAMES);\n \telse if (!strcmp(arg, \"--color\"))\ndiff --git a/diff.h b/diff.h\nindex 1e2506c..f28fed5 100644\n--- a/diff.h\n+++ b/diff.h\n@@ -79,6 +79,7 @@ typedef struct strbuf *(*diff_prefix_fn_t)(struct diff_options *opt, void *data)\n #define DIFF_OPT_IGNORE_DIRTY_SUBMODULES (1 << 26)\n #define DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG (1 << 27)\n #define DIFF_OPT_DETECT_BULK_MOVES  (1 << 28)\n+#define DIFF_OPT_HIDE_DIR_RENAME_DETAILS (1 << 29)\n \n #define DIFF_OPT_TST(opts, flag)    ((opts)->flags & DIFF_OPT_##flag)\n #define DIFF_OPT_SET(opts, flag)    ((opts)->flags |= DIFF_OPT_##flag)\n@@ -268,6 +269,8 @@ extern void diffcore_fix_diff_index(struct diff_options *);\n \"                try unchanged files as candidate for copy detection.\\n\" \\\n \"  --detect-bulk-moves\\n\" \\\n \"                detect moves of all files of a single directory.\\n\" \\\n+\"  --hide-bulk-move-details\\n\" \\\n+\"                hide individual files moves in a bulk move.\\n\" \\\n \"  -l<n>         limit rename attempts up to <n> paths.\\n\" \\\n \"  -O<file>      reorder diffs according to the <file>.\\n\" \\\n \"  -S<string>    find filepair whose only one side contains the string.\\n\" \\\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex 44df490..3d4b863 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -435,6 +435,34 @@ declare_sorted_array_insertonly_elem(static, struct diff_bulk_rename, register_b\n \t\t\t\t     bulkmove_candidates_cmp, bulkmove_candidates_init);\n \n /*\n+ * Marks as such file_rename if it is made uninteresting by dir_rename.\n+ * Returns -1 if the file_rename is outside of the range in which given\n+ * renames concerned by dir_rename are to be found (ie. end of loop),\n+ * 0 otherwise.\n+ */\n+static int maybe_mark_uninteresting(struct diff_rename_dst *file_rename,\n+\t\t\t\t    struct diff_bulk_rename *dir_rename,\n+\t\t\t\t    int one_len, int two_len)\n+{\n+\tif (!file_rename->pair) /* file add */\n+\t\treturn 0;\n+\tif (strncmp(file_rename->two->path,\n+\t\t    dir_rename->two->path, two_len))\n+\t\treturn -1;\n+\tif (strncmp(file_rename->pair->one->path,\n+\t\t    dir_rename->one->path, one_len) ||\n+\t    !basename_same(file_rename->pair->one, file_rename->two) ||\n+\t    file_rename->pair->score != MAX_SCORE)\n+\t\treturn 0;\n+\n+\tfile_rename->pair->uninteresting_rename = 1;\n+\tdebug_bulkmove((\"%s* -> %s* makes %s -> %s uninteresting\\n\",\n+\t\t\tdir_rename->one->path, dir_rename->two->path,\n+\t\t\tfile_rename->pair->one->path, file_rename->two->path));\n+\treturn 0;\n+}\n+\n+/*\n  * Copy dirname of src into dst, suitable to append a filename without\n  * an additional \"/\".\n  * Only handles relative paths since there is no absolute path in a git repo.\n@@ -642,11 +670,44 @@ static void check_one_bulk_move(struct diff_filepair *dstpair)\n  * Take all file renames recorded so far and check if they could cause\n  * a bulk move to be detected.\n  */\n-static void diffcore_bulk_moves(void)\n+static void diffcore_bulk_moves(int opt_hide_renames)\n {\n \tint i;\n \tfor (i = 0; i < rename_dst_nr; i++)\n \t\tcheck_one_bulk_move(rename_dst[i].pair);\n+\n+\tif (opt_hide_renames) {\n+\t\t/* flag as \"uninteresting\" those candidates hidden by dir move */\n+\t\tstruct diff_bulk_rename *candidate;\n+\t\tfor (candidate = bulkmove_candidates;\n+\t\t     candidate < bulkmove_candidates + bulkmove_candidates_nr;\n+\t\t     candidate++) {\n+\t\t\tint two_idx, i, one_len, two_len;\n+\t\t\tstruct diff_rename_dst *two_sample;\n+\t\t\tif (candidate->discarded)\n+\t\t\t\tcontinue;\n+\n+\t\t\t/* bisect to any entry within candidate dst dir */\n+\t\t\ttwo_sample = locate_rename_dst_dir(candidate->two->path);\n+\t\t\tif (!two_sample) {\n+\t\t\t\tdie(\"PANIC: %s candidate of rename not in target tree (from %s)\\n\",\n+\t\t\t\t    candidate->two->path, candidate->one->path);\n+\t\t\t}\n+\t\t\ttwo_idx = two_sample - rename_dst;\n+\n+\t\t\t/* now remove extraneous 100% files inside. */\n+\t\t\tone_len = strlen(candidate->one->path);\n+\t\t\ttwo_len = strlen(candidate->two->path);\n+\t\t\tfor (i = two_idx; i < rename_dst_nr; i++)\n+\t\t\t\tif (maybe_mark_uninteresting(rename_dst+i, candidate,\n+\t\t\t\t\t\t\t     one_len, two_len) < 0)\n+\t\t\t\t\tbreak;\n+\t\t\tfor (i = two_idx-1; i >= 0; i--)\n+\t\t\t\tif (maybe_mark_uninteresting(rename_dst+i, candidate,\n+\t\t\t\t\t\t\t     one_len, two_len) < 0)\n+\t\t\t\t\tbreak;\n+\t\t}\n+\t}\n }\n \n void diffcore_rename(struct diff_options *options)\n@@ -818,7 +879,7 @@ void diffcore_rename(struct diff_options *options)\n \n \t/* Now possibly factorize those renames and copies. */\n \tif (DIFF_OPT_TST(options, DETECT_BULK_MOVES))\n-\t\tdiffcore_bulk_moves();\n+\t\tdiffcore_bulk_moves(DIFF_OPT_TST(options, HIDE_DIR_RENAME_DETAILS));\n \n \tDIFF_QUEUE_CLEAR(&outq);\n \n@@ -853,7 +914,8 @@ void diffcore_rename(struct diff_options *options)\n \t\t\tstruct diff_rename_dst *dst =\n \t\t\t\tlocate_rename_dst(p->two);\n \t\t\tif (dst && dst->pair) {\n-\t\t\t\tdiff_q(&outq, dst->pair);\n+\t\t\t\tif (!dst->pair->uninteresting_rename)\n+\t\t\t\t\tdiff_q(&outq, dst->pair);\n \t\t\t\tpair_to_free = p;\n \t\t\t}\n \t\t\telse\ndiff --git a/diffcore.h b/diffcore.h\nindex 6dab95b..a4eb8e1 100644\n--- a/diffcore.h\n+++ b/diffcore.h\n@@ -69,6 +69,7 @@ struct diff_filepair {\n \tunsigned broken_pair : 1;\n \tunsigned renamed_pair : 1;\n \tunsigned is_unmerged : 1;\n+\tunsigned uninteresting_rename : 1;\n \tunsigned is_bulkmove : 1;\n };\n #define DIFF_PAIR_UNMERGED(p) ((p)->is_unmerged)\n-- \n1.7.2.3\n"}]}