{"thread":{"id":"54806","subject":"[PATCH 04/11] merge-ort: implement compare_pairs() and collect_renames()","startedAt":"2020-12-09T19:42:32Z","lastAt":"2020-12-15T22:10:59Z","messageCount":65,"participants":["Elijah Newren via GitGitGadget","Derrick Stolee","Elijah Newren","Johannes Schindelin","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":11},"messages":[{"id":"411903","messageId":"207bb9a837cb855721daa88caaad80e37cb40ffe.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 04/11] merge-ort: implement compare_pairs() and collect_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:20Z","receivedAt":"2020-12-09T19:42:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 27 +++++++++++++++++++++++++--\n 1 file changed, 25 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 1ff637e57af..3cdf8124b85 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -625,7 +625,13 @@ static int process_renames(struct merge_options *opt,\n \n static int compare_pairs(const void *a_, const void *b_)\n {\n-\tdie(\"Not yet implemented.\");\n+\tconst struct diff_filepair *a = *((const struct diff_filepair **)a_);\n+\tconst struct diff_filepair *b = *((const struct diff_filepair **)b_);\n+\n+\tint cmp = strcmp(a->one->path, b->one->path);\n+\tif (cmp)\n+\t\treturn cmp;\n+\treturn a->score - b->score;\n }\n \n /* Call diffcore_rename() to compute which files have changed on given side */\n@@ -671,7 +677,24 @@ static int collect_renames(struct merge_options *opt,\n \t\t\t struct diff_queue_struct *result,\n \t\t\t unsigned side_index)\n {\n-\tdie(\"Not yet implemented.\");\n+\tint i, clean = 1;\n+\tstruct diff_queue_struct *side_pairs;\n+\tstruct rename_info *renames = opt->priv->renames;\n+\n+\tside_pairs = &renames->pairs[side_index];\n+\n+\tfor (i = 0; i < side_pairs->nr; ++i) {\n+\t\tstruct diff_filepair *p = side_pairs->queue[i];\n+\n+\t\tif (p->status != 'R') {\n+\t\t\tdiff_free_filepair(p);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tp->score = side_index;\n+\t\tresult->queue[result->nr++] = p;\n+\t}\n+\n+\treturn clean;\n }\n \n static int detect_and_process_renames(struct merge_options *opt,\n-- \ngitgitgadget\n\n"},{"id":"411904","messageId":"ba30bc8686ef11115b369d351b4447c75a7bb9b5.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 03/11] merge-ort: implement detect_regular_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:19Z","receivedAt":"2020-12-09T19:42:42Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nBased heavily on merge-recursive's get_diffpairs() function.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 32 +++++++++++++++++++++++++++++++-\n 1 file changed, 31 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 92b765dd3f0..1ff637e57af 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -634,7 +634,33 @@ static void detect_regular_renames(struct merge_options *opt,\n \t\t\t\t struct tree *side,\n \t\t\t\t unsigned side_index)\n {\n-\tdie(\"Not yet implemented.\");\n+\tstruct diff_options diff_opts;\n+\tstruct rename_info *renames = opt->priv->renames;\n+\n+\trepo_diff_setup(opt->repo, &diff_opts);\n+\tdiff_opts.flags.recursive = 1;\n+\tdiff_opts.flags.rename_empty = 0;\n+\tdiff_opts.detect_rename = DIFF_DETECT_RENAME;\n+\tdiff_opts.rename_limit = opt->rename_limit;\n+\tif (opt->rename_limit <= 0)\n+\t\tdiff_opts.rename_limit = 1000;\n+\tdiff_opts.rename_score = opt->rename_score;\n+\tdiff_opts.show_rename_progress = opt->show_rename_progress;\n+\tdiff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n+\tdiff_setup_done(&diff_opts);\n+\tdiff_tree_oid(&merge_base->object.oid, &side->object.oid, \"\",\n+\t\t &diff_opts);\n+\tdiffcore_std(&diff_opts);\n+\n+\tif (diff_opts.needed_rename_limit > opt->priv->renames->needed_limit)\n+\t\topt->priv->renames->needed_limit = diff_opts.needed_rename_limit;\n+\n+\trenames->pairs[side_index] = diff_queued_diff;\n+\n+\tdiff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n+\tdiff_queued_diff.nr = 0;\n+\tdiff_queued_diff.queue = NULL;\n+\tdiff_flush(&diff_opts);\n }\n \n /*\n@@ -1379,6 +1405,10 @@ void merge_switch_to_result(struct merge_options *opt,\n \t\t\tprintf(\"%s\", sb->buf);\n \t\t}\n \t\tstring_list_clear(&olist, 0);\n+\n+\t\t/* Also include needed rename limit adjustment now */\n+\t\tdiff_warn_rename_limit(\"merge.renamelimit\",\n+\t\t\t\t opti->renames->needed_limit, 0);\n \t}\n \n \tmerge_finalize(opt, result);\n-- \ngitgitgadget\n\n"},{"id":"411905","messageId":"b9e0e1a60b92a6a220193bf9fe80796df91126a7.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:18Z","receivedAt":"2020-12-09T19:42:53Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++-------\n 1 file changed, 60 insertions(+), 8 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 90baedac407..92b765dd3f0 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -617,20 +617,72 @@ static int handle_content_merge(struct merge_options *opt,\n \n /*** Function Grouping: functions related to regular rename detection ***/\n \n+static int process_renames(struct merge_options *opt,\n+\t\t\t struct diff_queue_struct *renames)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+static int compare_pairs(const void *a_, const void *b_)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+/* Call diffcore_rename() to compute which files have changed on given side */\n+static void detect_regular_renames(struct merge_options *opt,\n+\t\t\t\t struct tree *merge_base,\n+\t\t\t\t struct tree *side,\n+\t\t\t\t unsigned side_index)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+/*\n+ * Get information of all renames which occurred in 'side_pairs', discarding\n+ * non-renames.\n+ */\n+static int collect_renames(struct merge_options *opt,\n+\t\t\t struct diff_queue_struct *result,\n+\t\t\t unsigned side_index)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n static int detect_and_process_renames(struct merge_options *opt,\n \t\t\t\t struct tree *merge_base,\n \t\t\t\t struct tree *side1,\n \t\t\t\t struct tree *side2)\n {\n-\tint clean = 1;\n+\tstruct diff_queue_struct combined;\n+\tstruct rename_info *renames = opt->priv->renames;\n+\tint s, clean = 1;\n+\n+\tmemset(&combined, 0, sizeof(combined));\n+\n+\tdetect_regular_renames(opt, merge_base, side1, 1);\n+\tdetect_regular_renames(opt, merge_base, side2, 2);\n+\n+\tALLOC_GROW(combined.queue,\n+\t\t renames->pairs[1].nr + renames->pairs[2].nr,\n+\t\t combined.alloc);\n+\tclean &= collect_renames(opt, &combined, 1);\n+\tclean &= collect_renames(opt, &combined, 2);\n+\tQSORT(combined.queue, combined.nr, compare_pairs);\n+\n+\tclean &= process_renames(opt, &combined);\n+\n+\t/* Free memory for renames->pairs[] and combined */\n+\tfor (s = 1; s <= 2; s++) {\n+\t\tfree(renames->pairs[s].queue);\n+\t\tDIFF_QUEUE_CLEAR(&renames->pairs[s]);\n+\t}\n+\tif (combined.nr) {\n+\t\tint i;\n+\t\tfor (i = 0; i < combined.nr; i++)\n+\t\t\tdiff_free_filepair(combined.queue[i]);\n+\t\tfree(combined.queue);\n+\t}\n \n-\t/*\n-\t * Rename detection works by detecting file similarity. Here we use\n-\t * a really easy-to-implement scheme: files are similar IFF they have\n-\t * the same filename. Therefore, by this scheme, there are no renames.\n-\t *\n-\t * TODO: Actually implement a real rename detection scheme.\n-\t */\n \treturn clean;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"411907","messageId":"35b070b9b7c495caed758362dcdaa61b724c1644.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 05/11] merge-ort: add basic outline for process_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:21Z","receivedAt":"2020-12-09T19:46:51Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nAdd code which determines which kind of special rename case each rename\ncorresponds to, but leave the handling of each type unimplemented for\nnow. Future commits will implement each one.\n\nThere is some tenuous resemblance to merge-recursive's\nprocess_renames(), but comparing the two is very unlikely to yield any\ninsights. merge-ort's process_renames() is a bit complex and I would\nprefer if I could simplify it more, but it is far easier to grok than\nmerge-recursive's function of the same name in my opinion. Plus,\nmerge-ort handles more rename conflict types than merge-recursive does.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 97 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 3cdf8124b85..faec29db955 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -620,7 +620,103 @@ static int handle_content_merge(struct merge_options *opt,\n static int process_renames(struct merge_options *opt,\n \t\t\t struct diff_queue_struct *renames)\n {\n-\tdie(\"Not yet implemented.\");\n+\tint clean_merge = 1, i;\n+\n+\tfor (i = 0; i < renames->nr; ++i) {\n+\t\tconst char *oldpath = NULL, *newpath;\n+\t\tstruct diff_filepair *pair = renames->queue[i];\n+\t\tstruct conflict_info *oldinfo = NULL, *newinfo = NULL;\n+\t\tstruct strmap_entry *old_ent, *new_ent;\n+\t\tunsigned int old_sidemask;\n+\t\tint target_index, other_source_index;\n+\t\tint source_deleted, collision, type_changed;\n+\n+\t\told_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n+\t\toldpath = old_ent->key;\n+\t\toldinfo = old_ent->value;\n+\n+\t\tnew_ent = strmap_get_entry(&opt->priv->paths, pair->two->path);\n+\t\tnewpath = new_ent->key;\n+\t\tnewinfo = new_ent->value;\n+\n+\t\t/*\n+\t\t * diff_filepairs have copies of pathnames, thus we have to\n+\t\t * use standard 'strcmp()' (negated) instead of '=='.\n+\t\t */\n+\t\tif (i+1 < renames->nr &&\n+\t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n+\t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n+\t\t\tconst char *pathnames[3];\n+\n+\t\t\tpathnames[0] = oldpath;\n+\t\t\tpathnames[1] = newpath;\n+\t\t\tpathnames[2] = renames->queue[i+1]->two->path;\n+\n+\t\t\tif (!strcmp(pathnames[1], pathnames[2])) {\n+\t\t\t\t/* Both sides renamed the same way. */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\n+\t\t\t\t/* We handled both renames, i.e. i+1 handled */\n+\t\t\t\ti++;\n+\t\t\t\t/* Move to next rename */\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\t/* This is a rename/rename(1to2) */\n+\t\t\tdie(\"Not yet implemented\");\n+\n+\t\t\ti++; /* We handled both renames, i.e. i+1 handled */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tVERIFY_CI(oldinfo);\n+\t\tVERIFY_CI(newinfo);\n+\t\ttarget_index = pair->score; /* from append_rename_pairs() */\n+\t\tassert(target_index == 1 || target_index == 2);\n+\t\tother_source_index = 3-target_index;\n+\t\told_sidemask = (1 << other_source_index); /* 2 or 4 */\n+\t\tsource_deleted = (oldinfo->filemask == 1);\n+\t\tcollision = ((newinfo->filemask & old_sidemask) != 0);\n+\t\ttype_changed = !source_deleted &&\n+\t\t\t(S_ISREG(oldinfo->stages[other_source_index].mode) !=\n+\t\t\t S_ISREG(newinfo->stages[target_index].mode));\n+\t\tif (type_changed && collision) {\n+\t\t\t/* special handling so later blocks can handle this */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t}\n+\n+\t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n+\n+\t\t/* Need to check for special types of rename conflicts... */\n+\t\tif (collision && !source_deleted) {\n+\t\t\t/* collision: rename/add or rename/rename(2to1) */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t} else if (collision && source_deleted) {\n+\t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t} else {\n+\t\t\t/* a few different cases... */\n+\t\t\tif (type_changed) {\n+\t\t\t\t/* rename vs. typechange */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t} else if (source_deleted) {\n+\t\t\t\t/* rename/delete */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t} else {\n+\t\t\t\t/* normal rename */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!type_changed) {\n+\t\t\t/* Mark the original as resolved by removal */\n+\t\t\toldinfo->merged.is_null = 1;\n+\t\t\toldinfo->merged.clean = 1;\n+\t\t}\n+\n+\t}\n+\n+\treturn clean_merge;\n }\n \n static int compare_pairs(const void *a_, const void *b_)\n-- \ngitgitgadget\n\n"},{"id":"411908","messageId":"9c79b9f4a096afeef620ff4fc1f85525923e3d02.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 06/11] merge-ort: add implementation of both sides renaming identically","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:22Z","receivedAt":"2020-12-09T19:47:28Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(1to1) handling, i.e. both sides of history\nrenaming a file but renaming the same way. This code replaces the\nfollowing from merge-recurisve.c:\n\n * all the 1to1 code in process_renames()\n * the RENAME_ONE_FILE_TO_ONE case of process_entry()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_rename_normal()\n * setup_rename_conflict_info()\n\nThe consolidation of four separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 21 +++++++++++++++++++--\n 1 file changed, 19 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex faec29db955..085e81196a5 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -647,14 +647,31 @@ static int process_renames(struct merge_options *opt,\n \t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n \t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n \t\t\tconst char *pathnames[3];\n+\t\t\tstruct version_info merged;\n+\t\t\tstruct conflict_info *base, *side1, *side2;\n+\t\t\tunsigned was_binary_blob = 0;\n \n \t\t\tpathnames[0] = oldpath;\n \t\t\tpathnames[1] = newpath;\n \t\t\tpathnames[2] = renames->queue[i+1]->two->path;\n \n+\t\t\tbase = strmap_get(&opt->priv->paths, pathnames[0]);\n+\t\t\tside1 = strmap_get(&opt->priv->paths, pathnames[1]);\n+\t\t\tside2 = strmap_get(&opt->priv->paths, pathnames[2]);\n+\n+\t\t\tVERIFY_CI(base);\n+\t\t\tVERIFY_CI(side1);\n+\t\t\tVERIFY_CI(side2);\n+\n \t\t\tif (!strcmp(pathnames[1], pathnames[2])) {\n-\t\t\t\t/* Both sides renamed the same way. */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\t/* Both sides renamed the same way */\n+\t\t\t\tassert(side1 == side2);\n+\t\t\t\tmemcpy(&side1->stages[0], &base->stages[0],\n+\t\t\t\t sizeof(merged));\n+\t\t\t\tside1->filemask |= (1 << 0);\n+\t\t\t\t/* Mark base as resolved by removal */\n+\t\t\t\tbase->merged.is_null = 1;\n+\t\t\t\tbase->merged.clean = 1;\n \n \t\t\t\t/* We handled both renames, i.e. i+1 handled */\n \t\t\t\ti++;\n-- \ngitgitgadget\n\n"},{"id":"411909","messageId":"14baa5874afc12f3ba9534c71b9d70f2c481c465.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 10/11] merge-ort: add implementation of normal rename handling","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:26Z","receivedAt":"2020-12-09T19:48:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement handling of normal renames. This code replaces the following\nfrom merge-recurisve.c:\n\n * the code relevant to RENAME_NORMAL in process_renames()\n * the RENAME_NORMAL case of process_entry()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_rename_normal()\n * setup_rename_conflict_info()\n\nThe consolidation of four separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\n(To be fair, the code for handling normal renames wasn't all that\ncomplicated beforehand, but it's still much simpler now.)\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 412a3b1da76..f2e4edf6506 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -843,7 +843,11 @@ static int process_renames(struct merge_options *opt,\n \t\t\t\t\t rename_branch, delete_branch);\n \t\t\t} else {\n \t\t\t\t/* normal rename */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\tmemcpy(&newinfo->stages[other_source_index],\n+\t\t\t\t &oldinfo->stages[other_source_index],\n+\t\t\t\t sizeof(newinfo->stages[0]));\n+\t\t\t\tnewinfo->filemask |= (1 << other_source_index);\n+\t\t\t\tnewinfo->pathnames[other_source_index] = oldpath;\n \t\t\t}\n \t\t}\n \n-- \ngitgitgadget\n\n"},{"id":"411910","messageId":"c069d34b15f8b28975898e089d64d166dd81b2aa.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 09/11] merge-ort: add implementation of rename/delete conflicts","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:25Z","receivedAt":"2020-12-09T19:49:11Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/delete conflicts, i.e. one side renames a file and the\nother deletes the file. This code replaces the following from\nmerge-recurisve.c:\n\n * the code relevant to RENAME_DELETE in process_renames()\n * the RENAME_DELETE case of process_entry()\n * handle_rename_delete()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_change_delete()\n * setup_rename_conflict_info()\n\nThe consolidation of five separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nTo be fair, there is a _slight_ tweak to process_entry() here, because\nrename/delete cases will also trigger the modify/delete codepath.\nHowever, we only want a modify/delete message to be printed for a\nrename/delete conflict if there is a content change in the renamed file\nin addition to the rename. So process_renames() and process_entry()\naren't quite fully orthogonal, but they are pretty close.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 47 +++++++++++++++++++++++++++++++++++++++--------\n 1 file changed, 39 insertions(+), 8 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 4ec6b0701f1..412a3b1da76 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -630,6 +630,7 @@ static int process_renames(struct merge_options *opt,\n \t\tunsigned int old_sidemask;\n \t\tint target_index, other_source_index;\n \t\tint source_deleted, collision, type_changed;\n+\t\tconst char *rename_branch = NULL, *delete_branch = NULL;\n \n \t\told_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n \t\toldpath = old_ent->key;\n@@ -752,6 +753,14 @@ static int process_renames(struct merge_options *opt,\n \t\tif (type_changed && collision) {\n \t\t\t/* special handling so later blocks can handle this */\n \t\t\tdie(\"Not yet implemented\");\n+\t\tif (source_deleted) {\n+\t\t\tif (target_index == 1) {\n+\t\t\t\trename_branch = opt->branch1;\n+\t\t\t\tdelete_branch = opt->branch2;\n+\t\t\t} else {\n+\t\t\t\trename_branch = opt->branch2;\n+\t\t\t\tdelete_branch = opt->branch1;\n+\t\t\t}\n \t\t}\n \n \t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n@@ -812,13 +821,26 @@ static int process_renames(struct merge_options *opt,\n \t\t\t\t \"to %s in %s, but deleted in %s.\"),\n \t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n \t\t} else {\n-\t\t\t/* a few different cases... */\n+\t\t\t/*\n+\t\t\t * a few different cases...start by copying the\n+\t\t\t * existing stage(s) from oldinfo over the newinfo\n+\t\t\t * and update the pathname(s).\n+\t\t\t */\n+\t\t\tmemcpy(&newinfo->stages[0], &oldinfo->stages[0],\n+\t\t\t sizeof(newinfo->stages[0]));\n+\t\t\tnewinfo->filemask |= (1 << 0);\n+\t\t\tnewinfo->pathnames[0] = oldpath;\n \t\t\tif (type_changed) {\n \t\t\t\t/* rename vs. typechange */\n \t\t\t\tdie(\"Not yet implemented\");\n \t\t\t} else if (source_deleted) {\n \t\t\t\t/* rename/delete */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\tnewinfo->path_conflict = 1;\n+\t\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed\"\n+\t\t\t\t\t \" to %s in %s, but deleted in %s.\"),\n+\t\t\t\t\t oldpath, newpath,\n+\t\t\t\t\t rename_branch, delete_branch);\n \t\t\t} else {\n \t\t\t\t/* normal rename */\n \t\t\t\tdie(\"Not yet implemented\");\n@@ -1346,12 +1368,21 @@ static void process_entry(struct merge_options *opt,\n \t\tmodify_branch = (side == 1) ? opt->branch1 : opt->branch2;\n \t\tdelete_branch = (side == 1) ? opt->branch2 : opt->branch1;\n \n-\t\tpath_msg(opt, path, 0,\n-\t\t\t _(\"CONFLICT (modify/delete): %s deleted in %s \"\n-\t\t\t \"and modified in %s. Version %s of %s left \"\n-\t\t\t \"in tree.\"),\n-\t\t\t path, delete_branch, modify_branch,\n-\t\t\t modify_branch, path);\n+\t\tif (ci->path_conflict &&\n+\t\t oideq(&ci->stages[0].oid, &ci->stages[side].oid)) {\n+\t\t\t/*\n+\t\t\t * This came from a rename/delete; no action to take,\n+\t\t\t * but avoid printing \"modify/delete\" conflict notice\n+\t\t\t * since the contents were not modified.\n+\t\t\t */\n+\t\t} else {\n+\t\t\tpath_msg(opt, path, 0,\n+\t\t\t\t _(\"CONFLICT (modify/delete): %s deleted in %s \"\n+\t\t\t\t \"and modified in %s. Version %s of %s left \"\n+\t\t\t\t \"in tree.\"),\n+\t\t\t\t path, delete_branch, modify_branch,\n+\t\t\t\t modify_branch, path);\n+\t\t}\n \t} else if (ci->filemask == 2 || ci->filemask == 4) {\n \t\t/* Added on one side */\n \t\tint side = (ci->filemask == 4) ? 2 : 1;\n-- \ngitgitgadget\n\n"},{"id":"411911","messageId":"ef8f315f828319a3390fde14e3aee6c5e587405e.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 01/11] merge-ort: add basic data structures for handling renames","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:17Z","receivedAt":"2020-12-09T19:51:10Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nThis will grow later, but we only need a few fields for basic rename\nhandling.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 25 +++++++++++++++++++++++++\n 1 file changed, 25 insertions(+)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex ef143348592..90baedac407 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -29,6 +29,25 @@\n #include \"unpack-trees.h\"\n #include \"xdiff-interface.h\"\n \n+struct rename_info {\n+\t/*\n+\t * pairs: pairing of filenames from diffcore_rename()\n+\t *\n+\t * Index 1 and 2 correspond to sides 1 & 2 as used in\n+\t * conflict_info.stages. Index 0 unused.\n+\t */\n+\tstruct diff_queue_struct pairs[3];\n+\n+\t/*\n+\t * needed_limit: value needed for inexact rename detection to run\n+\t *\n+\t * If the current rename limit wasn't high enough for inexact\n+\t * rename detection to run, this records the limit needed. Otherwise,\n+\t * this value remains 0.\n+\t */\n+\tint needed_limit;\n+};\n+\n struct merge_options_internal {\n \t/*\n \t * paths: primary data structure in all of merge ort.\n@@ -96,6 +115,11 @@ struct merge_options_internal {\n \t */\n \tstruct strmap output;\n \n+\t/*\n+\t * renames: various data relating to rename detection\n+\t */\n+\tstruct rename_info *renames;\n+\n \t/*\n \t * current_dir_name: temporary var used in collect_merge_info_callback()\n \t *\n@@ -1356,6 +1380,7 @@ static void merge_start(struct merge_options *opt, struct merge_result *result)\n \n \t/* Initialization of opt->priv, our internal merge data */\n \topt->priv = xcalloc(1, sizeof(*opt->priv));\n+\topt->priv->renames = xcalloc(1, sizeof(*opt->priv->renames));\n \n \t/*\n \t * Although we initialize opt->priv->paths with strdup_strings=0,\n-- \ngitgitgadget\n\n"},{"id":"411912","messageId":"ab15f85f6988ffe55e84b7aa3d804fcee4189805.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 08/11] merge-ort: add implementation of rename collisions","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:24Z","receivedAt":"2020-12-09T19:51:10Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(2to1) and rename/add handling, i.e. a file is\nrenamed into a location where another file is added (with that other\nfile either being a plain add or itself coming from a rename). Note\nthat rename collisions can also have a special case stacked on top: the\nfile being renamed on one side of history is deleted on the other\n(yielding either a rename/add/delete conflict or perhaps a\nrename/rename(2to1)/delete[/delete]) conflict.\n\nOne thing to note here is that when there is a double rename, the code\nin question only handles one of them at a time; a later iteration\nthrough the loop will handle the other. After they've both been\nhandled, process_entry()'s normal add/add code can handle the collision.\n\nThis code replaces the following from merge-recurisve.c:\n\n * all the 2to1 code in process_renames()\n * the RENAME_TWO_FILES_TO_ONE case of process_entry()\n * handle_rename_rename_2to1()\n * handle_rename_add()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_file_collision()\n * setup_rename_conflict_info()\n\nThe consolidation of six separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++---\n 1 file changed, 51 insertions(+), 3 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 75e638a23eb..4ec6b0701f1 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -759,10 +759,58 @@ static int process_renames(struct merge_options *opt,\n \t\t/* Need to check for special types of rename conflicts... */\n \t\tif (collision && !source_deleted) {\n \t\t\t/* collision: rename/add or rename/rename(2to1) */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\tconst char *pathnames[3];\n+\t\t\tstruct version_info merged;\n+\n+\t\t\tstruct conflict_info *base, *side1, *side2;\n+\t\t\tunsigned clean;\n+\n+\t\t\tpathnames[0] = oldpath;\n+\t\t\tpathnames[other_source_index] = oldpath;\n+\t\t\tpathnames[target_index] = newpath;\n+\n+\t\t\tbase = strmap_get(&opt->priv->paths, pathnames[0]);\n+\t\t\tside1 = strmap_get(&opt->priv->paths, pathnames[1]);\n+\t\t\tside2 = strmap_get(&opt->priv->paths, pathnames[2]);\n+\n+\t\t\tVERIFY_CI(base);\n+\t\t\tVERIFY_CI(side1);\n+\t\t\tVERIFY_CI(side2);\n+\n+\t\t\tclean = handle_content_merge(opt, pair->one->path,\n+\t\t\t\t\t\t &base->stages[0],\n+\t\t\t\t\t\t &side1->stages[1],\n+\t\t\t\t\t\t &side2->stages[2],\n+\t\t\t\t\t\t pathnames,\n+\t\t\t\t\t\t 1 + 2*opt->priv->call_depth,\n+\t\t\t\t\t\t &merged);\n+\n+\t\t\tmemcpy(&newinfo->stages[target_index], &merged,\n+\t\t\t sizeof(merged));\n+\t\t\tif (!clean) {\n+\t\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t\t _(\"CONFLICT (rename involved in \"\n+\t\t\t\t\t \"collision): rename of %s -> %s has \"\n+\t\t\t\t\t \"content conflicts AND collides \"\n+\t\t\t\t\t \"with another path; this may result \"\n+\t\t\t\t\t \"in nested conflict markers.\"),\n+\t\t\t\t\t oldpath, newpath);\n+\t\t\t}\n \t\t} else if (collision && source_deleted) {\n-\t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\t/*\n+\t\t\t * rename/add/delete or rename/rename(2to1)/delete:\n+\t\t\t * since oldpath was deleted on the side that didn't\n+\t\t\t * do the rename, there's not much of a content merge\n+\t\t\t * we can do for the rename. oldinfo->merged.is_null\n+\t\t\t * was already set, so we just leave things as-is so\n+\t\t\t * they look like an add/add conflict.\n+\t\t\t */\n+\n+\t\t\tnewinfo->path_conflict = 1;\n+\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed \"\n+\t\t\t\t \"to %s in %s, but deleted in %s.\"),\n+\t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n \t\t} else {\n \t\t\t/* a few different cases... */\n \t\t\tif (type_changed) {\n-- \ngitgitgadget\n\n"},{"id":"411913","messageId":"476553e2b2049452db0b453150ecab40d011ab0d.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 11/11] merge-ort: add implementation of type-changed rename handling","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:27Z","receivedAt":"2020-12-09T19:51:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement cases where renames are involved in type changes (i.e. the\nside of history that didn't rename the file changed its type from a\nregular file to a symlink or submodule). There was some code to handle\nthis in merge-recursive but only in the special case when the renamed\nfile had no content changes. The code here works differently -- it\nknows process_entry() can handle mode conflicts, so it does a few\nminimal tweaks to ensure process_entry() can just finish the job as\nneeded.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 33 +++++++++++++++++++++++++++++++--\n 1 file changed, 31 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex f2e4edf6506..64b23c8aa2a 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -752,7 +752,32 @@ static int process_renames(struct merge_options *opt,\n \t\t\t S_ISREG(newinfo->stages[target_index].mode));\n \t\tif (type_changed && collision) {\n \t\t\t/* special handling so later blocks can handle this */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\t/*\n+\t\t\t * if type_changed && collision are both true, then this\n+\t\t\t * was really a double rename, but one side wasn't\n+\t\t\t * detected due to lack of break detection. I.e.\n+\t\t\t * something like\n+\t\t\t * orig: has normal file 'foo'\n+\t\t\t * side1: renames 'foo' to 'bar', adds 'foo' symlink\n+\t\t\t * side2: renames 'foo' to 'bar'\n+\t\t\t * In this case, the foo->bar rename on side1 won't be\n+\t\t\t * detected because the new symlink named 'foo' is\n+\t\t\t * there and we don't do break detection. But we detect\n+\t\t\t * this here because we don't want to merge the content\n+\t\t\t * of the foo symlink with the foo->bar file, so we\n+\t\t\t * have some logic to handle this special case. The\n+\t\t\t * easiest way to do that is make 'bar' on side1 not\n+\t\t\t * be considered a colliding file but the other part\n+\t\t\t * of a normal rename. If the file is very different,\n+\t\t\t * well we're going to get content merge conflicts\n+\t\t\t * anyway so it doesn't hurt. And if the colliding\n+\t\t\t * file also has a different type, that'll be handled\n+\t\t\t * by the content merge logic in process_entry() too.\n+\t\t\t *\n+\t\t\t * See also t6430, 'rename vs. rename/symlink'\n+\t\t\t */\n+\t\t\tcollision = 0;\n+\t\t}\n \t\tif (source_deleted) {\n \t\t\tif (target_index == 1) {\n \t\t\t\trename_branch = opt->branch1;\n@@ -832,7 +857,11 @@ static int process_renames(struct merge_options *opt,\n \t\t\tnewinfo->pathnames[0] = oldpath;\n \t\t\tif (type_changed) {\n \t\t\t\t/* rename vs. typechange */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\t/* Mark the original as resolved by removal */\n+\t\t\t\tmemcpy(&oldinfo->stages[0].oid, &null_oid,\n+\t\t\t\t sizeof(oldinfo->stages[0].oid));\n+\t\t\t\toldinfo->stages[0].mode = 0;\n+\t\t\t\toldinfo->filemask &= 0x06;\n \t\t\t} else if (source_deleted) {\n \t\t\t\t/* rename/delete */\n \t\t\t\tnewinfo->path_conflict = 1;\n-- \ngitgitgadget\n"},{"id":"411914","messageId":"d4595397052568ea6ea0cf46190374c74140fed7.1607542887.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH 07/11] merge-ort: add implementation of both sides renaming differently","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:23Z","receivedAt":"2020-12-09T19:51:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(1to2) handling, i.e. both sides of history\nrenaming a file and rename it differently. This code replaces the\nfollowing from merge-recurisve.c:\n\n * all the 1to2 code in process_renames()\n * the RENAME_ONE_FILE_TO_TWO case of process_entry()\n * handle_rename_rename_1to2()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_file_collision()\n * setup_rename_conflict_info()\n\nThe consolidation of five separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nTo be fair, there is a _slight_ tweak to process_entry() here to make\nsure that the two different paths aren't marked as clean but are left in\na conflicted state. So process_renames() and process_entry() aren't\nquite entirely orthogonal, but they are pretty close.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++---\n 1 file changed, 55 insertions(+), 3 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 085e81196a5..75e638a23eb 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -680,7 +680,59 @@ static int process_renames(struct merge_options *opt,\n \t\t\t}\n \n \t\t\t/* This is a rename/rename(1to2) */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\tclean_merge = handle_content_merge(opt,\n+\t\t\t\t\t\t\t pair->one->path,\n+\t\t\t\t\t\t\t &base->stages[0],\n+\t\t\t\t\t\t\t &side1->stages[1],\n+\t\t\t\t\t\t\t &side2->stages[2],\n+\t\t\t\t\t\t\t pathnames,\n+\t\t\t\t\t\t\t 1 + 2 * opt->priv->call_depth,\n+\t\t\t\t\t\t\t &merged);\n+\t\t\tif (!clean_merge &&\n+\t\t\t merged.mode == side1->stages[1].mode &&\n+\t\t\t oideq(&merged.oid, &side1->stages[1].oid)) {\n+\t\t\t\twas_binary_blob = 1;\n+\t\t\t}\n+\t\t\tmemcpy(&side1->stages[1], &merged, sizeof(merged));\n+\t\t\tif (was_binary_blob) {\n+\t\t\t\t/*\n+\t\t\t\t * Getting here means we were attempting to\n+\t\t\t\t * merge a binary blob.\n+\t\t\t\t *\n+\t\t\t\t * Since we can't merge binaries,\n+\t\t\t\t * handle_content_merge() just takes one\n+\t\t\t\t * side. But we don't want to copy the\n+\t\t\t\t * contents of one side to both paths. We\n+\t\t\t\t * used the contents of side1 above for\n+\t\t\t\t * side1->stages, let's use the contents of\n+\t\t\t\t * side2 for side2->stages below.\n+\t\t\t\t */\n+\t\t\t\toidcpy(&merged.oid, &side2->stages[2].oid);\n+\t\t\t\tmerged.mode = side2->stages[2].mode;\n+\t\t\t}\n+\t\t\tmemcpy(&side2->stages[2], &merged, sizeof(merged));\n+\n+\t\t\tside1->path_conflict = 1;\n+\t\t\tside2->path_conflict = 1;\n+\t\t\t/*\n+\t\t\t * TODO: For renames we normally remove the path at the\n+\t\t\t * old name. It would thus seem consistent to do the\n+\t\t\t * same for rename/rename(1to2) cases, but we haven't\n+\t\t\t * done so traditionally and a number of the regression\n+\t\t\t * tests now encode an expectation that the file is\n+\t\t\t * left there at stage 1. If we ever decide to change\n+\t\t\t * this, add the following two lines here:\n+\t\t\t * base->merged.is_null = 1;\n+\t\t\t * base->merged.clean = 1;\n+\t\t\t * and remove the setting of base->path_conflict to 1.\n+\t\t\t */\n+\t\t\tbase->path_conflict = 1;\n+\t\t\tpath_msg(opt, oldpath, 0,\n+\t\t\t\t _(\"CONFLICT (rename/rename): %s renamed to \"\n+\t\t\t\t \"%s in %s and to %s in %s.\"),\n+\t\t\t\t pathnames[0],\n+\t\t\t\t pathnames[1], opt->branch1,\n+\t\t\t\t pathnames[2], opt->branch2);\n \n \t\t\ti++; /* We handled both renames, i.e. i+1 handled */\n \t\t\tcontinue;\n@@ -1257,13 +1309,13 @@ static void process_entry(struct merge_options *opt,\n \t\tint side = (ci->filemask == 4) ? 2 : 1;\n \t\tci->merged.result.mode = ci->stages[side].mode;\n \t\toidcpy(&ci->merged.result.oid, &ci->stages[side].oid);\n-\t\tci->merged.clean = !ci->df_conflict;\n+\t\tci->merged.clean = !ci->df_conflict && !ci->path_conflict;\n \t} else if (ci->filemask == 1) {\n \t\t/* Deleted on both sides */\n \t\tci->merged.is_null = 1;\n \t\tci->merged.result.mode = 0;\n \t\toidcpy(&ci->merged.result.oid, &null_oid);\n-\t\tci->merged.clean = 1;\n+\t\tci->merged.clean = !ci->path_conflict;\n \t}\n \n \t/*\n-- \ngitgitgadget\n\n"},{"id":"411915","messageId":"pull.812.git.1607542887.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":null,"subject":"[PATCH 00/11] merge-ort: add basic rename detection","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-09T19:41:16Z","receivedAt":"2020-12-09T19:52:12Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This series builds on en/merge-ort-2 and adds basic rename detection to\nmerge-ort.\n\nThe first five patches set up the basic algorithm structure. Patches 6-10\nimplement different type of rename-related cases, one-by-one. Patch 11 also\nimplements a rename-related case, but one that merge-recursive doesn't\nreally handle.\n\nA couple early patches mimic or even copy from merge-recursive, but in later\npatches the implementation here diverges heavily from merge-recursive's.\nPatches 6-10 refer to this repeatedly, all having a slight variant of the\nfollowing paragraph in their commit messages:\n\nThe consolidation of $NUM separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nThis patch series does not make more tests pass under\nGIT_TEST_MERGE_ALGORITHM=ort by itself, because once renames are handled\nthen content merging needs to happen and that code still does a die(\"Not yet\nimplemented.\"). I'll be soon submitting parallel patches for more basic\nconflict handling and recursiveness, and when all three series are merged\ndown (in any order), it will drop the number of test failures under\nGIT_TEST_MERGE_ALGORITHM=ort from 1448 to 60.\n\nElijah Newren (11):\n merge-ort: add basic data structures for handling renames\n merge-ort: add initial outline for basic rename detection\n merge-ort: implement detect_regular_renames()\n merge-ort: implement compare_pairs() and collect_renames()\n merge-ort: add basic outline for process_renames()\n merge-ort: add implementation of both sides renaming identically\n merge-ort: add implementation of both sides renaming differently\n merge-ort: add implementation of rename collisions\n merge-ort: add implementation of rename/delete conflicts\n merge-ort: add implementation of normal rename handling\n merge-ort: add implementation of type-changed rename handling\n\n merge-ort.c | 439 ++++++++++++++++++++++++++++++++++++++++++++++++++--\n 1 file changed, 423 insertions(+), 16 deletions(-)\n\n\nbase-commit: 2f73290465428ae9d088819b8a07bc5c4efe4a8b\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-812%2Fnewren%2Fort-renames-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-812/newren/ort-renames-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/812\n-- \ngitgitgadget\n"},{"id":"412035","messageId":"38f68a01-4d22-d88d-dfda-c85f67340819@gmail.com","threadId":"54806","inReplyTo":"ef8f315f828319a3390fde14e3aee6c5e587405e.1607542887.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 01/11] merge-ort: add basic data structures for handling renames","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-11T02:03:24Z","receivedAt":"2020-12-11T02:05:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> This will grow later, but we only need a few fields for basic rename\n> handling.\n\nPerhaps these things will be extremely clear as the patch\nseries continues, but...\n\n> +struct rename_info {\n> +\t/*\n> +\t * pairs: pairing of filenames from diffcore_rename()\n> +\t *\n> +\t * Index 1 and 2 correspond to sides 1 & 2 as used in\n> +\t * conflict_info.stages. Index 0 unused.\n\nHm. This seems wasteful. I'm sure that you have a reason to use\nindex 0 in the future instead of just avoiding instances of [i-1]\nindexes.\n\n> +\t */\n> +\tstruct diff_queue_struct pairs[3];\n> +\n> +\t/*\n> +\t * needed_limit: value needed for inexact rename detection to run\n> +\t *\n> +\t * If the current rename limit wasn't high enough for inexact\n> +\t * rename detection to run, this records the limit needed. Otherwise,\n> +\t * this value remains 0.\n> +\t */\n> +\tint needed_limit;\n> +};\n> +\n> struct merge_options_internal {\n> \t/*\n> \t * paths: primary data structure in all of merge ort.\n> @@ -96,6 +115,11 @@ struct merge_options_internal {\n> \t */\n> \tstruct strmap output;\n> \n> +\t/*\n> +\t * renames: various data relating to rename detection\n> +\t */\n> +\tstruct rename_info *renames;\n> +\n\nAnd here, you create this as a pointer, but...\n> \t/* Initialization of opt->priv, our internal merge data */\n> \topt->priv = xcalloc(1, sizeof(*opt->priv));\n> +\topt->priv->renames = xcalloc(1, sizeof(*opt->priv->renames));\n\n...unconditionally allocate it here. Perhaps there are other cases\nwhere 'struct merge_options_internal' is allocated without the renames\nmember?\n\nSearching merge-ort.c at this point does not appear to have any\nother allocations of opt->priv or struct merge_options_internal.\nPerhaps it would be best to include struct rename_info not as a\npointer?\n\nIf you do have a reason to keep it as a pointer, then perhaps it\nshould be freed in clear_internal_opts()?\n\nThanks,\n-Stolee\n"},{"id":"412036","messageId":"84a4d97b-8496-4c83-5d32-19f57e17a871@gmail.com","threadId":"54806","inReplyTo":"b9e0e1a60b92a6a220193bf9fe80796df91126a7.1607542887.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-11T02:39:42Z","receivedAt":"2020-12-11T02:41:26Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Signed-off-by: Elijah Newren \n> ---\n> merge-ort.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++-------\n> 1 file changed, 60 insertions(+), 8 deletions(-)\n> \n> diff --git a/merge-ort.c b/merge-ort.c\n> index 90baedac407..92b765dd3f0 100644\n> --- a/merge-ort.c\n> +++ b/merge-ort.c\n> @@ -617,20 +617,72 @@ static int handle_content_merge(struct merge_options *opt,\n> \n> /*** Function Grouping: functions related to regular rename detection ***/\n> \n> +static int process_renames(struct merge_options *opt,\n> +\t\t\t struct diff_queue_struct *renames)\n> +static int compare_pairs(const void *a_, const void *b_)\n> +/* Call diffcore_rename() to compute which files have changed on given side */\n> +static void detect_regular_renames(struct merge_options *opt,\n> +\t\t\t\t struct tree *merge_base,\n> +\t\t\t\t struct tree *side,\n> +\t\t\t\t unsigned side_index)\n> +static int collect_renames(struct merge_options *opt,\n> +\t\t\t struct diff_queue_struct *result,\n> +\t\t\t unsigned side_index)\n\nstandard \"I promise this will follow soon!\" strategy, OK.\n\n> static int detect_and_process_renames(struct merge_options *opt,\n> \t\t\t\t struct tree *merge_base,\n> \t\t\t\t struct tree *side1,\n> \t\t\t\t struct tree *side2)\n> {\n> -\tint clean = 1;\n> +\tstruct diff_queue_struct combined;\n> +\tstruct rename_info *renames = opt->priv->renames;\n\n(Re: my concerns that we don't need 'renames' to be a pointer,\nthis could easily be \"renames = &opt->priv.renames;\")\n\n> +\tint s, clean = 1;\n> +\n> +\tmemset(&combined, 0, sizeof(combined));\n> +\n> +\tdetect_regular_renames(opt, merge_base, side1, 1);\n> +\tdetect_regular_renames(opt, merge_base, side2, 2);\n\nFind the renames in each side's diff.\n\nI think the use of \"1\" and \"2\" here might be better situated\nfor an enum. Perhaps:\n\nenum merge_side {\n\tMERGE_SIDE1 = 0,\n\tMERGE_SIDE2 = 1,\n};\n\n(Note, I shift these values to 0 and 1, respectively, allowing\nus to truncate the pairs array to two entries while still\nbeing mentally clear.)\n\n> +\n> +\tALLOC_GROW(combined.queue,\n> +\t\t renames->pairs[1].nr + renames->pairs[2].nr,\n> +\t\t combined.alloc);\n> +\tclean &= collect_renames(opt, &combined, 1);\n> +\tclean &= collect_renames(opt, &combined, 2);\n\nMagic numbers again.\n\n> +\tQSORT(combined.queue, combined.nr, compare_pairs);\n> +\n> +\tclean &= process_renames(opt, &combined);\n\nI need to mentally remember that \"clean\" is a return state,\nbut _not_ a fail/success result. Even though we are using\n\"&=\" here, it shouldn't be \"&&=\" or even \"if (method()) return 1;\"\n\nLooking at how \"clean\" is used in struct merge_result, I\nwonder if there is a reason to use an \"int\" over a simple\n\"unsigned\" or even \"unsigned clean:1;\" You use -1 in places\nas well as a case of \"mi->clean = !!resolved;\"\n\nIf there is more meaning to values other than \"clean\" or\n\"!clean\", then an enum might be valuable.\n\n> +\t/* Free memory for renames->pairs[] and combined */\n> +\tfor (s = 1; s <= 2; s++) {\n> +\t\tfree(renames->pairs[s].queue);\n> +\t\tDIFF_QUEUE_CLEAR(&renames->pairs[s]);\n> +\t}\n\nThis loop is particularly unusual. Perhaps it would be\nbetter to do this instead:\n\n\tfree(renames->pairs[MERGE_SIDE1].queue);\n\tfree(renames->pairs[MERGE_SIDE2].queue);\n\tDIFF_QUEUE_CLEAR(&renames->pairs[MERGE_SIDE1]);\n\tDIFF_QUEUE_CLEAR(&renames->pairs[MERGE_SIDE2]);\n\n> +\tif (combined.nr) {\n> +\t\tint i;\n> +\t\tfor (i = 0; i < combined.nr; i++)\n> +\t\t\tdiff_free_filepair(combined.queue[i]);\n> +\t\tfree(combined.queue);\n> +\t}\n> \n> -\t/*\n> -\t * Rename detection works by detecting file similarity. Here we use\n> -\t * a really easy-to-implement scheme: files are similar IFF they have\n> -\t * the same filename. Therefore, by this scheme, there are no renames.\n> -\t *\n> -\t * TODO: Actually implement a real rename detection scheme.\n> -\t */\n> \treturn clean;\n\nI notice that this change causes detect_and_process_renames() to\nchange from an \"unhelpful result, but success\" to \"die() always\".\n\nI wonder if there is value in swapping the order of the patches\nto implement the static methods first. Of course, you hit the\n\"unreferenced static method\" problem, so maybe your strategy is\nbetter after all.\n\nThanks,\n-Stolee\n"},{"id":"412037","messageId":"59863009-1a11-ba74-fdd3-6271977b6ec9@gmail.com","threadId":"54806","inReplyTo":"ba30bc8686ef11115b369d351b4447c75a7bb9b5.1607542887.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 03/11] merge-ort: implement detect_regular_renames()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-11T02:54:34Z","receivedAt":"2020-12-11T02:56:06Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Based heavily on merge-recursive's get_diffpairs() function.\n\n(You're not kidding, and I should have looked here before making\nsome comments below.)\n\n> Signed-off-by: Elijah Newren \n> ---\n> merge-ort.c | 32 +++++++++++++++++++++++++++++++-\n> 1 file changed, 31 insertions(+), 1 deletion(-)\n> \n> diff --git a/merge-ort.c b/merge-ort.c\n> index 92b765dd3f0..1ff637e57af 100644\n> --- a/merge-ort.c\n> +++ b/merge-ort.c\n> @@ -634,7 +634,33 @@ static void detect_regular_renames(struct merge_options *opt,\n> \t\t\t\t struct tree *side,\n> \t\t\t\t unsigned side_index)\n> {\n> -\tdie(\"Not yet implemented.\");\n> +\tstruct diff_options diff_opts;\n> +\tstruct rename_info *renames = opt->priv->renames;\n> +\n> +\trepo_diff_setup(opt->repo, &diff_opts);\n> +\tdiff_opts.flags.recursive = 1;\n> +\tdiff_opts.flags.rename_empty = 0;\n> +\tdiff_opts.detect_rename = DIFF_DETECT_RENAME;\n> +\tdiff_opts.rename_limit = opt->rename_limit;\n\nI assume that opt->rename_limit has been initialized properly\nagainst merge.renameLimit/diff.renameLimit in another location...\n\n> +\tif (opt->rename_limit <= 0)\n> +\t\tdiff_opts.rename_limit = 1000;\n\n(I made the following comments before thinking to look at\nget_diffpairs() which behaves in an equivalent way with this\n\"1000\" constant limit. I'm not sure if there is a reason why\nthis limit is different from the _other_ limits I discovered,\nbut it might still be good to reduce magic literal ints by\ngrouping this \"1000\" into a const or macro.)\n\n...and this just assigns the default again. Why is this done\nhere instead of inside the diff machinery? Also, wouldn't a\ndiff.renameLimit = 0 imply no renames, not \"use default\"?\n\nI notice that the docs don't make this explicit:\n\ndiff.renameLimit::\n\tThe number of files to consider when performing the copy/rename\n\tdetection; equivalent to the 'git diff' option `-l`. This setting\n\thas no effect if rename detection is turned off.\n\nbut also too_many_rename_candidates() has this strange\ndefault check:\n\n\t/*\n\t * This basically does a test for the rename matrix not\n\t * growing larger than a \"rename_limit\" square matrix, ie:\n\t *\n\t * num_create * num_src > rename_limit * rename_limit\n\t */\n\tif (rename_limit <= 0)\n\t\trename_limit = 32767;\n\nthis is... a much larger limit than I would think is reasonable.\n\nOf course, diff_rename_limit_default is set to 400 inside diff.c.\nShould that be extracted as a constant so we can repeat it here?\n\n> +\tdiff_opts.rename_score = opt->rename_score;\n> +\tdiff_opts.show_rename_progress = opt->show_rename_progress;\n> +\tdiff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n> +\tdiff_setup_done(&diff_opts);\n> +\tdiff_tree_oid(&merge_base->object.oid, &side->object.oid, \"\",\n> +\t\t &diff_opts);\n> +\tdiffcore_std(&diff_opts);\n> +\n> +\tif (diff_opts.needed_rename_limit > opt->priv->renames->needed_limit)\n> +\t\topt->priv->renames->needed_limit = diff_opts.needed_rename_limit;\n> +\n> +\trenames->pairs[side_index] = diff_queued_diff;\n> +\n> +\tdiff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n> +\tdiff_queued_diff.nr = 0;\n> +\tdiff_queued_diff.queue = NULL;\n> +\tdiff_flush(&diff_opts);\n> }\n> \n> /*\n> @@ -1379,6 +1405,10 @@ void merge_switch_to_result(struct merge_options *opt,\n> \t\t\tprintf(\"%s\", sb->buf);\n> \t\t}\n> \t\tstring_list_clear(&olist, 0);\n> +\n> +\t\t/* Also include needed rename limit adjustment now */\n> +\t\tdiff_warn_rename_limit(\"merge.renamelimit\",\n> +\t\t\t\t opti->renames->needed_limit, 0);\n\nI suppose this new call is appropriate in this patch, since you assign\nthe value inside detect_regular_renames(), but it might be good to\ndescribe its presence in the commit message.\n\nThanks,\n-Stolee\n\n"},{"id":"412039","messageId":"8d744966-2344-1dbc-f2a2-7411be3bfc8b@gmail.com","threadId":"54806","inReplyTo":"207bb9a837cb855721daa88caaad80e37cb40ffe.1607542887.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 04/11] merge-ort: implement compare_pairs() and collect_renames()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-11T03:00:49Z","receivedAt":"2020-12-11T03:02:46Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n\nPerhaps worth pointing out comparison to score_compare() \n\n> Signed-off-by: Elijah Newren \n> ---\n> merge-ort.c | 27 +++++++++++++++++++++++++--\n> 1 file changed, 25 insertions(+), 2 deletions(-)\n> \n> diff --git a/merge-ort.c b/merge-ort.c\n> index 1ff637e57af..3cdf8124b85 100644\n> --- a/merge-ort.c\n> +++ b/merge-ort.c\n> @@ -625,7 +625,13 @@ static int process_renames(struct merge_options *opt,\n> \n> static int compare_pairs(const void *a_, const void *b_)\n> {\n> -\tdie(\"Not yet implemented.\");\n> +\tconst struct diff_filepair *a = *((const struct diff_filepair **)a_);\n> +\tconst struct diff_filepair *b = *((const struct diff_filepair **)b_);\n> +\n> +\tint cmp = strcmp(a->one->path, b->one->path);\n> +\tif (cmp)\n> +\t\treturn cmp;\n> +\treturn a->score - b->score;\n\nHm. I wasn't sure what would happen when subtracting these\n\"unsigned short\" scores, but I see that score_compare() does\nthe same. Any potential for an existing, hidden bug here?\n\nThanks,\n-Stolee\n"},{"id":"412040","messageId":"3691e3f1-8840-9b19-5fff-02fcb846de70@gmail.com","threadId":"54806","inReplyTo":"35b070b9b7c495caed758362dcdaa61b724c1644.1607542887.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 05/11] merge-ort: add basic outline for process_renames()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-11T03:24:23Z","receivedAt":"2020-12-11T03:26:33Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Add code which determines which kind of special rename case each rename\n> corresponds to, but leave the handling of each type unimplemented for\n> now. Future commits will implement each one.\n> \n> There is some tenuous resemblance to merge-recursive's\n> process_renames(), but comparing the two is very unlikely to yield any\n> insights. merge-ort's process_renames() is a bit complex and I would\n> prefer if I could simplify it more, but it is far easier to grok than\n> merge-recursive's function of the same name in my opinion. Plus,\n> merge-ort handles more rename conflict types than merge-recursive does.\n> \n> Signed-off-by: Elijah Newren \n> ---\n> merge-ort.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++-\n> 1 file changed, 97 insertions(+), 1 deletion(-)\n> \n> diff --git a/merge-ort.c b/merge-ort.c\n> index 3cdf8124b85..faec29db955 100644\n> --- a/merge-ort.c\n> +++ b/merge-ort.c\n> @@ -620,7 +620,103 @@ static int handle_content_merge(struct merge_options *opt,\n> static int process_renames(struct merge_options *opt,\n> \t\t\t struct diff_queue_struct *renames)\n> {\n> -\tdie(\"Not yet implemented.\");\n> +\tint clean_merge = 1, i;\n> +\n> +\tfor (i = 0; i < renames->nr; ++i) {\n> +\t\tconst char *oldpath = NULL, *newpath;\n\nThis \"= NULL\" is not necessary, since you initialize it to\nold_ent->key unconditionally.\n\n> +\t\tstruct diff_filepair *pair = renames->queue[i];\n> +\t\tstruct conflict_info *oldinfo = NULL, *newinfo = NULL;\n\nThese, too.\n\n> +\t\tstruct strmap_entry *old_ent, *new_ent;\n> +\t\tunsigned int old_sidemask;\n> +\t\tint target_index, other_source_index;\n> +\t\tint source_deleted, collision, type_changed;\n> +\n> +\t\told_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n> +\t\toldpath = old_ent->key;\n> +\t\toldinfo = old_ent->value;\n> +\n> +\t\tnew_ent = strmap_get_entry(&opt->priv->paths, pair->two->path);\n> +\t\tnewpath = new_ent->key;\n> +\t\tnewinfo = new_ent->value;\n\nThis is moving data around. I wonder if there is any possibility that\nold_ent or new_ent could be NULL here, and we should check for that?\n(The \"any possibility\" is probably \"is there a chance of a bug in the\nearlier logic that didn't cause a failure yet, but would cause a SEGFAULT\nhere?\".)\n\n> +\t\t/*\n> +\t\t * diff_filepairs have copies of pathnames, thus we have to\n> +\t\t * use standard 'strcmp()' (negated) instead of '=='.\n> +\t\t */\n> +\t\tif (i+1 < renames->nr &&\n\nnit: I tend to prefer \"i + 1\".\n\n> +\t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n> +\t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n> +\t\t\tconst char *pathnames[3];\n> +\n> +\t\t\tpathnames[0] = oldpath;\n> +\t\t\tpathnames[1] = newpath;\n> +\t\t\tpathnames[2] = renames->queue[i+1]->two->path;\n> +\n> +\t\t\tif (!strcmp(pathnames[1], pathnames[2])) {\n> +\t\t\t\t/* Both sides renamed the same way. */\n> +\t\t\t\tdie(\"Not yet implemented\");\n> +\n> +\t\t\t\t/* We handled both renames, i.e. i+1 handled */\n> +\t\t\t\ti++;\n> +\t\t\t\t/* Move to next rename */\n> +\t\t\t\tcontinue;\n> +\t\t\t}\n> +\n> +\t\t\t/* This is a rename/rename(1to2) */\n> +\t\t\tdie(\"Not yet implemented\");\n\nInteresting that you chose to do some internal logic to split this\ncase, but have both die(). Perhaps that is wise, but also this could\nhave been a die() at the start, along with the pathnames[] initialization\nin a later patch that implements the 1to1 case (leaving the 1to2 case\nto die()).\n\n> +\t\t\ti++; /* We handled both renames, i.e. i+1 handled */\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\tVERIFY_CI(oldinfo);\n> +\t\tVERIFY_CI(newinfo);\n> +\t\ttarget_index = pair->score; /* from append_rename_pairs() */\n\nHm. I don't see append_rename_pairs() anywhere else in the codebase.\nDo you mean record_rename_pair()? But in that case, I don't understand\nthe following assertion:\n\n> +\t\tassert(target_index == 1 || target_index == 2)> +\t\tother_source_index = 3-target_index;\n\nnit: \"3 - target_index\"\n\n> +\t\told_sidemask = (1 << other_source_index); /* 2 or 4 */\n> +\t\tsource_deleted = (oldinfo->filemask == 1);\n\nThis oldinfo->filemask check made me go to the declaration to find\n\n\t/*\n\t * For filemask and dirmask, see tree-walk.h's struct traverse_info,\n\t * particularly the documentation above the \"fn\" member. Note that\n\t * filemask = mask & ~dirmask from that documentation.\n\t */\n\tunsigned filemask:3;\n\tunsigned dirmask:3;\n\nPerhaps I've missed my window to complain about this comment pointing to\na comment in another struct definition instead of something like:\n\n\t/*\n\t * The ith bit corresponds to whether the ith entry is a file\n\t * (filemask) or a directory (dirmask). Thus, filemask & dirmask\n\t * is always zero and filemask | dirmask == 7 always.\n\t */\n\nAnd of course, looking at this struct provides the justification for\nusing \"1\" and \"2\" for the \"sides\" and wasting the 0th value, because\nit is consistent with the three entries here, using 0 as the base.\n\nThus, the comment about not using the 0 position could use a reference\nto these triples. I still think that using an enum would help here.\n\nComing back to the line in question, \"filemask == 1\" _does_ mean that\nthe file exists only in the base. Deleted indeed.\n\n> +\t\tcollision = ((newinfo->filemask & old_sidemask) != 0);\n> +\t\ttype_changed = !source_deleted &&\n> +\t\t\t(S_ISREG(oldinfo->stages[other_source_index].mode) !=\n> +\t\t\t S_ISREG(newinfo->stages[target_index].mode));\n> +\t\tif (type_changed && collision) {\n> +\t\t\t/* special handling so later blocks can handle this */\n> +\t\t\tdie(\"Not yet implemented\");\n> +\t\t}\n> +\n> +\t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n> +\n> +\t\t/* Need to check for special types of rename conflicts... */\n> +\t\tif (collision && !source_deleted) {\n> +\t\t\t/* collision: rename/add or rename/rename(2to1) */\n> +\t\t\tdie(\"Not yet implemented\");\n> +\t\t} else if (collision && source_deleted) {\n> +\t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n\nHow did we get to three actions here? I'll probably learn when\nit is implemented.\n\n> +\t\t\tdie(\"Not yet implemented\");\n> +\t\t} else {\n> +\t\t\t/* a few different cases... */\n> +\t\t\tif (type_changed) {\n> +\t\t\t\t/* rename vs. typechange */\n> +\t\t\t\tdie(\"Not yet implemented\");\n> +\t\t\t} else if (source_deleted) {\n> +\t\t\t\t/* rename/delete */\n> +\t\t\t\tdie(\"Not yet implemented\");\n> +\t\t\t} else {\n> +\t\t\t\t/* normal rename */\n> +\t\t\t\tdie(\"Not yet implemented\");\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!type_changed) {\n> +\t\t\t/* Mark the original as resolved by removal */\n> +\t\t\toldinfo->merged.is_null = 1;\n> +\t\t\toldinfo->merged.clean = 1;\n> +\t\t}\n> +\n> +\t}\n> +\n> +\treturn clean_merge;\n\nI'm glad you separated out this case organization from the implementations.\nIt's still a big dense, so I'll probably need to revisit as I see you fill\nin the rest.\n\nThanks,\n-Stolee\n"},{"id":"412041","messageId":"fb33f41f-ab00-fcb8-b64f-496c0e0450e7@gmail.com","threadId":"54806","inReplyTo":"9c79b9f4a096afeef620ff4fc1f85525923e3d02.1607542887.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 06/11] merge-ort: add implementation of both sides renaming identically","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-11T03:32:30Z","receivedAt":"2020-12-11T03:34:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Implement rename/rename(1to1) handling, i.e. both sides of history\n> renaming a file but renaming the same way. This code replaces the\n> following from merge-recurisve.c:\n> \n> * all the 1to1 code in process_renames()\n> * the RENAME_ONE_FILE_TO_ONE case of process_entry()\n> \n> Also, there is some shared code from merge-recursive.c for multiple\n> different rename cases which we will no longer need for this case (or\n> other rename cases):\n> \n> * handle_rename_normal()\n> * setup_rename_conflict_info()\n> \n> The consolidation of four separate codepaths into one is made possible\n> by a change in design: process_renames() tweaks the conflict_info\n> entries within opt->priv->paths such that process_entry() can then\n> handle all the non-rename conflict types (directory/file, modify/delete,\n> etc.) orthogonally. This means we're much less likely to miss special\n> implementation of some kind of combination of conflict types (see\n> commits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n> 2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\n> directory/file conflict handling in ort\", 2020-10-26) for more details).\n> That, together with letting worktree/index updating be handled\n> orthogonally in the merge_switch_to_result() function, dramatically\n> simplifies the code for various special rename cases.\n> \n> Signed-off-by: Elijah Newren \n> ---\n> merge-ort.c | 21 +++++++++++++++++++--\n> 1 file changed, 19 insertions(+), 2 deletions(-)\n> \n> diff --git a/merge-ort.c b/merge-ort.c\n> index faec29db955..085e81196a5 100644\n> --- a/merge-ort.c\n> +++ b/merge-ort.c\n> @@ -647,14 +647,31 @@ static int process_renames(struct merge_options *opt,\n> \t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n> \t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n> \t\t\tconst char *pathnames[3];\n> +\t\t\tstruct version_info merged;\n> +\t\t\tstruct conflict_info *base, *side1, *side2;\n> +\t\t\tunsigned was_binary_blob = 0;\n\nSince you are adding to the declarations here, I suppose it would be\nreasonable to include the 1to2/1to1 split here instead of the previous\npatch, if that seems useful to reduce the complexity of that patch.\n \n> \t\t\tpathnames[0] = oldpath;\n> \t\t\tpathnames[1] = newpath;\n> \t\t\tpathnames[2] = renames->queue[i+1]->two->path;\n>\n> +\t\t\tbase = strmap_get(&opt->priv->paths, pathnames[0]);\n> +\t\t\tside1 = strmap_get(&opt->priv->paths, pathnames[1]);\n> +\t\t\tside2 = strmap_get(&opt->priv->paths, pathnames[2]);\n> +\n> +\t\t\tVERIFY_CI(base);\n> +\t\t\tVERIFY_CI(side1);\n> +\t\t\tVERIFY_CI(side2);\n> +\n> \t\t\tif (!strcmp(pathnames[1], pathnames[2])) {\n> -\t\t\t\t/* Both sides renamed the same way. */\n> -\t\t\t\tdie(\"Not yet implemented\");\n> +\t\t\t\t/* Both sides renamed the same way */\n> +\t\t\t\tassert(side1 == side2);\n> +\t\t\t\tmemcpy(&side1->stages[0], &base->stages[0],\n> +\t\t\t\t sizeof(merged));\n> +\t\t\t\tside1->filemask |= (1 << 0);\n> +\t\t\t\t/* Mark base as resolved by removal */\n> +\t\t\t\tbase->merged.is_null = 1;\n> +\t\t\t\tbase->merged.clean = 1;\n\nLooks good.\n\nThanks,\n-Stolee\n\n"},{"id":"412042","messageId":"5b9f5db0-b4d5-f60d-b5c5-2f8376d4bf12@gmail.com","threadId":"54806","inReplyTo":"d4595397052568ea6ea0cf46190374c74140fed7.1607542887.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 07/11] merge-ort: add implementation of both sides renaming differently","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-11T03:39:55Z","receivedAt":"2020-12-11T03:41:28Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Implement rename/rename(1to2) handling, i.e. both sides of history\n> renaming a file and rename it differently. This code replaces the\n> following from merge-recurisve.c:\n> \n> * all the 1to2 code in process_renames()\n> * the RENAME_ONE_FILE_TO_TWO case of process_entry()\n> * handle_rename_rename_1to2()\n> \n> Also, there is some shared code from merge-recursive.c for multiple\n> different rename cases which we will no longer need for this case (or\n> other rename cases):\n> \n> * handle_file_collision()\n> * setup_rename_conflict_info()\n> \n> The consolidation of five separate codepaths into one is made possible\n> by a change in design:\n\nExcellent!\n\n> \t\t\t/* This is a rename/rename(1to2) */\n> -\t\t\tdie(\"Not yet implemented\");\n> +\t\t\tclean_merge = handle_content_merge(opt,\n> +\t\t\t\t\t\t\t pair->one->path,\n> +\t\t\t\t\t\t\t &base->stages[0],\n> +\t\t\t\t\t\t\t &side1->stages[1],\n> +\t\t\t\t\t\t\t &side2->stages[2],\n> +\t\t\t\t\t\t\t pathnames,\n> +\t\t\t\t\t\t\t 1 + 2 * opt->priv->call_depth,\n> +\t\t\t\t\t\t\t &merged);\n\n(this method currently die()s. ok)\n\n> +\t\t\tif (!clean_merge &&\n> +\t\t\t merged.mode == side1->stages[1].mode &&\n> +\t\t\t oideq(&merged.oid, &side1->stages[1].oid)) {\n> +\t\t\t\twas_binary_blob = 1;\n> +\t\t\t}\n\nnit: Extraneous braces?\n\n> +\t\t\tmemcpy(&side1->stages[1], &merged, sizeof(merged));\n> +\t\t\tif (was_binary_blob) {\n> +\t\t\t\t/*\n> +\t\t\t\t * Getting here means we were attempting to\n> +\t\t\t\t * merge a binary blob.\n> +\t\t\t\t *\n> +\t\t\t\t * Since we can't merge binaries,\n> +\t\t\t\t * handle_content_merge() just takes one\n> +\t\t\t\t * side. But we don't want to copy the\n> +\t\t\t\t * contents of one side to both paths. We\n> +\t\t\t\t * used the contents of side1 above for\n> +\t\t\t\t * side1->stages, let's use the contents of\n> +\t\t\t\t * side2 for side2->stages below.\n> +\t\t\t\t */\n> +\t\t\t\toidcpy(&merged.oid, &side2->stages[2].oid);\n> +\t\t\t\tmerged.mode = side2->stages[2].mode;\n> +\t\t\t}\n> +\t\t\tmemcpy(&side2->stages[2], &merged, sizeof(merged));\n> +\n> +\t\t\tside1->path_conflict = 1;\n> +\t\t\tside2->path_conflict = 1;\n> +\t\t\t/*\n> +\t\t\t * TODO: For renames we normally remove the path at the\n> +\t\t\t * old name. It would thus seem consistent to do the\n> +\t\t\t * same for rename/rename(1to2) cases, but we haven't\n> +\t\t\t * done so traditionally and a number of the regression\n> +\t\t\t * tests now encode an expectation that the file is\n> +\t\t\t * left there at stage 1. If we ever decide to change\n> +\t\t\t * this, add the following two lines here:\n> +\t\t\t * base->merged.is_null = 1;\n> +\t\t\t * base->merged.clean = 1;\n> +\t\t\t * and remove the setting of base->path_conflict to 1.\n> +\t\t\t */\n> +\t\t\tbase->path_conflict = 1;\n\nI'm getting the point of the review/evening where I'm starting to gloss\nover these important details. Time to take a break (after this patch).\n\n> +\t\t\tpath_msg(opt, oldpath, 0,\n> +\t\t\t\t _(\"CONFLICT (rename/rename): %s renamed to \"\n> +\t\t\t\t \"%s in %s and to %s in %s.\"),\n> +\t\t\t\t pathnames[0],\n> +\t\t\t\t pathnames[1], opt->branch1,\n> +\t\t\t\t pathnames[2], opt->branch2);\n\nThis output differs a bit from handle_rename_rename_1to2() in\nmerge-recursive.c:\n\n\toutput(opt, 1, _(\"CONFLICT (rename/rename): \"\n\t \"Rename \\\"%s\\\"->\\\"%s\\\" in branch \\\"%s\\\" \"\n\t \"rename \\\"%s\\\"->\\\"%s\\\" in \\\"%s\\\"%s\"),\n\t o->path, a->path, ci->ren1->branch,\n\t o->path, b->path, ci->ren2->branch,\n\t opt->priv->call_depth ? _(\" (left unresolved)\") : \"\");\n\nHow much do we want to have _exact_ output matches between the\ntwo strategies, at least in the short term?\n\n> @@ -1257,13 +1309,13 @@ static void process_entry(struct merge_options *opt,\n> \t\tint side = (ci->filemask == 4) ? 2 : 1;\n> \t\tci->merged.result.mode = ci->stages[side].mode;\n> \t\toidcpy(&ci->merged.result.oid, &ci->stages[side].oid);\n> -\t\tci->merged.clean = !ci->df_conflict;\n> +\t\tci->merged.clean = !ci->df_conflict && !ci->path_conflict;\n> \t} else if (ci->filemask == 1) {\n> \t\t/* Deleted on both sides */\n> \t\tci->merged.is_null = 1;\n> \t\tci->merged.result.mode = 0;\n> \t\toidcpy(&ci->merged.result.oid, &null_oid);\n> -\t\tci->merged.clean = 1;\n> +\t\tci->merged.clean = !ci->path_conflict;\n\nThese exist because this is the first time we assign path_conflict.\nSure.\n\nThanks,\n-Stolee\n\n"},{"id":"412063","messageId":"CABPp-BGNvZv9KNnnwGEz_sv9MV8zNTRjXGQ=gBd6kEy+NuEJbg@mail.gmail.com","threadId":"54806","inReplyTo":"84a4d97b-8496-4c83-5d32-19f57e17a871@gmail.com","subject":"Re: [PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-11T09:40:48Z","receivedAt":"2020-12-11T09:42:39Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Dec 10, 2020 at 6:39 PM Derrick Stolee wrote:\n>\n> On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > Signed-off-by: Elijah Newren \n> > ---\n> > merge-ort.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++-------\n> > 1 file changed, 60 insertions(+), 8 deletions(-)\n> >\n> > diff --git a/merge-ort.c b/merge-ort.c\n> > index 90baedac407..92b765dd3f0 100644\n> > --- a/merge-ort.c\n> > +++ b/merge-ort.c\n> > @@ -617,20 +617,72 @@ static int handle_content_merge(struct merge_options *opt,\n> >\n> > /*** Function Grouping: functions related to regular rename detection ***/\n> >\n> > +static int process_renames(struct merge_options *opt,\n> > + struct diff_queue_struct *renames)\n> > +static int compare_pairs(const void *a_, const void *b_)\n> > +/* Call diffcore_rename() to compute which files have changed on given side */\n> > +static void detect_regular_renames(struct merge_options *opt,\n> > + struct tree *merge_base,\n> > + struct tree *side,\n> > + unsigned side_index)\n> > +static int collect_renames(struct merge_options *opt,\n> > + struct diff_queue_struct *result,\n> > + unsigned side_index)\n>\n> standard \"I promise this will follow soon!\" strategy, OK.\n>\n> > static int detect_and_process_renames(struct merge_options *opt,\n> > struct tree *merge_base,\n> > struct tree *side1,\n> > struct tree *side2)\n> > {\n> > - int clean = 1;\n> > + struct diff_queue_struct combined;\n> > + struct rename_info *renames = opt->priv->renames;\n>\n> (Re: my concerns that we don't need 'renames' to be a pointer,\n> this could easily be \"renames = &opt->priv.renames;\")\n\nYeah, there'll be a lot of these...\n\n>\n> > + int s, clean = 1;\n> > +\n> > + memset(&combined, 0, sizeof(combined));\n> > +\n> > + detect_regular_renames(opt, merge_base, side1, 1);\n> > + detect_regular_renames(opt, merge_base, side2, 2);\n>\n> Find the renames in each side's diff.\n>\n> I think the use of \"1\" and \"2\" here might be better situated\n> for an enum. Perhaps:\n\nI can see where you're coming from...but that's a monumentally huge\nshift. What about all my \"loops over the sides\"? Sure, the ones that\nare only two lines long could be just turned into code without a loop,\nbut when the loop is 140 lines long, that doesn't make much sense.\nLoop over enum ranges?\n\nAre all the variables that track an index still okay? Do I need to\nalso introduce an enum for all the filemask/dirmask/match_mask/etc.\nvariables:\n\nenum merge_mask {\n MERGE_JUST_BASE = 1,\n MERGE_JUST_SIDE1 = 2,\n MERGE_BASE_AND_SIDE1 = 3,\n MERGE_JUST_SIDE2 = 4,\n MERGE_BASE_AND_SIDE2 = 5,\n MERGE_SIDE1_AND_SIDE2 = 6,\n MERGE_BASE_AND_SIDE1_AND_SIDE2 = 7\n}\n\n? That seems like a pretty big pain to use.\n\nAlso, what about the code that uses a side index to get the other side\nindex? Or my conversions from side indices to masks (or vice versa)?\nI tend to put comments by these, but there's a _lot_ of them.\n\n\nI suspect this one would take a week to change, and I'd miss several\nlocations, and....some cases would certainly look cleaner but I\nsuspect some would be far uglier and end up being unchanged and then\nleave us with a mess of trying to understand both.\n\nWhat if I added a big comment near the top of the file that we've got\ndozens of variables that are arrays of size 3 which are meant to be\nindexed by the \"side\" of the three-way merge that it is tracking\ninformation for:\n 0: merge_base\n 1: side1\n 2: side2\n(though several of the variables might have index 0 unused since it\ndoesn't track anything specifically for the merge base), and further\nthat masks are used in certain variables which try to track which\nsides are present or match, with 2< enum merge_side {\n> MERGE_SIDE1 = 0,\n> MERGE_SIDE2 = 1,\n> };\n>\n> (Note, I shift these values to 0 and 1, respectively, allowing\n> us to truncate the pairs array to two entries while still\n> being mentally clear.)\n\nAs mentioned in the previous patch, the shift of the indices would\ncause me at least a large amount of mental confusion and I suspect it\nwould for others too. Both conflict_info.stages[] and\nconflict_info.pathnames[] are meant to be indexed by the side of the\nmerge (or the base), but using conflict_info.stages[MERGE_SIDE1] or\nconflict_info.pathnames[MERGE_SIDE2] as you have them defined here\nwould provide the wrong answer. Since there's a conflict_info per\nfile and it is used all over the code, this would just be ripe for\noff-by-one errors.\n\nSince both conflict_info.stages[] and renames->pairs[] are meant to be\nindexed by the merge side, this kind of conflict is inevitable. The\nonly clean solution is making both be arrays of size three, and just\nskipping index 0 in the variables that don't need to track something\nfor the merge_base.\n\n> > +\n> > + ALLOC_GROW(combined.queue,\n> > + renames->pairs[1].nr + renames->pairs[2].nr,\n> > + combined.alloc);\n> > + clean &= collect_renames(opt, &combined, 1);\n> > + clean &= collect_renames(opt, &combined, 2);\n>\n> Magic numbers again.\n>\n> > + QSORT(combined.queue, combined.nr, compare_pairs);\n> > +\n> > + clean &= process_renames(opt, &combined);\n>\n> I need to mentally remember that \"clean\" is a return state,\n> but _not_ a fail/success result. Even though we are using\n> \"&=\" here, it shouldn't be \"&&=\" or even \"if (method()) return 1;\"\n>\n> Looking at how \"clean\" is used in struct merge_result, I\n> wonder if there is a reason to use an \"int\" over a simple\n> \"unsigned\" or even \"unsigned clean:1;\" You use -1 in places\n> as well as a case of \"mi->clean = !!resolved;\"\n\nHeh, when I used an unsigned for a boolean, Jonathan Tan asked why I\ndidn't use an int. When I use an int for a boolean, you ask why I\ndon't use an unsigned. I think my stock answer should just be that\nthe other reviewer suggested it. ;-)\n\n> If there is more meaning to values other than \"clean\" or\n> \"!clean\", then an enum might be valuable.\n\nYeah, this came from unpack-trees.c and merge-recursive.c, where the\nvalue is usually 1 (clean) or 0 (not clean), but the special value of\n-1 signals something went wrong enough that we need to stop further\nprocessing and return up the call chain for any necessary cleanup\n(e.g. removal of lock files). The value of -1 is only used for things\nlike \"disk-is-full, can't write any more files to the working\ndirectory\", or \"failed to read one of the trees involved in the merge\nfrom the git object store\".\n\n-1, though, is the return value from unpack_trees(), traverse_trees(),\nand perhaps other places in the code, so I'd be worried about\nattempting to use a different special value for fear that I'd miss\nconverting the return value I got from one of those to the new special\nvalue.\n\nmerge-ort has far fewer locations where -1 appears (in part because\nthe checkout() code is an external function rather than being\nsprinkled everywhere), and it tends to cause the code to return\nimmediately, so most all call sites can assume a simple boolean value\nof either 0 or 1.\n\n> > + /* Free memory for renames->pairs[] and combined */\n> > + for (s = 1; s <= 2; s++) {\n> > + free(renames->pairs[s].queue);\n> > + DIFF_QUEUE_CLEAR(&renames->pairs[s]);\n> > + }\n>\n> This loop is particularly unusual. Perhaps it would be\n> better to do this instead:\n>\n> free(renames->pairs[MERGE_SIDE1].queue);\n> free(renames->pairs[MERGE_SIDE2].queue);\n> DIFF_QUEUE_CLEAR(&renames->pairs[MERGE_SIDE1]);\n> DIFF_QUEUE_CLEAR(&renames->pairs[MERGE_SIDE2]);\n\nIf this were the only one, then I'd absolutely agree. There's 7 such\nloops in the version of merge-ort.c in the 'ort' branch. I can't get\nrid of all of them, because even though some are short, some of those\nare very long for-loops. (The long ones use \"side\" for the loop\ncounter instead of \"s\" -- maybe I should use \"side\" even on the short\nones?)\n\nThere's also another 6 that loop over the sides including the\nmerge-base (thus including index 0). If those count, it's closer to\n13.\n\n> > + if (combined.nr) {\n> > + int i;\n> > + for (i = 0; i < combined.nr; i++)\n> > + diff_free_filepair(combined.queue[i]);\n> > + free(combined.queue);\n> > + }\n> >\n> > - /*\n> > - * Rename detection works by detecting file similarity. Here we use\n> > - * a really easy-to-implement scheme: files are similar IFF they have\n> > - * the same filename. Therefore, by this scheme, there are no renames.\n> > - *\n> > - * TODO: Actually implement a real rename detection scheme.\n> > - */\n> > return clean;\n>\n> I notice that this change causes detect_and_process_renames() to\n> change from an \"unhelpful result, but success\" to \"die() always\".\n>\n> I wonder if there is value in swapping the order of the patches\n> to implement the static methods first. Of course, you hit the\n> \"unreferenced static method\" problem, so maybe your strategy is\n> better after all.\n\nI used to do that kind of thing, but the unreferenced static method\nproblem is really annoying. It means the code doesn't even compile,\nwhich is a bad state to submit patches in. I can work around that by\nadding \"(void)unused_funcname;\" expressions somewhere in the code, but\nreviewers tend to be even more surprised by those.\n"},{"id":"412064","messageId":"CABPp-BG5_PzmKMV6YMs+fkTB-NFqDbJhduNoOWY5nt+wCE006Q@mail.gmail.com","threadId":"54806","inReplyTo":"38f68a01-4d22-d88d-dfda-c85f67340819@gmail.com","subject":"Re: [PATCH 01/11] merge-ort: add basic data structures for handling renames","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-11T09:41:09Z","receivedAt":"2020-12-11T09:42:55Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Dec 10, 2020 at 6:03 PM Derrick Stolee wrote:\n>\n> On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > This will grow later, but we only need a few fields for basic rename\n> > handling.\n>\n> Perhaps these things will be extremely clear as the patch\n> series continues, but...\n>\n> > +struct rename_info {\n> > + /*\n> > + * pairs: pairing of filenames from diffcore_rename()\n> > + *\n> > + * Index 1 and 2 correspond to sides 1 & 2 as used in\n> > + * conflict_info.stages. Index 0 unused.\n>\n> Hm. This seems wasteful. I'm sure that you have a reason to use\n> index 0 in the future instead of just avoiding instances of [i-1]\n> indexes.\n\nYes, it is...and it gets more wasteful when I increase the number of\nfields that are arrays of size 3 with none of them using index 0.\nCurrently, there's only 1 such field; later there will be 10.\n\nHowever, this does not scale with the number of files or size of the\nrepository or anything like that; it's a flat overhead. At this point\nin my patch submissions, that overhead is 16 bytes per merge. Later\nwhen I have 10 variables that are arrays of size three, it'll be 940\nbytes per merge.\n\nI'm not planning on using index 0 later; the reason for this really is\nto avoid off-by-one errors (it's one of the two biggest problems in\ncomputer science, right?). The off-by-one problem becomes huge when\nyou consider all the references:\n\n* The conflict_info type has stages which is an array of size three --\nindex 0 is always the base commit, index 1 is side1, and index 2 is\nside2. There is one of these per path involved in the merge, and are\nused all over the place, so it's nice to think in terms of \"1 is\nside1, 2 is side2\". (There is also a pathnames variable of size three\nwith the same indexing rules, and a bunch of bitmasks that rely on\n2<<0 == base, 2<<1 == side1, and 2<<2 == side2.)\n\n* These other 10 variables that are arrays of size 3 in the\nrename_info struct are all keeping track of information for side1 and\nside2. When you consider the number of references for all 10 of them\ncombined across the codebase, it adds up to quite a bit.\n\nI'm certain that if I would have had to use off-by-one indexing for\nthese 10 variables, while using not-off-by-one indexing for the stages\nand pathnames in conflict_info, I'm certain I would have messed it up\nmany dozen times and spent countless hours tracking down bugs. And I\nthink the result would be a lot harder to review. And future\ndevelopers would come along and fall into that trap and get various\nindices off.\n\nI'm willing to pay a one-time overhead of 940 bytes to avoid that.\n\n> > + */\n> > + struct diff_queue_struct pairs[3];\n> > +\n> > + /*\n> > + * needed_limit: value needed for inexact rename detection to run\n> > + *\n> > + * If the current rename limit wasn't high enough for inexact\n> > + * rename detection to run, this records the limit needed. Otherwise,\n> > + * this value remains 0.\n> > + */\n> > + int needed_limit;\n> > +};\n> > +\n> > struct merge_options_internal {\n> > /*\n> > * paths: primary data structure in all of merge ort.\n> > @@ -96,6 +115,11 @@ struct merge_options_internal {\n> > */\n> > struct strmap output;\n> >\n> > + /*\n> > + * renames: various data relating to rename detection\n> > + */\n> > + struct rename_info *renames;\n> > +\n>\n> And here, you create this as a pointer, but...\n> > /* Initialization of opt->priv, our internal merge data */\n> > opt->priv = xcalloc(1, sizeof(*opt->priv));\n> > + opt->priv->renames = xcalloc(1, sizeof(*opt->priv->renames));\n>\n> ...unconditionally allocate it here. Perhaps there are other cases\n> where 'struct merge_options_internal' is allocated without the renames\n> member?\n>\n> Searching merge-ort.c at this point does not appear to have any\n> other allocations of opt->priv or struct merge_options_internal.\n> Perhaps it would be best to include struct rename_info not as a\n> pointer?\n\nThat's a really good point; I'll try it out.\n\n> If you do have a reason to keep it as a pointer, then perhaps it\n> should be freed in clear_internal_opts()?\n\nEek. It's there in my 'ort' branch, but one of the problems trying to\nrearrange and clean things up to make nice digestible series is that\nyou sometimes forget to bring important parts along. Whoops; good\ncatch. I'm going to try just turning renames into an embedded struct\ninstead of a pointer, though. If it doesn't work out, I'll make sure\nto clear it.\n"},{"id":"412092","messageId":"CABPp-BGwPOgBFGtptUTAF=yVczTeW+dvQCNKVeZS4Hej0r5=kg@mail.gmail.com","threadId":"54806","inReplyTo":"59863009-1a11-ba74-fdd3-6271977b6ec9@gmail.com","subject":"Re: [PATCH 03/11] merge-ort: implement detect_regular_renames()","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-11T17:38:35Z","receivedAt":"2020-12-11T19:25:09Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Dec 10, 2020 at 6:54 PM Derrick Stolee wrote:\n>\n> On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > Based heavily on merge-recursive's get_diffpairs() function.\n>\n> (You're not kidding, and I should have looked here before making\n> some comments below.)\n\nI can provide some extra background on all the crazy magic numbers and\nnon-sensical treatment of tiny values, though. And since you were so\ncurious about these, I have an excuse to dump more info on you than\nyou probably were bargaining for... :-)\n\n> > Signed-off-by: Elijah Newren \n> > ---\n> > merge-ort.c | 32 +++++++++++++++++++++++++++++++-\n> > 1 file changed, 31 insertions(+), 1 deletion(-)\n> >\n> > diff --git a/merge-ort.c b/merge-ort.c\n> > index 92b765dd3f0..1ff637e57af 100644\n> > --- a/merge-ort.c\n> > +++ b/merge-ort.c\n> > @@ -634,7 +634,33 @@ static void detect_regular_renames(struct merge_options *opt,\n> > struct tree *side,\n> > unsigned side_index)\n> > {\n> > - die(\"Not yet implemented.\");\n> > + struct diff_options diff_opts;\n> > + struct rename_info *renames = opt->priv->renames;\n> > +\n> > + repo_diff_setup(opt->repo, &diff_opts);\n> > + diff_opts.flags.recursive = 1;\n> > + diff_opts.flags.rename_empty = 0;\n> > + diff_opts.detect_rename = DIFF_DETECT_RENAME;\n> > + diff_opts.rename_limit = opt->rename_limit;\n>\n> I assume that opt->rename_limit has been initialized properly\n> against merge.renameLimit/diff.renameLimit in another location...\n\nYes, see init_merge_options() and merge_recursive_config() in\nmerge-recursive.c. People using merge-ort will nevertheless be using\nsome functions out of merge-recursive.c...for now.\n\n> > + if (opt->rename_limit <= 0)\n> > + diff_opts.rename_limit = 1000;\n>\n> (I made the following comments before thinking to look at\n> get_diffpairs() which behaves in an equivalent way with this\n> \"1000\" constant limit. I'm not sure if there is a reason why\n> this limit is different from the _other_ limits I discovered,\n> but it might still be good to reduce magic literal ints by\n> grouping this \"1000\" into a const or macro.)\n\nI'll discuss the value of 1000 later...\n\n> ...and this just assigns the default again. Why is this done\n> here instead of inside the diff machinery? Also, wouldn't a\n> diff.renameLimit = 0 imply no renames, not \"use default\"?\n\nYes, I totally agree that would make more sense, but backward\ncompatibility sometimes requires violating common sense. See commit\n89973554b5 (\"diffcore-rename: make diff-tree -l0 mean -l\",\n2017-11-29). For the same reasons discussed in that commit, I'm\nhesitant to change what is used here; it's a backward compatibility\nconcern now.\n\nOne reason opt->rename_limit could be 0 is if some caller does the following:\n\n merge_options opt;\n memset(&opt, 0, sizeof(opt));\n opt.ancestor = ....;\n /* forget to set opt.rename_limit */\n merge_incore_nonrecursive(&opt, ...);\n\nThe most likely reason for a negative value is probably that\ninit_merge_options() in merge-recursive.c set opt->rename_limit to -1.\nHaving init_merge_options() set the value to the actual default\nprobably would have made more sense, but the\nassign-it-to-negative-one-and-deal-with-it-later goes back to the\nintroduction of init_merge_options() in 2008. Actually, if you ignore\ninit_merge_options() the same thing was being done before back in 2007\nas soon as any limit handling was introduced to the code.\n\nSince init_merge_options() is shared for now between merge-recurisve.c\nand merge-ort.c, any updates I make here would necessitate similar\ncode updates to merge-recursive.c.\n\nAlso, it's not just internal code callers. Someone could set\nmerge.renameLimit or diff.renameLimit in their repository (or their\nglobal .gitconfig) to a non-positive value and get this behavior of\ntreat-non-positive-as-whatever-the-default-is.\n\n> I notice that the docs don't make this explicit:\n>\n> diff.renameLimit::\n> The number of files to consider when performing the copy/rename\n> detection; equivalent to the 'git diff' option `-l`. This setting\n> has no effect if rename detection is turned off.\n\nSee also https://lore.kernel.org/git/20180426162339.db6b4855fedb5e5244ba7dd1@google.com/\nwhere we talked about documenting the special value of 0 (in that case\nfor diff -l, though merge.renameLimit should have one too), but we\nobviously never got around to it. Yet. (I did at least put it on my\nprojects list, though things sometimes languish there for years.)\n\n> but also too_many_rename_candidates() has this strange\n> default check:\n>\n> /*\n> * This basically does a test for the rename matrix not\n> * growing larger than a \"rename_limit\" square matrix, ie:\n> *\n> * num_create * num_src > rename_limit * rename_limit\n> */\n> if (rename_limit <= 0)\n> rename_limit = 32767;\n>\n> this is... a much larger limit than I would think is reasonable.\n\nThe value of 32767 came from backward compatibility and in particular\nfrom the exact same commit referenced above -- 89973554b5\n(\"diffcore-rename: make diff-tree -l0 mean -l\", 2017-11-29).\n\nAlso, perhaps this value is *smaller* than reasonable -- I've used\nvalues like 48941 before on real world repositories. (And I'm not\npicking a random large value to report; *that* exact value came up\nenough times that I remember that particular one.) If 0 (or negative)\nis supposed to mean \"large\", then shouldn't it handle values people\nuse on real world repositories? (Not that I care that much, because I\nthink the usage of 0 to mean \"large\" is kind of illogical, so I'll\navoid it and discourage others from using it.)\n\nI do know where the 32767 came from, though. Once upon a time, 32767\nwas \"the biggest supported value possible\" and in fact any other\nnumber was silently capped to 32767. This of course led to a number\nof issues. See commit 9f7e4bfa3b (\"diff: remove silent clamp of\nrenameLimit\", 2017-11-13) and perhaps also commits b520abf1c8\n(\"sequencer: warn when internal merge may be suboptimal due to\nrenameLimit\", 2017-11-13) and d6861d0258 (\"progress: fix progress\nmeters when dealing with lots of work\", 2017-11-13).\n\n> Of course, diff_rename_limit_default is set to 400 inside diff.c.\n> Should that be extracted as a constant so we can repeat it here?\n\nI think it makes sense to have merge have a higher default rename\nlimit than diffs. I can see folks just doing a \"git log -p\" and not\nwanting individual commits to take a long time, especially since it's\nnot at all clear that most the commits are going to be of interest to\nthe user. In contrast, when merging, the commits are definitely of\ninterest to the user, and spending a little more time on a few commits\nprovides a nice payoff.\n\nAlso, merges provide progress meters on rename detection; I don't\nthink that log -p does. I think that the presence of progress meters\nmakes it easier to deal with larger values as well.\n\nIt may also be worth noting that both of these numbers were modified\nin the same commit in the past and retained distinct values; see\ncommit 92c57e5c1d (\"bump rename limit defaults (again)\", 2011-02-19).\n\nAfter all my rename optimizations, all those cases that used to\nrequire limits in the 20k ~ 50k range can now all complete with a\nlimit under 1000, and quite rapidly. (It was really hard to get one\nof them under 1000, though. It stubbornly required a value of 1002\nuntil I figured out another optimization allowing me to avoid\ndetecting more renames without any change in behavior.) It's nice\nthat it's fast, and it's also nice that rename detection just works\ninstead of having the merge throw a warning that the limit was too\nlow, doing the merge all wrong, and expecting the user to undo the\nmerge, set the limit higher, and redo it.\n\n400 definitely isn't high enough. I'm actually tempted to double the\n1000 to buy more room. Since the last bump was about a decade ago and\nnoted that processors had gotten faster, since the bump before it\nperhaps it is time to bump it again.\n\nAll that said, it could possibly make sense to define 1000 as a\nspecial constant near the top of the file and then use it via whatever\nmacro/constant/variable name we give it. Such a change would make it\nharder to compare this patch to get_diffpairs() in merge-recursive.c,\nthough...\n\n> > + diff_opts.rename_score = opt->rename_score;\n> > + diff_opts.show_rename_progress = opt->show_rename_progress;\n> > + diff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n> > + diff_setup_done(&diff_opts);\n> > + diff_tree_oid(&merge_base->object.oid, &side->object.oid, \"\",\n> > + &diff_opts);\n> > + diffcore_std(&diff_opts);\n> > +\n> > + if (diff_opts.needed_rename_limit > opt->priv->renames->needed_limit)\n> > + opt->priv->renames->needed_limit = diff_opts.needed_rename_limit;\n> > +\n> > + renames->pairs[side_index] = diff_queued_diff;\n> > +\n> > + diff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n> > + diff_queued_diff.nr = 0;\n> > + diff_queued_diff.queue = NULL;\n> > + diff_flush(&diff_opts);\n> > }\n> >\n> > /*\n> > @@ -1379,6 +1405,10 @@ void merge_switch_to_result(struct merge_options *opt,\n> > printf(\"%s\", sb->buf);\n> > }\n> > string_list_clear(&olist, 0);\n> > +\n> > + /* Also include needed rename limit adjustment now */\n> > + diff_warn_rename_limit(\"merge.renamelimit\",\n> > + opti->renames->needed_limit, 0);\n>\n> I suppose this new call is appropriate in this patch, since you assign\n> the value inside detect_regular_renames(), but it might be good to\n> describe its presence in the commit message.\n\nSure, I can add a note.\n"},{"id":"412093","messageId":"CABPp-BErHFVQ0Omrnn7c=x+apwXmd=T8=xdPTkypb5DOJPWNuw@mail.gmail.com","threadId":"54806","inReplyTo":"8d744966-2344-1dbc-f2a2-7411be3bfc8b@gmail.com","subject":"Re: [PATCH 04/11] merge-ort: implement compare_pairs() and collect_renames()","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-11T18:43:21Z","receivedAt":"2020-12-11T20:13:14Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Dec 10, 2020 at 7:00 PM Derrick Stolee wrote:\n>\n> On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n>\n> Perhaps worth pointing out comparison to score_compare()\n\nThat comparison might be slightly misleading due to this line from\ncollect_renames():\n+ p->score = side_index;\nSince diffcore-rename has already used the percentage similarity to\ndetermine if two files are a rename and has recorded that in a\nseparate field, I don't need the percentage similarity anymore. So\nthe score is no longer needful at this point. However, I needed a way\nto somehow record which side of the merge each diff_filepair came from\nand I can't just add a field to the diff_filepair struct (especially\nsince its only use is in merge-ort). I know, I know, I'm evil.\nCreating a new struct just so I could have something that contained a\ndiff_filepair and another auxiliary field just felt so ugly, so I just\nreused this one field instead. And I did use that field to \"rank\" or\n\"sort\" the pairs, so doesn't that make it a valid \"score\"? :-)\n\nI should probably add a big comment about this just above that line.\nI've meant to do that a multiple different times, but oddly enough\nthis thought has only occurred to me while I'm out running or\notherwise away from the computer. Until now, of course.\n\n> > Signed-off-by: Elijah Newren \n> > ---\n> > merge-ort.c | 27 +++++++++++++++++++++++++--\n> > 1 file changed, 25 insertions(+), 2 deletions(-)\n> >\n> > diff --git a/merge-ort.c b/merge-ort.c\n> > index 1ff637e57af..3cdf8124b85 100644\n> > --- a/merge-ort.c\n> > +++ b/merge-ort.c\n> > @@ -625,7 +625,13 @@ static int process_renames(struct merge_options *opt,\n> >\n> > static int compare_pairs(const void *a_, const void *b_)\n> > {\n> > - die(\"Not yet implemented.\");\n> > + const struct diff_filepair *a = *((const struct diff_filepair **)a_);\n> > + const struct diff_filepair *b = *((const struct diff_filepair **)b_);\n> > +\n> > + int cmp = strcmp(a->one->path, b->one->path);\n> > + if (cmp)\n> > + return cmp;\n> > + return a->score - b->score;\n>\n> Hm. I wasn't sure what would happen when subtracting these\n> \"unsigned short\" scores, but I see that score_compare() does\n> the same. Any potential for an existing, hidden bug here?\n\nIn the case of compare_pairs(), a->score and b->score have a minimum\nvalue of 1 and a maximum value of 2 (note above where I set score to\nthe side index). I believe most platforms will have an int big enough\nto store the result of that subtraction.\n\nI'm not sure why I bother with the secondary sort, though. It shouldn't matter.\n\nWhich is probably a good thing, because strcmp(a, b) gives us\nascending order, and a - b gives us descending order. That's messed\nup. Honestly, it doesn't matter because all I really needed from the\nsort was for diff_filepairs with the same source name to be adjacent\n(so that I can check for rename/rename(1to2) conflicts be comparing\nadjacent pairs), but still it's annoying that the function contradicts\nitself on the desired order. And it'll trigger whenever the same path\nis renamed by both sides of history, which we have a number of\ntestcases for in the testsuite. So that confirms that the secondary\nsort just doesn't matter. I'll get rid of it and just use strcmp.\n"},{"id":"412094","messageId":"CABPp-BHoYFnJLNr6g11QBVcdMNBjJYOQoXPz31+gCrYhwnK01g@mail.gmail.com","threadId":"54806","inReplyTo":"3691e3f1-8840-9b19-5fff-02fcb846de70@gmail.com","subject":"Re: [PATCH 05/11] merge-ort: add basic outline for process_renames()","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-11T20:03:40Z","receivedAt":"2020-12-11T21:21:19Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Dec 10, 2020 at 7:24 PM Derrick Stolee wrote:\n>\n> On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > Add code which determines which kind of special rename case each rename\n> > corresponds to, but leave the handling of each type unimplemented for\n> > now. Future commits will implement each one.\n> >\n> > There is some tenuous resemblance to merge-recursive's\n> > process_renames(), but comparing the two is very unlikely to yield any\n> > insights. merge-ort's process_renames() is a bit complex and I would\n> > prefer if I could simplify it more, but it is far easier to grok than\n> > merge-recursive's function of the same name in my opinion. Plus,\n> > merge-ort handles more rename conflict types than merge-recursive does.\n> >\n> > Signed-off-by: Elijah Newren \n> > ---\n> > merge-ort.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++-\n> > 1 file changed, 97 insertions(+), 1 deletion(-)\n> >\n> > diff --git a/merge-ort.c b/merge-ort.c\n> > index 3cdf8124b85..faec29db955 100644\n> > --- a/merge-ort.c\n> > +++ b/merge-ort.c\n> > @@ -620,7 +620,103 @@ static int handle_content_merge(struct merge_options *opt,\n> > static int process_renames(struct merge_options *opt,\n> > struct diff_queue_struct *renames)\n> > {\n> > - die(\"Not yet implemented.\");\n> > + int clean_merge = 1, i;\n> > +\n> > + for (i = 0; i < renames->nr; ++i) {\n> > + const char *oldpath = NULL, *newpath;\n>\n> This \"= NULL\" is not necessary, since you initialize it to\n> old_ent->key unconditionally.\n>\n> > + struct diff_filepair *pair = renames->queue[i];\n> > + struct conflict_info *oldinfo = NULL, *newinfo = NULL;\n>\n> These, too.\n\nOh, man, so many frustrations here. You are, of course correct. The\nreason it's here, though...\n\nThe code I took this from is a bit more complex due to directory\nrenames, cached renames from previous steps (think of rebases or\ncherry-picks, where there was a rename from the old base to the new\nbase), and trivial directory resolutions.\n\nIn the more complex code, the initializations aren't needed either;\nthe variables are never used uninitialized.\n\nBUT certain versions of gcc don't recognize that they are never used\nuninitialized and throw errors saying they might be. Newer gcc\nversions recognize that everything is kosher and compiles fine, but\nIIRC, the CentOS 7 version of gcc does not. I want the code to\ncompile there under DEVELOPER=1 too, and I couldn't find an easy way\nto restructure to make it clearer to the compiler. So I was forced to\nadd these useless initializations.\n\n...and I just didn't think to rip them out for this preliminary patch.\n\n\nI can rip them out if you want, but I'll be forced to add them back\nlater -- in patches where code flow analysis will also suggest it\nisn't needed.\n\n> > + struct strmap_entry *old_ent, *new_ent;\n> > + unsigned int old_sidemask;\n> > + int target_index, other_source_index;\n> > + int source_deleted, collision, type_changed;\n> > +\n> > + old_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n> > + oldpath = old_ent->key;\n> > + oldinfo = old_ent->value;\n> > +\n> > + new_ent = strmap_get_entry(&opt->priv->paths, pair->two->path);\n> > + newpath = new_ent->key;\n> > + newinfo = new_ent->value;\n>\n> This is moving data around. I wonder if there is any possibility that\n> old_ent or new_ent could be NULL here, and we should check for that?\n> (The \"any possibility\" is probably \"is there a chance of a bug in the\n> earlier logic that didn't cause a failure yet, but would cause a SEGFAULT\n> here?\".)\n\nGood question. The only chance at this point in the code of this\nhappening is if someone has introduced a severe bug in the code\nelsewhere. Any paths that show up in rename detection have to come\nfrom the trees, and in collect_merge_info() we walk over the full\ntrees and store every path from any of the trees in opt->priv->paths.\n\nOnce directory rename detection, caching of renames, and trivial\ndirectory resolution are added to the mix, it suddenly becomes\npossible for these to be NULL. So, the version on 'ort' does have\nchecks. This just represents me trying to find chunks of the code\nthat can be submitted upstream in a fashion where each piece makes\nsense.\n\n> > + /*\n> > + * diff_filepairs have copies of pathnames, thus we have to\n> > + * use standard 'strcmp()' (negated) instead of '=='.\n> > + */\n> > + if (i+1 < renames->nr &&\n>\n> nit: I tend to prefer \"i + 1\".\n\nYou might have to keep reminding me; I think I've got more of these in\nvarious places. I'll fix it up, though.\n\n> > + !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n> > + /* Handle rename/rename(1to2) or rename/rename(1to1) */\n> > + const char *pathnames[3];\n> > +\n> > + pathnames[0] = oldpath;\n> > + pathnames[1] = newpath;\n> > + pathnames[2] = renames->queue[i+1]->two->path;\n> > +\n> > + if (!strcmp(pathnames[1], pathnames[2])) {\n> > + /* Both sides renamed the same way. */\n> > + die(\"Not yet implemented\");\n> > +\n> > + /* We handled both renames, i.e. i+1 handled */\n> > + i++;\n> > + /* Move to next rename */\n> > + continue;\n> > + }\n> > +\n> > + /* This is a rename/rename(1to2) */\n> > + die(\"Not yet implemented\");\n>\n> Interesting that you chose to do some internal logic to split this\n> case, but have both die(). Perhaps that is wise, but also this could\n> have been a die() at the start, along with the pathnames[] initialization\n> in a later patch that implements the 1to1 case (leaving the 1to2 case\n> to die()).\n\nYeah, that'd also be a fair way to split up the patches. Doing this\nthe way I did above allowed me to separate the conflict detection and\nconflict handling patches -- later patches in the series were allowed\nto focus on \"this is how rename/delete is handled\", \"this is how\nrename/rename(1to2) conflicts are handled\", etc. Your split would\nmean combining the detection and handling logic for at least one of\nthe conflict types into the same patch. That'd still work fine, but\nit just wasn't what I came up with due to my focus on the\ndetection/handling split.\n\n> > + i++; /* We handled both renames, i.e. i+1 handled */\n> > + continue;\n> > + }\n> > +\n> > + VERIFY_CI(oldinfo);\n> > + VERIFY_CI(newinfo);\n> > + target_index = pair->score; /* from append_rename_pairs() */\n>\n> Hm. I don't see append_rename_pairs() anywhere else in the codebase.\n\nOh, whoops, that function doesn't even exist in the 'ort' branch\nanymore either. pair->score was set in collect_renames(), in the\nprevious patch. It was set to the side index. Looks like I at least\ndocumented it once on one side of its usage.\n\n> Do you mean record_rename_pair()? But in that case, I don't understand\n> the following assertion:\n>\n> > + assert(target_index == 1 || target_index == 2)>\n\nSee collect_renames() setting of p->score to side_index.\n\n> > + other_source_index = 3-target_index;\n>\n> nit: \"3 - target_index\"\n\nYep, like I said, you'll have to keep reminding me. I will fix this up.\n\n> > + old_sidemask = (1 << other_source_index); /* 2 or 4 */\n> > + source_deleted = (oldinfo->filemask == 1);\n>\n> This oldinfo->filemask check made me go to the declaration to find\n>\n> /*\n> * For filemask and dirmask, see tree-walk.h's struct traverse_info,\n> * particularly the documentation above the \"fn\" member. Note that\n> * filemask = mask & ~dirmask from that documentation.\n> */\n> unsigned filemask:3;\n> unsigned dirmask:3;\n>\n> Perhaps I've missed my window to complain about this comment pointing to\n> a comment in another struct definition instead of something like:\n\nIt would be nicer placed in as a comment on patch 1 of\nen/merge-ort-impl[1]. But it's definitely not too late -- that series\nis still in 'seen', you reviewed an earlier round of it[2] (sadly ALSO\nlabelled as v2), and it was mostly waiting for you and Jonathan to\ngive a thumbs up on whether you were happy with the changes I made to\nthe series[3]. Feel free to say that my changes to that series looks\nokay, except that I need to update the description for filemask and\ndirmask. :-)\n\n[1] https://lore.kernel.org/git/2568ec92c6d96dc51aff4a411900eaec8d32ce27.1607114890.git.gitgitgadget@gmail.com/\n[2] https://lore.kernel.org/git/75170ee7-525e-31fc-f6bd-6dfac12b00c8@gmail.com/\n[3] See final portion of\nhttps://lore.kernel.org/git/CABPp-BGcyRURykePOafjcE1z9J8U5awF=PZw1ufx+8Ow+k3j3w@mail.gmail.com/\n\n> /*\n> * The ith bit corresponds to whether the ith entry is a file\n> * (filemask) or a directory (dirmask). Thus, filemask & dirmask\n> * is always zero and filemask | dirmask == 7 always.\n> */\n\nThe first part of this comment looks great. The last part is false,\nthough -- the ith entry might not be either a file or a directory.\nFor example, if the merge base had a file that both sides deleted,\nfilemask == 1 and dirmask == 0.\n\nI'd be happy to use your wording before the final \"and filemask |\ndirmask == 7 always\" bit, but I think it'd be nice to also keep a \"see\nalso tree-walk.h...\" comment,\n\n> And of course, looking at this struct provides the justification for\n> using \"1\" and \"2\" for the \"sides\" and wasting the 0th value, because\n> it is consistent with the three entries here, using 0 as the base.\n\nYaay, looks like I didn't need to convince you after all! At least\nnot on that point...\n\n> Thus, the comment about not using the 0 position could use a reference\n> to these triples. I still think that using an enum would help here.\n\nI'm genuinely curious about your thoughts on the various other\nquestions I raised about that point in the earlier patches to this\nseries and your thoughts on my other suggestion there.\n\n> Coming back to the line in question, \"filemask == 1\" _does_ mean that\n> the file exists only in the base. Deleted indeed.\n\nThere's going to be a lot more of these later, especially in the basic\nconflict handling series... :-)\n\n> > + collision = ((newinfo->filemask & old_sidemask) != 0);\n> > + type_changed = !source_deleted &&\n> > + (S_ISREG(oldinfo->stages[other_source_index].mode) !=\n> > + S_ISREG(newinfo->stages[target_index].mode));\n> > + if (type_changed && collision) {\n> > + /* special handling so later blocks can handle this */\n> > + die(\"Not yet implemented\");\n> > + }\n> > +\n> > + assert(source_deleted || oldinfo->filemask & old_sidemask);\n> > +\n> > + /* Need to check for special types of rename conflicts... */\n> > + if (collision && !source_deleted) {\n> > + /* collision: rename/add or rename/rename(2to1) */\n> > + die(\"Not yet implemented\");\n> > + } else if (collision && source_deleted) {\n> > + /* rename/add/delete or rename/rename(2to1)/delete */\n>\n> How did we get to three actions here? I'll probably learn when\n> it is implemented.\n\nMultiple actions that appear unrelated on one side suddenly get glued\ntogether after the other side's rename detection.\n\nrename/add/delete comes about like so:\n\nBase version: file A exists\nSide 1: deletes A, adds unrelated B\nSide 2: renames A -> B\n\nrename/rename(2to1)/delete comes about as follows:\n\nBase version: file A and file B both exist.\nSide 1: delete A, rename B->C\nSide 2, rename A->C\n\n(You could also have rename/rename(2to1)/delete/delete if side 2 also\ndeleted B, but it doesn't present any additional complications for the\ncode.)\n\nYou can find all kinds of crazy rename cases in t6422 and t6416.\n\"mod6\" is fun. Perhaps I should add a comment somewhere referencing\nthese testcases?\n\n\n...and if you're really curious...\n\nAll of these conflict types were much worse with merge-recursive.c,\nbecause sometimes each code path had to consider the combination of\nall possible conflicts. Thus if you were ever worried about a\nrename/rename(1to2)/content conflict/file location/(D/F)/(D/F)/\nappearing[4], there might need to be a single specific codepath that\nhandled all of those simultaneously. merge-ort makes these more\northogonal. For example, one place in merge-ort handles all\ndirectory/file conflicts, regardless of what other conflicts they are\npart of, whereas in merge-recursive there was directory/file conflict\nhandling code shotgun blasted everywhere and probably missing from\nseveral specific conflict types.\n\n[4] You can read this conflict as: both sides modify a file in\nconflicting way (\"content conflict\"), both rename that file but to\ndifferent paths (\"rename/rename(1to2)\"), one side renames the\ndirectory which the other side had renamed that file into causing it\nto possibly need a transitive rename (\"file location\"; for example, if\none side renamed A -> B/C, and the other side renamed B/ -> Beta/),\nand each side puts a directory in the way of the other's path (each of\nthe \"D/F\", e.g. if the side that renamed B/ -> Beta/ also added a\nBeta/C/ directory to be in the way of A getting renamed to end up\nthere).)\n\n> > + die(\"Not yet implemented\");\n> > + } else {\n> > + /* a few different cases... */\n> > + if (type_changed) {\n> > + /* rename vs. typechange */\n> > + die(\"Not yet implemented\");\n> > + } else if (source_deleted) {\n> > + /* rename/delete */\n> > + die(\"Not yet implemented\");\n> > + } else {\n> > + /* normal rename */\n> > + die(\"Not yet implemented\");\n> > + }\n> > + }\n> > +\n> > + if (!type_changed) {\n> > + /* Mark the original as resolved by removal */\n> > + oldinfo->merged.is_null = 1;\n> > + oldinfo->merged.clean = 1;\n> > + }\n> > +\n> > + }\n> > +\n> > + return clean_merge;\n>\n> I'm glad you separated out this case organization from the implementations.\n> It's still a big dense, so I'll probably need to revisit as I see you fill\n> in the rest.\n>\n> Thanks,\n> -Stolee\n"},{"id":"412098","messageId":"CABPp-BE8Pjv6cPKBggWWyDutyb4686vXgJy4P-JqoH03KJzGdQ@mail.gmail.com","threadId":"54806","inReplyTo":"5b9f5db0-b4d5-f60d-b5c5-2f8376d4bf12@gmail.com","subject":"Re: [PATCH 07/11] merge-ort: add implementation of both sides renaming differently","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-11T21:56:55Z","receivedAt":"2020-12-11T22:54:01Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Dec 10, 2020 at 7:39 PM Derrick Stolee wrote:\n>\n> On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > Implement rename/rename(1to2) handling, i.e. both sides of history\n> > renaming a file and rename it differently. This code replaces the\n> > following from merge-recurisve.c:\n> >\n> > * all the 1to2 code in process_renames()\n> > * the RENAME_ONE_FILE_TO_TWO case of process_entry()\n> > * handle_rename_rename_1to2()\n> >\n> > Also, there is some shared code from merge-recursive.c for multiple\n> > different rename cases which we will no longer need for this case (or\n> > other rename cases):\n> >\n> > * handle_file_collision()\n> > * setup_rename_conflict_info()\n> >\n> > The consolidation of five separate codepaths into one is made possible\n> > by a change in design:\n>\n> Excellent!\n>\n> > /* This is a rename/rename(1to2) */\n> > - die(\"Not yet implemented\");\n> > + clean_merge = handle_content_merge(opt,\n> > + pair->one->path,\n> > + &base->stages[0],\n> > + &side1->stages[1],\n> > + &side2->stages[2],\n> > + pathnames,\n> > + 1 + 2 * opt->priv->call_depth,\n> > + &merged);\n>\n> (this method currently die()s. ok)\n>\n> > + if (!clean_merge &&\n> > + merged.mode == side1->stages[1].mode &&\n> > + oideq(&merged.oid, &side1->stages[1].oid)) {\n> > + was_binary_blob = 1;\n> > + }\n>\n> nit: Extraneous braces?\n\nYeah.\n\n> > + memcpy(&side1->stages[1], &merged, sizeof(merged));\n> > + if (was_binary_blob) {\n> > + /*\n> > + * Getting here means we were attempting to\n> > + * merge a binary blob.\n> > + *\n> > + * Since we can't merge binaries,\n> > + * handle_content_merge() just takes one\n> > + * side. But we don't want to copy the\n> > + * contents of one side to both paths. We\n> > + * used the contents of side1 above for\n> > + * side1->stages, let's use the contents of\n> > + * side2 for side2->stages below.\n> > + */\n> > + oidcpy(&merged.oid, &side2->stages[2].oid);\n> > + merged.mode = side2->stages[2].mode;\n> > + }\n> > + memcpy(&side2->stages[2], &merged, sizeof(merged));\n> > +\n> > + side1->path_conflict = 1;\n> > + side2->path_conflict = 1;\n> > + /*\n> > + * TODO: For renames we normally remove the path at the\n> > + * old name. It would thus seem consistent to do the\n> > + * same for rename/rename(1to2) cases, but we haven't\n> > + * done so traditionally and a number of the regression\n> > + * tests now encode an expectation that the file is\n> > + * left there at stage 1. If we ever decide to change\n> > + * this, add the following two lines here:\n> > + * base->merged.is_null = 1;\n> > + * base->merged.clean = 1;\n> > + * and remove the setting of base->path_conflict to 1.\n> > + */\n> > + base->path_conflict = 1;\n>\n> I'm getting the point of the review/evening where I'm starting to gloss\n> over these important details. Time to take a break (after this patch).\n\nI'm surprised you didn't take a break after giving a talk at GitHub\nUniverse earlier yesterday. I don't think I got anything done the\nrest of the day after I gave my talk at GitMerge 2020. Nice job on\nyour talk, by the way; I'm sending it along to some others to watch.\n\n> > + path_msg(opt, oldpath, 0,\n> > + _(\"CONFLICT (rename/rename): %s renamed to \"\n> > + \"%s in %s and to %s in %s.\"),\n> > + pathnames[0],\n> > + pathnames[1], opt->branch1,\n> > + pathnames[2], opt->branch2);\n>\n> This output differs a bit from handle_rename_rename_1to2() in\n> merge-recursive.c:\n>\n> output(opt, 1, _(\"CONFLICT (rename/rename): \"\n> \"Rename \\\"%s\\\"->\\\"%s\\\" in branch \\\"%s\\\" \"\n> \"rename \\\"%s\\\"->\\\"%s\\\" in \\\"%s\\\"%s\"),\n> o->path, a->path, ci->ren1->branch,\n> o->path, b->path, ci->ren2->branch,\n> opt->priv->call_depth ? _(\" (left unresolved)\") : \"\");\n>\n> How much do we want to have _exact_ output matches between the\n> two strategies, at least in the short term?\n\nGood question. I started with such a goal, and discarded it when I\ndiscovered it was unrealistic and even actively harmful -- at least in\nthe extreme of exact matching in all cases. I've already updated the\nregression tests to expect differences in output and behavior in a few\ndifferent series that merged down months ago[1][2][3].\n [1] https://lore.kernel.org/git/pull.827.v3.git.git.1597098559.gitgitgadget@gmail.com/\n [2] https://lore.kernel.org/git/pull.769.v2.git.1603731704.gitgitgadget@gmail.com/\n [3] https://lore.kernel.org/git/pull.879.git.git.1602794790.gitgitgadget@gmail.com/\n\n Of particular note from those series, at least as far as output\nmessages, are the following commits:\n 1f3c9ba707 (\"t6425: be more flexible with rename/delete conflict\nmessages\", 2020-08-10)\n 2a7c16c980 (\"t6422, t6426: be more flexible for add/add conflicts\ninvolving renames\", 2020-08-10)\n c8c35f6a02 (\"merge tests: expect slight differences in output for\nrecursive vs. ort\", 2020-10-26)\n\nSummarizing those, I split some conflict messages into multiple\nmessages, changed the name of some conflict types that get reported,\navoided having some messages go to stdout while others go to stderr\n(based on who added them rather than differences in severity of the\nmessage), and even just changed some messages (because once you've\naccepted the other changes, exact matching just doesn't matter).\nOther commits in the series also note how I changed the merge behavior\nin addition to the output in various cases.\n\nThe rename/delete conflict commit message is particularly\nillustrative. It demonstrates why exact matching of output messages\nis unachievable short of keeping a completely busted code design.\n\nWe can certainly tweak individual messages if we feel something will\nmake them clearer, but matching isn't a goal for me.\n\n> > @@ -1257,13 +1309,13 @@ static void process_entry(struct merge_options *opt,\n> > int side = (ci->filemask == 4) ? 2 : 1;\n> > ci->merged.result.mode = ci->stages[side].mode;\n> > oidcpy(&ci->merged.result.oid, &ci->stages[side].oid);\n> > - ci->merged.clean = !ci->df_conflict;\n> > + ci->merged.clean = !ci->df_conflict && !ci->path_conflict;\n> > } else if (ci->filemask == 1) {\n> > /* Deleted on both sides */\n> > ci->merged.is_null = 1;\n> > ci->merged.result.mode = 0;\n> > oidcpy(&ci->merged.result.oid, &null_oid);\n> > - ci->merged.clean = 1;\n> > + ci->merged.clean = !ci->path_conflict;\n>\n> These exist because this is the first time we assign path_conflict.\n> Sure.\n>\n> Thanks,\n> -Stolee\n\n\nThanks for all your detailed reviews!\n"},{"id":"412131","messageId":"CABPp-BHa0zehQd-axmb4bF6fR4PTWwGu9uLjOzgTW8_Gu12iZA@mail.gmail.com","threadId":"54806","inReplyTo":"84a4d97b-8496-4c83-5d32-19f57e17a871@gmail.com","subject":"Re: [PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-13T07:47:06Z","receivedAt":"2020-12-13T07:48:20Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nSorry for two different email responses to the same email...\n\nAddressing the comments on this patchset mean re-submitting\nen/merge-ort-impl, and causing conflicts in en/merge-ort-2 and this\nseries en/merge-ort-3. Since gitgitgadget will not allow me to submit\npatches against a series that isn't published by Junio, I'll need to\nask Junio to temporarily drop both of these series, then later\nresubmit en/merge-ort-2 after he publishes my updates to\nen/merge-ort-impl. Then when he publishes my updates to\nen/merge-ort-2, I'll be able to submit my already-rebased patches for\nen/merge-ort-3.\n\nA couple extra comments below...\n\nOn Thu, Dec 10, 2020 at 6:39 PM Derrick Stolee wrote:\n>\n> On 12/9/2020 2:41 PM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > Signed-off-by: Elijah Newren \n> > ---\n> > merge-ort.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++-------\n> > 1 file changed, 60 insertions(+), 8 deletions(-)\n> >\n> > diff --git a/merge-ort.c b/merge-ort.c\n> > index 90baedac407..92b765dd3f0 100644\n> > --- a/merge-ort.c\n> > +++ b/merge-ort.c\n> > @@ -617,20 +617,72 @@ static int handle_content_merge(struct merge_options *opt,\n> >\n> > /*** Function Grouping: functions related to regular rename detection ***/\n> >\n> > +static int process_renames(struct merge_options *opt,\n> > + struct diff_queue_struct *renames)\n> > +static int compare_pairs(const void *a_, const void *b_)\n> > +/* Call diffcore_rename() to compute which files have changed on given side */\n> > +static void detect_regular_renames(struct merge_options *opt,\n> > + struct tree *merge_base,\n> > + struct tree *side,\n> > + unsigned side_index)\n> > +static int collect_renames(struct merge_options *opt,\n> > + struct diff_queue_struct *result,\n> > + unsigned side_index)\n>\n> standard \"I promise this will follow soon!\" strategy, OK.\n>\n> > static int detect_and_process_renames(struct merge_options *opt,\n> > struct tree *merge_base,\n> > struct tree *side1,\n> > struct tree *side2)\n> > {\n> > - int clean = 1;\n> > + struct diff_queue_struct combined;\n> > + struct rename_info *renames = opt->priv->renames;\n>\n> (Re: my concerns that we don't need 'renames' to be a pointer,\n> this could easily be \"renames = &opt->priv.renames;\")\n>\n> > + int s, clean = 1;\n> > +\n> > + memset(&combined, 0, sizeof(combined));\n> > +\n> > + detect_regular_renames(opt, merge_base, side1, 1);\n> > + detect_regular_renames(opt, merge_base, side2, 2);\n>\n> Find the renames in each side's diff.\n>\n> I think the use of \"1\" and \"2\" here might be better situated\n> for an enum. Perhaps:\n>\n> enum merge_side {\n> MERGE_SIDE1 = 0,\n> MERGE_SIDE2 = 1,\n> };\n>\n> (Note, I shift these values to 0 and 1, respectively, allowing\n> us to truncate the pairs array to two entries while still\n> being mentally clear.)\n\nSo, after mulling it over for a while, I created a\n\nenum merge_side {\n MERGE_BASE = 0,\n MERGE_SIDE1 = 1,\n MERGE_SIDE2 = 2\n};\n\nand I made use of it in several places. I just avoided going to an\nextreme with it (e.g. adding another enum for masks or changing all\npossibly relevant variables from ints to enum merge_side), and used it\nmore as a document-when-values-are-meant-to-refer-to-sides-of-the-merge\nkind of thing. Of course, this affects two previous patchsets and not\njust this one, so I'll have to post a _lot_ of new patches... :-)\n\n> > +\n> > + ALLOC_GROW(combined.queue,\n> > + renames->pairs[1].nr + renames->pairs[2].nr,\n> > + combined.alloc);\n> > + clean &= collect_renames(opt, &combined, 1);\n> > + clean &= collect_renames(opt, &combined, 2);\n>\n> Magic numbers again.\n>\n> > + QSORT(combined.queue, combined.nr, compare_pairs);\n> > +\n> > + clean &= process_renames(opt, &combined);\n>\n> I need to mentally remember that \"clean\" is a return state,\n> but _not_ a fail/success result. Even though we are using\n> \"&=\" here, it shouldn't be \"&&=\" or even \"if (method()) return 1;\"\n>\n> Looking at how \"clean\" is used in struct merge_result, I\n> wonder if there is a reason to use an \"int\" over a simple\n> \"unsigned\" or even \"unsigned clean:1;\" You use -1 in places\n> as well as a case of \"mi->clean = !!resolved;\"\n\nSomething I missed in my reply yesterday...\n\nNote that mi->clean is NOT from struct merge_result. It is from\nstruct merged_info, and in that struct it IS defined as \"unsigned\nclean:1\", i.e. it is a true boolean. The merged_info.clean field is\nused to determine whether a specific path merged cleanly.\n\n\"clean\" from struct merge_result is whether the entirety of the merge\nwas clean or not. It's almost a boolean, but allows for a\n\"catastrophic problem encountered\" value. I added the following\ncomment:\n/*\n* Whether the merge is clean; possible values:\n* 1: clean\n* 0: not clean (merge conflicts)\n* <0: operation aborted prematurely. (object database\n* unreadable, disk full, etc.) Worktree may be left in an\n* inconsistent state if operation failed near the end.\n*/\n\nThis also means that I either abort and return a negative value, or I\ncan continue treating merge_result's \"clean\" field as a boolean.\n\nBut again, this isn't new to this patchset; it affects the patchset\nbefore the patchset before this one.\n\n> If there is more meaning to values other than \"clean\" or\n> \"!clean\", then an enum might be valuable.\n>\n> > + /* Free memory for renames->pairs[] and combined */\n> > + for (s = 1; s <= 2; s++) {\n> > + free(renames->pairs[s].queue);\n> > + DIFF_QUEUE_CLEAR(&renames->pairs[s]);\n> > + }\n>\n> This loop is particularly unusual. Perhaps it would be\n> better to do this instead:\n>\n> free(renames->pairs[MERGE_SIDE1].queue);\n> free(renames->pairs[MERGE_SIDE2].queue);\n> DIFF_QUEUE_CLEAR(&renames->pairs[MERGE_SIDE1]);\n> DIFF_QUEUE_CLEAR(&renames->pairs[MERGE_SIDE2]);\n>\n> > + if (combined.nr) {\n> > + int i;\n> > + for (i = 0; i < combined.nr; i++)\n> > + diff_free_filepair(combined.queue[i]);\n> > + free(combined.queue);\n> > + }\n> >\n> > - /*\n> > - * Rename detection works by detecting file similarity. Here we use\n> > - * a really easy-to-implement scheme: files are similar IFF they have\n> > - * the same filename. Therefore, by this scheme, there are no renames.\n> > - *\n> > - * TODO: Actually implement a real rename detection scheme.\n> > - */\n> > return clean;\n>\n> I notice that this change causes detect_and_process_renames() to\n> change from an \"unhelpful result, but success\" to \"die() always\".\n>\n> I wonder if there is value in swapping the order of the patches\n> to implement the static methods first. Of course, you hit the\n> \"unreferenced static method\" problem, so maybe your strategy is\n> better after all.\n>\n> Thanks,\n> -Stolee\n"},{"id":"412167","messageId":"7b0aafae-cd57-d4f7-ac85-238471428d24@gmail.com","threadId":"54806","inReplyTo":"CABPp-BHa0zehQd-axmb4bF6fR4PTWwGu9uLjOzgTW8_Gu12iZA@mail.gmail.com","subject":"Re: [PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-14T14:33:15Z","receivedAt":"2020-12-14T14:34:19Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/13/2020 2:47 AM, Elijah Newren wrote:\n> Hi,\n> \n> Sorry for two different email responses to the same email...\n> \n> Addressing the comments on this patchset mean re-submitting\n> en/merge-ort-impl, and causing conflicts in en/merge-ort-2 and this\n> series en/merge-ort-3. Since gitgitgadget will not allow me to submit\n> patches against a series that isn't published by Junio, I'll need to\n> ask Junio to temporarily drop both of these series, then later\n> resubmit en/merge-ort-2 after he publishes my updates to\n> en/merge-ort-impl. Then when he publishes my updates to\n> en/merge-ort-2, I'll be able to submit my already-rebased patches for\n> en/merge-ort-3.\n\nLet's chat privately about perhaps creatin\n \n> A couple extra comments below...\n\n\n>>> + int s, clean = 1;\n>>> +\n>>> + memset(&combined, 0, sizeof(combined));\n>>> +\n>>> + detect_regular_renames(opt, merge_base, side1, 1);\n>>> + detect_regular_renames(opt, merge_base, side2, 2);\n>>\n>> Find the renames in each side's diff.\n>>\n>> I think the use of \"1\" and \"2\" here might be better situated\n>> for an enum. Perhaps:\n>>\n>> enum merge_side {\n>> MERGE_SIDE1 = 0,\n>> MERGE_SIDE2 = 1,\n>> };\n>>\n>> (Note, I shift these values to 0 and 1, respectively, allowing\n>> us to truncate the pairs array to two entries while still\n>> being mentally clear.)\n> \n> So, after mulling it over for a while, I created a\n> \n> enum merge_side {\n> MERGE_BASE = 0,\n> MERGE_SIDE1 = 1,\n> MERGE_SIDE2 = 2\n> };\n> \n> and I made use of it in several places. I just avoided going to an\n> extreme with it (e.g. adding another enum for masks or changing all\n> possibly relevant variables from ints to enum merge_side), and used it\n> more as a document-when-values-are-meant-to-refer-to-sides-of-the-merge\n> kind of thing. Of course, this affects two previous patchsets and not\n> just this one, so I'll have to post a _lot_ of new patches... :-)\n\nI appreciate using names for the meaning behind a numerical constant.\nYou mentioned in the other thread that this will eventually expand to\na list of 10 entries, which is particularly frightening if we don't\nget some control over it now.\n\nI generally prefer using types to convey meaning as well, but I'm\nwilling to relax on this because I believe C won't complain if you\npass a literal int into an enum-typed parameter, so the compiler\ndoesn't help enough in that sense.\n\n> Something I missed in my reply yesterday...\n> \n> Note that mi->clean is NOT from struct merge_result. It is from\n> struct merged_info, and in that struct it IS defined as \"unsigned\n> clean:1\", i.e. it is a true boolean. The merged_info.clean field is\n> used to determine whether a specific path merged cleanly.\n> \n> \"clean\" from struct merge_result is whether the entirety of the merge\n> was clean or not. It's almost a boolean, but allows for a\n> \"catastrophic problem encountered\" value. I added the following\n> comment:\n> /*\n> * Whether the merge is clean; possible values:\n> * 1: clean\n> * 0: not clean (merge conflicts)\n> * <0: operation aborted prematurely. (object database\n> * unreadable, disk full, etc.) Worktree may be left in an\n> * inconsistent state if operation failed near the end.\n> */\n> \n> This also means that I either abort and return a negative value, or I\n> can continue treating merge_result's \"clean\" field as a boolean.\n\nHaving this comment helps a lot!\n \n> But again, this isn't new to this patchset; it affects the patchset\n> before the patchset before this one.\n\nRight, when I had the current change checked out, I don't see the\npatch that introduced the 'clean' member (though, I _could_ have\nblamed to find out). Instead, I just got confused and thought it\nworth a question. Your comment prevents this question in the future.\n\nThanks,\n-Stolee\n"},{"id":"412169","messageId":"nycvar.QRO.7.76.6.2012141641230.25979@tvgsbejvaqbjf.bet","threadId":"54806","inReplyTo":"7b0aafae-cd57-d4f7-ac85-238471428d24@gmail.com","subject":"Re: [PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-12-14T15:42:17Z","receivedAt":"2020-12-14T15:44:32Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Elijah & Stolee,\n\nOn Mon, 14 Dec 2020, Derrick Stolee wrote:\n\n> On 12/13/2020 2:47 AM, Elijah Newren wrote:\n> >\n> > Sorry for two different email responses to the same email...\n> >\n> > Addressing the comments on this patchset mean re-submitting\n> > en/merge-ort-impl, and causing conflicts in en/merge-ort-2 and this\n> > series en/merge-ort-3. Since gitgitgadget will not allow me to submit\n> > patches against a series that isn't published by Junio, I'll need to\n> > ask Junio to temporarily drop both of these series, then later\n> > resubmit en/merge-ort-2 after he publishes my updates to\n> > en/merge-ort-impl. Then when he publishes my updates to\n> > en/merge-ort-2, I'll be able to submit my already-rebased patches for\n> > en/merge-ort-3.\n>\n> Let's chat privately about perhaps creatin\n\nYes, I am totally willing to push up temporary branches if that helps you,\nor even giving you push permissions to do that.\n\nCiao,\nDscho\n"},{"id":"412171","messageId":"CABPp-BHyNuhNtc6JQgQRBsQFbobzYyS6fs=nopRuv8c1+9zW_w@mail.gmail.com","threadId":"54806","inReplyTo":"nycvar.QRO.7.76.6.2012141641230.25979@tvgsbejvaqbjf.bet","subject":"Re: [PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-14T16:11:03Z","receivedAt":"2020-12-14T16:12:11Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nOn Mon, Dec 14, 2020 at 7:42 AM Johannes Schindelin\n wrote:\n>\n> Hi Elijah & Stolee,\n>\n> On Mon, 14 Dec 2020, Derrick Stolee wrote:\n>\n> > On 12/13/2020 2:47 AM, Elijah Newren wrote:\n> > >\n> > > Sorry for two different email responses to the same email...\n> > >\n> > > Addressing the comments on this patchset mean re-submitting\n> > > en/merge-ort-impl, and causing conflicts in en/merge-ort-2 and this\n> > > series en/merge-ort-3. Since gitgitgadget will not allow me to submit\n> > > patches against a series that isn't published by Junio, I'll need to\n> > > ask Junio to temporarily drop both of these series, then later\n> > > resubmit en/merge-ort-2 after he publishes my updates to\n> > > en/merge-ort-impl. Then when he publishes my updates to\n> > > en/merge-ort-2, I'll be able to submit my already-rebased patches for\n> > > en/merge-ort-3.\n> >\n> > Let's chat privately about perhaps creatin\n>\n> Yes, I am totally willing to push up temporary branches if that helps you,\n> or even giving you push permissions to do that.\n>\n> Ciao,\n> Dscho\n\nGiven the amount of changes left to push up, I suspect there'll be\nmore cases where it'd be useful. If I could get push permissions, and\na suggested namespace to use for such temporary branches, that'd help.\n\nIn this particular case, though, one of my two fears was already\nrealized -- Junio jumped in and did the work of rebasing and conflict\nresolving for en/merge-ort-2 and en/merge-ort-3. I didn't want to\nburden him with that extra work, but what he pushed up for\nen/merge-ort-2 is identical to what I have. So, all I have to do is\npush en/merge-ort-3 with the extra changes I have in it. So this\nparticular time is taken care of.\n\nThanks!\n"},{"id":"412172","messageId":"78621ca07887e39a41ad9cff7b8dd5dcc9b33fbc.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 01/11] merge-ort: add basic data structures for handling renames","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:30Z","receivedAt":"2020-12-14T16:23:00Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nThis will grow later, but we only need a few fields for basic rename\nhandling.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 24 ++++++++++++++++++++++++\n 1 file changed, 24 insertions(+)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 414e7b7eeac..1c1a7fa4bf1 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -46,6 +46,25 @@ enum merge_side {\n \tMERGE_SIDE2 = 2\n };\n \n+struct rename_info {\n+\t/*\n+\t * pairs: pairing of filenames from diffcore_rename()\n+\t *\n+\t * Index 1 and 2 correspond to sides 1 & 2 as used in\n+\t * conflict_info.stages. Index 0 unused.\n+\t */\n+\tstruct diff_queue_struct pairs[3];\n+\n+\t/*\n+\t * needed_limit: value needed for inexact rename detection to run\n+\t *\n+\t * If the current rename limit wasn't high enough for inexact\n+\t * rename detection to run, this records the limit needed. Otherwise,\n+\t * this value remains 0.\n+\t */\n+\tint needed_limit;\n+};\n+\n struct merge_options_internal {\n \t/*\n \t * paths: primary data structure in all of merge ort.\n@@ -113,6 +132,11 @@ struct merge_options_internal {\n \t */\n \tstruct strmap output;\n \n+\t/*\n+\t * renames: various data relating to rename detection\n+\t */\n+\tstruct rename_info renames;\n+\n \t/*\n \t * current_dir_name: temporary var used in collect_merge_info_callback()\n \t *\n-- \ngitgitgadget\n\n"},{"id":"412173","messageId":"620fc64032d26e374b2fb52a044564725e023f0f.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 03/11] merge-ort: implement detect_regular_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:32Z","receivedAt":"2020-12-14T16:23:00Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nBased heavily on merge-recursive's get_diffpairs() function, and also\nincludes the necessary paired call to diff_warn_rename_limit() so that\nusers will be warned if merge.renameLimit is not sufficiently large for\nrename detection to run.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 32 +++++++++++++++++++++++++++++++-\n 1 file changed, 31 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 8552f5e2318..66f84d39b43 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -661,7 +661,33 @@ static void detect_regular_renames(struct merge_options *opt,\n \t\t\t\t struct tree *side,\n \t\t\t\t unsigned side_index)\n {\n-\tdie(\"Not yet implemented.\");\n+\tstruct diff_options diff_opts;\n+\tstruct rename_info *renames = &opt->priv->renames;\n+\n+\trepo_diff_setup(opt->repo, &diff_opts);\n+\tdiff_opts.flags.recursive = 1;\n+\tdiff_opts.flags.rename_empty = 0;\n+\tdiff_opts.detect_rename = DIFF_DETECT_RENAME;\n+\tdiff_opts.rename_limit = opt->rename_limit;\n+\tif (opt->rename_limit <= 0)\n+\t\tdiff_opts.rename_limit = 1000;\n+\tdiff_opts.rename_score = opt->rename_score;\n+\tdiff_opts.show_rename_progress = opt->show_rename_progress;\n+\tdiff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n+\tdiff_setup_done(&diff_opts);\n+\tdiff_tree_oid(&merge_base->object.oid, &side->object.oid, \"\",\n+\t\t &diff_opts);\n+\tdiffcore_std(&diff_opts);\n+\n+\tif (diff_opts.needed_rename_limit > renames->needed_limit)\n+\t\trenames->needed_limit = diff_opts.needed_rename_limit;\n+\n+\trenames->pairs[side_index] = diff_queued_diff;\n+\n+\tdiff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n+\tdiff_queued_diff.nr = 0;\n+\tdiff_queued_diff.queue = NULL;\n+\tdiff_flush(&diff_opts);\n }\n \n /*\n@@ -1406,6 +1432,10 @@ void merge_switch_to_result(struct merge_options *opt,\n \t\t\tprintf(\"%s\", sb->buf);\n \t\t}\n \t\tstring_list_clear(&olist, 0);\n+\n+\t\t/* Also include needed rename limit adjustment now */\n+\t\tdiff_warn_rename_limit(\"merge.renamelimit\",\n+\t\t\t\t opti->renames.needed_limit, 0);\n \t}\n \n \tmerge_finalize(opt, result);\n-- \ngitgitgadget\n\n"},{"id":"412174","messageId":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.git.1607542887.gitgitgadget@gmail.com","subject":"[PATCH v2 00/11] merge-ort: add basic rename detection","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:29Z","receivedAt":"2020-12-14T16:23:00Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This series builds on en/merge-ort-2 and adds basic rename detection to\nmerge-ort.\n\nChanges since v1 (all due to feedback from Stolee's reviews):\n\n * embedded struct rename_info directly in struct merge_options_internal (no\n longer a pointer)\n * expanded use of new enum merge_side and its new MERGE_BASE, MERGE_SIDE1,\n MERGE_SIDE2 constants\n * removed unnecessary secondary sort in compare_pairs()\n * improved commit messages\n * document p->score reuse with better comment(s)\n * space around operators\n\nElijah Newren (11):\n merge-ort: add basic data structures for handling renames\n merge-ort: add initial outline for basic rename detection\n merge-ort: implement detect_regular_renames()\n merge-ort: implement compare_pairs() and collect_renames()\n merge-ort: add basic outline for process_renames()\n merge-ort: add implementation of both sides renaming identically\n merge-ort: add implementation of both sides renaming differently\n merge-ort: add implementation of rename collisions\n merge-ort: add implementation of rename/delete conflicts\n merge-ort: add implementation of normal rename handling\n merge-ort: add implementation of type-changed rename handling\n\n merge-ort.c | 445 ++++++++++++++++++++++++++++++++++++++++++++++++++--\n 1 file changed, 429 insertions(+), 16 deletions(-)\n\n\nbase-commit: c5a6f65527aa3b6f5d7cf25437a88d8727ab0646\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-812%2Fnewren%2Fort-renames-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-812/newren/ort-renames-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/812\n\nRange-diff vs v1:\n\n 1: ef8f315f828 ! 1: 78621ca0788 merge-ort: add basic data structures for handling renames\n @@ Commit message\n Signed-off-by: Elijah Newren \n \n ## merge-ort.c ##\n -@@\n - #include \"unpack-trees.h\"\n - #include \"xdiff-interface.h\"\n +@@ merge-ort.c: enum merge_side {\n + \tMERGE_SIDE2 = 2\n + };\n \n +struct rename_info {\n +\t/*\n @@ merge-ort.c: struct merge_options_internal {\n +\t/*\n +\t * renames: various data relating to rename detection\n +\t */\n -+\tstruct rename_info *renames;\n ++\tstruct rename_info renames;\n +\n \t/*\n \t * current_dir_name: temporary var used in collect_merge_info_callback()\n \t *\n -@@ merge-ort.c: static void merge_start(struct merge_options *opt, struct merge_result *result)\n - \n - \t/* Initialization of opt->priv, our internal merge data */\n - \topt->priv = xcalloc(1, sizeof(*opt->priv));\n -+\topt->priv->renames = xcalloc(1, sizeof(*opt->priv->renames));\n - \n - \t/*\n - \t * Although we initialize opt->priv->paths with strdup_strings=0,\n 2: b9e0e1a60b9 ! 2: d846decf40b merge-ort: add initial outline for basic rename detection\n @@ merge-ort.c: static int handle_content_merge(struct merge_options *opt,\n {\n -\tint clean = 1;\n +\tstruct diff_queue_struct combined;\n -+\tstruct rename_info *renames = opt->priv->renames;\n ++\tstruct rename_info *renames = &opt->priv->renames;\n +\tint s, clean = 1;\n +\n +\tmemset(&combined, 0, sizeof(combined));\n +\n -+\tdetect_regular_renames(opt, merge_base, side1, 1);\n -+\tdetect_regular_renames(opt, merge_base, side2, 2);\n ++\tdetect_regular_renames(opt, merge_base, side1, MERGE_SIDE1);\n ++\tdetect_regular_renames(opt, merge_base, side2, MERGE_SIDE2);\n +\n +\tALLOC_GROW(combined.queue,\n +\t\t renames->pairs[1].nr + renames->pairs[2].nr,\n +\t\t combined.alloc);\n -+\tclean &= collect_renames(opt, &combined, 1);\n -+\tclean &= collect_renames(opt, &combined, 2);\n ++\tclean &= collect_renames(opt, &combined, MERGE_SIDE1);\n ++\tclean &= collect_renames(opt, &combined, MERGE_SIDE2);\n +\tQSORT(combined.queue, combined.nr, compare_pairs);\n +\n +\tclean &= process_renames(opt, &combined);\n +\n +\t/* Free memory for renames->pairs[] and combined */\n -+\tfor (s = 1; s <= 2; s++) {\n ++\tfor (s = MERGE_SIDE1; s <= MERGE_SIDE2; s++) {\n +\t\tfree(renames->pairs[s].queue);\n +\t\tDIFF_QUEUE_CLEAR(&renames->pairs[s]);\n +\t}\n 3: ba30bc8686e ! 3: 620fc64032d merge-ort: implement detect_regular_renames()\n @@ Metadata\n ## Commit message ##\n merge-ort: implement detect_regular_renames()\n \n - Based heavily on merge-recursive's get_diffpairs() function.\n + Based heavily on merge-recursive's get_diffpairs() function, and also\n + includes the necessary paired call to diff_warn_rename_limit() so that\n + users will be warned if merge.renameLimit is not sufficiently large for\n + rename detection to run.\n \n Signed-off-by: Elijah Newren \n \n @@ merge-ort.c: static void detect_regular_renames(struct merge_options *opt,\n {\n -\tdie(\"Not yet implemented.\");\n +\tstruct diff_options diff_opts;\n -+\tstruct rename_info *renames = opt->priv->renames;\n ++\tstruct rename_info *renames = &opt->priv->renames;\n +\n +\trepo_diff_setup(opt->repo, &diff_opts);\n +\tdiff_opts.flags.recursive = 1;\n @@ merge-ort.c: static void detect_regular_renames(struct merge_options *opt,\n +\t\t &diff_opts);\n +\tdiffcore_std(&diff_opts);\n +\n -+\tif (diff_opts.needed_rename_limit > opt->priv->renames->needed_limit)\n -+\t\topt->priv->renames->needed_limit = diff_opts.needed_rename_limit;\n ++\tif (diff_opts.needed_rename_limit > renames->needed_limit)\n ++\t\trenames->needed_limit = diff_opts.needed_rename_limit;\n +\n +\trenames->pairs[side_index] = diff_queued_diff;\n +\n @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,\n +\n +\t\t/* Also include needed rename limit adjustment now */\n +\t\tdiff_warn_rename_limit(\"merge.renamelimit\",\n -+\t\t\t\t opti->renames->needed_limit, 0);\n ++\t\t\t\t opti->renames.needed_limit, 0);\n \t}\n \n \tmerge_finalize(opt, result);\n 4: 207bb9a837c ! 4: 9382dc4d50b merge-ort: implement compare_pairs() and collect_renames()\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n +\tconst struct diff_filepair *a = *((const struct diff_filepair **)a_);\n +\tconst struct diff_filepair *b = *((const struct diff_filepair **)b_);\n +\n -+\tint cmp = strcmp(a->one->path, b->one->path);\n -+\tif (cmp)\n -+\t\treturn cmp;\n -+\treturn a->score - b->score;\n ++\treturn strcmp(a->one->path, b->one->path);\n }\n \n /* Call diffcore_rename() to compute which files have changed on given side */\n @@ merge-ort.c: static int collect_renames(struct merge_options *opt,\n -\tdie(\"Not yet implemented.\");\n +\tint i, clean = 1;\n +\tstruct diff_queue_struct *side_pairs;\n -+\tstruct rename_info *renames = opt->priv->renames;\n ++\tstruct rename_info *renames = &opt->priv->renames;\n +\n +\tside_pairs = &renames->pairs[side_index];\n +\n @@ merge-ort.c: static int collect_renames(struct merge_options *opt,\n +\t\t\tdiff_free_filepair(p);\n +\t\t\tcontinue;\n +\t\t}\n ++\n ++\t\t/*\n ++\t\t * p->score comes back from diffcore_rename_extended() with\n ++\t\t * the similarity of the renamed file. The similarity is\n ++\t\t * was used to determine that the two files were related\n ++\t\t * and are a rename, which we have already used, but beyond\n ++\t\t * that we have no use for the similarity. So p->score is\n ++\t\t * now irrelevant. However, process_renames() will need to\n ++\t\t * know which side of the merge this rename was associated\n ++\t\t * with, so overwrite p->score with that value.\n ++\t\t */\n +\t\tp->score = side_index;\n +\t\tresult->queue[result->nr++] = p;\n +\t}\n 5: 35b070b9b7c ! 5: d20fab8d403 merge-ort: add basic outline for process_renames()\n @@ merge-ort.c: static int handle_content_merge(struct merge_options *opt,\n +\t\t * diff_filepairs have copies of pathnames, thus we have to\n +\t\t * use standard 'strcmp()' (negated) instead of '=='.\n +\t\t */\n -+\t\tif (i+1 < renames->nr &&\n ++\t\tif (i + 1 < renames->nr &&\n +\t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n +\t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n +\t\t\tconst char *pathnames[3];\n @@ merge-ort.c: static int handle_content_merge(struct merge_options *opt,\n +\n +\t\tVERIFY_CI(oldinfo);\n +\t\tVERIFY_CI(newinfo);\n -+\t\ttarget_index = pair->score; /* from append_rename_pairs() */\n ++\t\ttarget_index = pair->score; /* from collect_renames() */\n +\t\tassert(target_index == 1 || target_index == 2);\n -+\t\tother_source_index = 3-target_index;\n ++\t\tother_source_index = 3 - target_index;\n +\t\told_sidemask = (1 << other_source_index); /* 2 or 4 */\n +\t\tsource_deleted = (oldinfo->filemask == 1);\n +\t\tcollision = ((newinfo->filemask & old_sidemask) != 0);\n 6: 9c79b9f4a09 ! 6: 15fff3dd0c4 merge-ort: add implementation of both sides renaming identically\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n +\t\t\t\tassert(side1 == side2);\n +\t\t\t\tmemcpy(&side1->stages[0], &base->stages[0],\n +\t\t\t\t sizeof(merged));\n -+\t\t\t\tside1->filemask |= (1 << 0);\n ++\t\t\t\tside1->filemask |= (1 << MERGE_BASE);\n +\t\t\t\t/* Mark base as resolved by removal */\n +\t\t\t\tbase->merged.is_null = 1;\n +\t\t\t\tbase->merged.clean = 1;\n 7: d4595397052 ! 7: d00e26be784 merge-ort: add implementation of both sides renaming differently\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n +\t\t\t\t\t\t\t &merged);\n +\t\t\tif (!clean_merge &&\n +\t\t\t merged.mode == side1->stages[1].mode &&\n -+\t\t\t oideq(&merged.oid, &side1->stages[1].oid)) {\n ++\t\t\t oideq(&merged.oid, &side1->stages[1].oid))\n +\t\t\t\twas_binary_blob = 1;\n -+\t\t\t}\n +\t\t\tmemcpy(&side1->stages[1], &merged, sizeof(merged));\n +\t\t\tif (was_binary_blob) {\n +\t\t\t\t/*\n 8: ab15f85f698 = 8: edd610321a0 merge-ort: add implementation of rename collisions\n 9: c069d34b15f ! 9: f017534243c merge-ort: add implementation of rename/delete conflicts\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n +\t\t\t */\n +\t\t\tmemcpy(&newinfo->stages[0], &oldinfo->stages[0],\n +\t\t\t sizeof(newinfo->stages[0]));\n -+\t\t\tnewinfo->filemask |= (1 << 0);\n ++\t\t\tnewinfo->filemask |= (1 << MERGE_BASE);\n +\t\t\tnewinfo->pathnames[0] = oldpath;\n \t\t\tif (type_changed) {\n \t\t\t\t/* rename vs. typechange */\n 10: 14baa5874af = 10: 22cb7110261 merge-ort: add implementation of normal rename handling\n 11: 476553e2b20 = 11: ff09ddb9caf merge-ort: add implementation of type-changed rename handling\n\n-- \ngitgitgadget\n"},{"id":"412175","messageId":"d846decf40be829590b88bb61fb19ae6f64c3162.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:31Z","receivedAt":"2020-12-14T16:23:07Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++-------\n 1 file changed, 60 insertions(+), 8 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 1c1a7fa4bf1..8552f5e2318 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -644,20 +644,72 @@ static int handle_content_merge(struct merge_options *opt,\n \n /*** Function Grouping: functions related to regular rename detection ***/\n \n+static int process_renames(struct merge_options *opt,\n+\t\t\t struct diff_queue_struct *renames)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+static int compare_pairs(const void *a_, const void *b_)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+/* Call diffcore_rename() to compute which files have changed on given side */\n+static void detect_regular_renames(struct merge_options *opt,\n+\t\t\t\t struct tree *merge_base,\n+\t\t\t\t struct tree *side,\n+\t\t\t\t unsigned side_index)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+/*\n+ * Get information of all renames which occurred in 'side_pairs', discarding\n+ * non-renames.\n+ */\n+static int collect_renames(struct merge_options *opt,\n+\t\t\t struct diff_queue_struct *result,\n+\t\t\t unsigned side_index)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n static int detect_and_process_renames(struct merge_options *opt,\n \t\t\t\t struct tree *merge_base,\n \t\t\t\t struct tree *side1,\n \t\t\t\t struct tree *side2)\n {\n-\tint clean = 1;\n+\tstruct diff_queue_struct combined;\n+\tstruct rename_info *renames = &opt->priv->renames;\n+\tint s, clean = 1;\n+\n+\tmemset(&combined, 0, sizeof(combined));\n+\n+\tdetect_regular_renames(opt, merge_base, side1, MERGE_SIDE1);\n+\tdetect_regular_renames(opt, merge_base, side2, MERGE_SIDE2);\n+\n+\tALLOC_GROW(combined.queue,\n+\t\t renames->pairs[1].nr + renames->pairs[2].nr,\n+\t\t combined.alloc);\n+\tclean &= collect_renames(opt, &combined, MERGE_SIDE1);\n+\tclean &= collect_renames(opt, &combined, MERGE_SIDE2);\n+\tQSORT(combined.queue, combined.nr, compare_pairs);\n+\n+\tclean &= process_renames(opt, &combined);\n+\n+\t/* Free memory for renames->pairs[] and combined */\n+\tfor (s = MERGE_SIDE1; s <= MERGE_SIDE2; s++) {\n+\t\tfree(renames->pairs[s].queue);\n+\t\tDIFF_QUEUE_CLEAR(&renames->pairs[s]);\n+\t}\n+\tif (combined.nr) {\n+\t\tint i;\n+\t\tfor (i = 0; i < combined.nr; i++)\n+\t\t\tdiff_free_filepair(combined.queue[i]);\n+\t\tfree(combined.queue);\n+\t}\n \n-\t/*\n-\t * Rename detection works by detecting file similarity. Here we use\n-\t * a really easy-to-implement scheme: files are similar IFF they have\n-\t * the same filename. Therefore, by this scheme, there are no renames.\n-\t *\n-\t * TODO: Actually implement a real rename detection scheme.\n-\t */\n \treturn clean;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"412176","messageId":"9382dc4d50b3cf675d607cf5d39be08542d8147e.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 04/11] merge-ort: implement compare_pairs() and collect_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:33Z","receivedAt":"2020-12-14T16:23:29Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 35 +++++++++++++++++++++++++++++++++--\n 1 file changed, 33 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 66f84d39b43..10550c542b8 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -652,7 +652,10 @@ static int process_renames(struct merge_options *opt,\n \n static int compare_pairs(const void *a_, const void *b_)\n {\n-\tdie(\"Not yet implemented.\");\n+\tconst struct diff_filepair *a = *((const struct diff_filepair **)a_);\n+\tconst struct diff_filepair *b = *((const struct diff_filepair **)b_);\n+\n+\treturn strcmp(a->one->path, b->one->path);\n }\n \n /* Call diffcore_rename() to compute which files have changed on given side */\n@@ -698,7 +701,35 @@ static int collect_renames(struct merge_options *opt,\n \t\t\t struct diff_queue_struct *result,\n \t\t\t unsigned side_index)\n {\n-\tdie(\"Not yet implemented.\");\n+\tint i, clean = 1;\n+\tstruct diff_queue_struct *side_pairs;\n+\tstruct rename_info *renames = &opt->priv->renames;\n+\n+\tside_pairs = &renames->pairs[side_index];\n+\n+\tfor (i = 0; i < side_pairs->nr; ++i) {\n+\t\tstruct diff_filepair *p = side_pairs->queue[i];\n+\n+\t\tif (p->status != 'R') {\n+\t\t\tdiff_free_filepair(p);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/*\n+\t\t * p->score comes back from diffcore_rename_extended() with\n+\t\t * the similarity of the renamed file. The similarity is\n+\t\t * was used to determine that the two files were related\n+\t\t * and are a rename, which we have already used, but beyond\n+\t\t * that we have no use for the similarity. So p->score is\n+\t\t * now irrelevant. However, process_renames() will need to\n+\t\t * know which side of the merge this rename was associated\n+\t\t * with, so overwrite p->score with that value.\n+\t\t */\n+\t\tp->score = side_index;\n+\t\tresult->queue[result->nr++] = p;\n+\t}\n+\n+\treturn clean;\n }\n \n static int detect_and_process_renames(struct merge_options *opt,\n-- \ngitgitgadget\n\n"},{"id":"412177","messageId":"edd610321a053b431def5b06bb2983c7f4a84547.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 08/11] merge-ort: add implementation of rename collisions","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:37Z","receivedAt":"2020-12-14T16:23:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(2to1) and rename/add handling, i.e. a file is\nrenamed into a location where another file is added (with that other\nfile either being a plain add or itself coming from a rename). Note\nthat rename collisions can also have a special case stacked on top: the\nfile being renamed on one side of history is deleted on the other\n(yielding either a rename/add/delete conflict or perhaps a\nrename/rename(2to1)/delete[/delete]) conflict.\n\nOne thing to note here is that when there is a double rename, the code\nin question only handles one of them at a time; a later iteration\nthrough the loop will handle the other. After they've both been\nhandled, process_entry()'s normal add/add code can handle the collision.\n\nThis code replaces the following from merge-recurisve.c:\n\n * all the 2to1 code in process_renames()\n * the RENAME_TWO_FILES_TO_ONE case of process_entry()\n * handle_rename_rename_2to1()\n * handle_rename_add()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_file_collision()\n * setup_rename_conflict_info()\n\nThe consolidation of six separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++---\n 1 file changed, 51 insertions(+), 3 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 19477cfae60..04a16837849 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -785,10 +785,58 @@ static int process_renames(struct merge_options *opt,\n \t\t/* Need to check for special types of rename conflicts... */\n \t\tif (collision && !source_deleted) {\n \t\t\t/* collision: rename/add or rename/rename(2to1) */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\tconst char *pathnames[3];\n+\t\t\tstruct version_info merged;\n+\n+\t\t\tstruct conflict_info *base, *side1, *side2;\n+\t\t\tunsigned clean;\n+\n+\t\t\tpathnames[0] = oldpath;\n+\t\t\tpathnames[other_source_index] = oldpath;\n+\t\t\tpathnames[target_index] = newpath;\n+\n+\t\t\tbase = strmap_get(&opt->priv->paths, pathnames[0]);\n+\t\t\tside1 = strmap_get(&opt->priv->paths, pathnames[1]);\n+\t\t\tside2 = strmap_get(&opt->priv->paths, pathnames[2]);\n+\n+\t\t\tVERIFY_CI(base);\n+\t\t\tVERIFY_CI(side1);\n+\t\t\tVERIFY_CI(side2);\n+\n+\t\t\tclean = handle_content_merge(opt, pair->one->path,\n+\t\t\t\t\t\t &base->stages[0],\n+\t\t\t\t\t\t &side1->stages[1],\n+\t\t\t\t\t\t &side2->stages[2],\n+\t\t\t\t\t\t pathnames,\n+\t\t\t\t\t\t 1 + 2*opt->priv->call_depth,\n+\t\t\t\t\t\t &merged);\n+\n+\t\t\tmemcpy(&newinfo->stages[target_index], &merged,\n+\t\t\t sizeof(merged));\n+\t\t\tif (!clean) {\n+\t\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t\t _(\"CONFLICT (rename involved in \"\n+\t\t\t\t\t \"collision): rename of %s -> %s has \"\n+\t\t\t\t\t \"content conflicts AND collides \"\n+\t\t\t\t\t \"with another path; this may result \"\n+\t\t\t\t\t \"in nested conflict markers.\"),\n+\t\t\t\t\t oldpath, newpath);\n+\t\t\t}\n \t\t} else if (collision && source_deleted) {\n-\t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\t/*\n+\t\t\t * rename/add/delete or rename/rename(2to1)/delete:\n+\t\t\t * since oldpath was deleted on the side that didn't\n+\t\t\t * do the rename, there's not much of a content merge\n+\t\t\t * we can do for the rename. oldinfo->merged.is_null\n+\t\t\t * was already set, so we just leave things as-is so\n+\t\t\t * they look like an add/add conflict.\n+\t\t\t */\n+\n+\t\t\tnewinfo->path_conflict = 1;\n+\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed \"\n+\t\t\t\t \"to %s in %s, but deleted in %s.\"),\n+\t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n \t\t} else {\n \t\t\t/* a few different cases... */\n \t\t\tif (type_changed) {\n-- \ngitgitgadget\n\n"},{"id":"412178","messageId":"22cb71102614f9d1fdd41a4b9524f11e08afa241.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 10/11] merge-ort: add implementation of normal rename handling","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:39Z","receivedAt":"2020-12-14T16:23:39Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement handling of normal renames. This code replaces the following\nfrom merge-recurisve.c:\n\n * the code relevant to RENAME_NORMAL in process_renames()\n * the RENAME_NORMAL case of process_entry()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_rename_normal()\n * setup_rename_conflict_info()\n\nThe consolidation of four separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\n(To be fair, the code for handling normal renames wasn't all that\ncomplicated beforehand, but it's still much simpler now.)\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 4150ccc35e1..9aac33c8e31 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -869,7 +869,11 @@ static int process_renames(struct merge_options *opt,\n \t\t\t\t\t rename_branch, delete_branch);\n \t\t\t} else {\n \t\t\t\t/* normal rename */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\tmemcpy(&newinfo->stages[other_source_index],\n+\t\t\t\t &oldinfo->stages[other_source_index],\n+\t\t\t\t sizeof(newinfo->stages[0]));\n+\t\t\t\tnewinfo->filemask |= (1 << other_source_index);\n+\t\t\t\tnewinfo->pathnames[other_source_index] = oldpath;\n \t\t\t}\n \t\t}\n \n-- \ngitgitgadget\n\n"},{"id":"412179","messageId":"15fff3dd0c48c301bfcb7fe335fa02fad1c686ab.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 06/11] merge-ort: add implementation of both sides renaming identically","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:35Z","receivedAt":"2020-12-14T16:25:09Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(1to1) handling, i.e. both sides of history\nrenaming a file but renaming the same way. This code replaces the\nfollowing from merge-recurisve.c:\n\n * all the 1to1 code in process_renames()\n * the RENAME_ONE_FILE_TO_ONE case of process_entry()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_rename_normal()\n * setup_rename_conflict_info()\n\nThe consolidation of four separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 21 +++++++++++++++++++--\n 1 file changed, 19 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex ebe275ef73c..4034ffcf501 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -674,14 +674,31 @@ static int process_renames(struct merge_options *opt,\n \t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n \t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n \t\t\tconst char *pathnames[3];\n+\t\t\tstruct version_info merged;\n+\t\t\tstruct conflict_info *base, *side1, *side2;\n+\t\t\tunsigned was_binary_blob = 0;\n \n \t\t\tpathnames[0] = oldpath;\n \t\t\tpathnames[1] = newpath;\n \t\t\tpathnames[2] = renames->queue[i+1]->two->path;\n \n+\t\t\tbase = strmap_get(&opt->priv->paths, pathnames[0]);\n+\t\t\tside1 = strmap_get(&opt->priv->paths, pathnames[1]);\n+\t\t\tside2 = strmap_get(&opt->priv->paths, pathnames[2]);\n+\n+\t\t\tVERIFY_CI(base);\n+\t\t\tVERIFY_CI(side1);\n+\t\t\tVERIFY_CI(side2);\n+\n \t\t\tif (!strcmp(pathnames[1], pathnames[2])) {\n-\t\t\t\t/* Both sides renamed the same way. */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\t/* Both sides renamed the same way */\n+\t\t\t\tassert(side1 == side2);\n+\t\t\t\tmemcpy(&side1->stages[0], &base->stages[0],\n+\t\t\t\t sizeof(merged));\n+\t\t\t\tside1->filemask |= (1 << MERGE_BASE);\n+\t\t\t\t/* Mark base as resolved by removal */\n+\t\t\t\tbase->merged.is_null = 1;\n+\t\t\t\tbase->merged.clean = 1;\n \n \t\t\t\t/* We handled both renames, i.e. i+1 handled */\n \t\t\t\ti++;\n-- \ngitgitgadget\n\n"},{"id":"412180","messageId":"d20fab8d403851ff9247d8b6bb950927f72b2729.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 05/11] merge-ort: add basic outline for process_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:34Z","receivedAt":"2020-12-14T16:25:09Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nAdd code which determines which kind of special rename case each rename\ncorresponds to, but leave the handling of each type unimplemented for\nnow. Future commits will implement each one.\n\nThere is some tenuous resemblance to merge-recursive's\nprocess_renames(), but comparing the two is very unlikely to yield any\ninsights. merge-ort's process_renames() is a bit complex and I would\nprefer if I could simplify it more, but it is far easier to grok than\nmerge-recursive's function of the same name in my opinion. Plus,\nmerge-ort handles more rename conflict types than merge-recursive does.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 97 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 10550c542b8..ebe275ef73c 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -647,7 +647,103 @@ static int handle_content_merge(struct merge_options *opt,\n static int process_renames(struct merge_options *opt,\n \t\t\t struct diff_queue_struct *renames)\n {\n-\tdie(\"Not yet implemented.\");\n+\tint clean_merge = 1, i;\n+\n+\tfor (i = 0; i < renames->nr; ++i) {\n+\t\tconst char *oldpath = NULL, *newpath;\n+\t\tstruct diff_filepair *pair = renames->queue[i];\n+\t\tstruct conflict_info *oldinfo = NULL, *newinfo = NULL;\n+\t\tstruct strmap_entry *old_ent, *new_ent;\n+\t\tunsigned int old_sidemask;\n+\t\tint target_index, other_source_index;\n+\t\tint source_deleted, collision, type_changed;\n+\n+\t\told_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n+\t\toldpath = old_ent->key;\n+\t\toldinfo = old_ent->value;\n+\n+\t\tnew_ent = strmap_get_entry(&opt->priv->paths, pair->two->path);\n+\t\tnewpath = new_ent->key;\n+\t\tnewinfo = new_ent->value;\n+\n+\t\t/*\n+\t\t * diff_filepairs have copies of pathnames, thus we have to\n+\t\t * use standard 'strcmp()' (negated) instead of '=='.\n+\t\t */\n+\t\tif (i + 1 < renames->nr &&\n+\t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n+\t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n+\t\t\tconst char *pathnames[3];\n+\n+\t\t\tpathnames[0] = oldpath;\n+\t\t\tpathnames[1] = newpath;\n+\t\t\tpathnames[2] = renames->queue[i+1]->two->path;\n+\n+\t\t\tif (!strcmp(pathnames[1], pathnames[2])) {\n+\t\t\t\t/* Both sides renamed the same way. */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\n+\t\t\t\t/* We handled both renames, i.e. i+1 handled */\n+\t\t\t\ti++;\n+\t\t\t\t/* Move to next rename */\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\t/* This is a rename/rename(1to2) */\n+\t\t\tdie(\"Not yet implemented\");\n+\n+\t\t\ti++; /* We handled both renames, i.e. i+1 handled */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tVERIFY_CI(oldinfo);\n+\t\tVERIFY_CI(newinfo);\n+\t\ttarget_index = pair->score; /* from collect_renames() */\n+\t\tassert(target_index == 1 || target_index == 2);\n+\t\tother_source_index = 3 - target_index;\n+\t\told_sidemask = (1 << other_source_index); /* 2 or 4 */\n+\t\tsource_deleted = (oldinfo->filemask == 1);\n+\t\tcollision = ((newinfo->filemask & old_sidemask) != 0);\n+\t\ttype_changed = !source_deleted &&\n+\t\t\t(S_ISREG(oldinfo->stages[other_source_index].mode) !=\n+\t\t\t S_ISREG(newinfo->stages[target_index].mode));\n+\t\tif (type_changed && collision) {\n+\t\t\t/* special handling so later blocks can handle this */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t}\n+\n+\t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n+\n+\t\t/* Need to check for special types of rename conflicts... */\n+\t\tif (collision && !source_deleted) {\n+\t\t\t/* collision: rename/add or rename/rename(2to1) */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t} else if (collision && source_deleted) {\n+\t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t} else {\n+\t\t\t/* a few different cases... */\n+\t\t\tif (type_changed) {\n+\t\t\t\t/* rename vs. typechange */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t} else if (source_deleted) {\n+\t\t\t\t/* rename/delete */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t} else {\n+\t\t\t\t/* normal rename */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!type_changed) {\n+\t\t\t/* Mark the original as resolved by removal */\n+\t\t\toldinfo->merged.is_null = 1;\n+\t\t\toldinfo->merged.clean = 1;\n+\t\t}\n+\n+\t}\n+\n+\treturn clean_merge;\n }\n \n static int compare_pairs(const void *a_, const void *b_)\n-- \ngitgitgadget\n\n"},{"id":"412181","messageId":"f017534243c967caa0137e6899f4e1c69ff02c2e.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 09/11] merge-ort: add implementation of rename/delete conflicts","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:38Z","receivedAt":"2020-12-14T16:25:40Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/delete conflicts, i.e. one side renames a file and the\nother deletes the file. This code replaces the following from\nmerge-recurisve.c:\n\n * the code relevant to RENAME_DELETE in process_renames()\n * the RENAME_DELETE case of process_entry()\n * handle_rename_delete()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_change_delete()\n * setup_rename_conflict_info()\n\nThe consolidation of five separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nTo be fair, there is a _slight_ tweak to process_entry() here, because\nrename/delete cases will also trigger the modify/delete codepath.\nHowever, we only want a modify/delete message to be printed for a\nrename/delete conflict if there is a content change in the renamed file\nin addition to the rename. So process_renames() and process_entry()\naren't quite fully orthogonal, but they are pretty close.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 47 +++++++++++++++++++++++++++++++++++++++--------\n 1 file changed, 39 insertions(+), 8 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 04a16837849..4150ccc35e1 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -657,6 +657,7 @@ static int process_renames(struct merge_options *opt,\n \t\tunsigned int old_sidemask;\n \t\tint target_index, other_source_index;\n \t\tint source_deleted, collision, type_changed;\n+\t\tconst char *rename_branch = NULL, *delete_branch = NULL;\n \n \t\told_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n \t\toldpath = old_ent->key;\n@@ -778,6 +779,14 @@ static int process_renames(struct merge_options *opt,\n \t\tif (type_changed && collision) {\n \t\t\t/* special handling so later blocks can handle this */\n \t\t\tdie(\"Not yet implemented\");\n+\t\tif (source_deleted) {\n+\t\t\tif (target_index == 1) {\n+\t\t\t\trename_branch = opt->branch1;\n+\t\t\t\tdelete_branch = opt->branch2;\n+\t\t\t} else {\n+\t\t\t\trename_branch = opt->branch2;\n+\t\t\t\tdelete_branch = opt->branch1;\n+\t\t\t}\n \t\t}\n \n \t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n@@ -838,13 +847,26 @@ static int process_renames(struct merge_options *opt,\n \t\t\t\t \"to %s in %s, but deleted in %s.\"),\n \t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n \t\t} else {\n-\t\t\t/* a few different cases... */\n+\t\t\t/*\n+\t\t\t * a few different cases...start by copying the\n+\t\t\t * existing stage(s) from oldinfo over the newinfo\n+\t\t\t * and update the pathname(s).\n+\t\t\t */\n+\t\t\tmemcpy(&newinfo->stages[0], &oldinfo->stages[0],\n+\t\t\t sizeof(newinfo->stages[0]));\n+\t\t\tnewinfo->filemask |= (1 << MERGE_BASE);\n+\t\t\tnewinfo->pathnames[0] = oldpath;\n \t\t\tif (type_changed) {\n \t\t\t\t/* rename vs. typechange */\n \t\t\t\tdie(\"Not yet implemented\");\n \t\t\t} else if (source_deleted) {\n \t\t\t\t/* rename/delete */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\tnewinfo->path_conflict = 1;\n+\t\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed\"\n+\t\t\t\t\t \" to %s in %s, but deleted in %s.\"),\n+\t\t\t\t\t oldpath, newpath,\n+\t\t\t\t\t rename_branch, delete_branch);\n \t\t\t} else {\n \t\t\t\t/* normal rename */\n \t\t\t\tdie(\"Not yet implemented\");\n@@ -1380,12 +1402,21 @@ static void process_entry(struct merge_options *opt,\n \t\tmodify_branch = (side == 1) ? opt->branch1 : opt->branch2;\n \t\tdelete_branch = (side == 1) ? opt->branch2 : opt->branch1;\n \n-\t\tpath_msg(opt, path, 0,\n-\t\t\t _(\"CONFLICT (modify/delete): %s deleted in %s \"\n-\t\t\t \"and modified in %s. Version %s of %s left \"\n-\t\t\t \"in tree.\"),\n-\t\t\t path, delete_branch, modify_branch,\n-\t\t\t modify_branch, path);\n+\t\tif (ci->path_conflict &&\n+\t\t oideq(&ci->stages[0].oid, &ci->stages[side].oid)) {\n+\t\t\t/*\n+\t\t\t * This came from a rename/delete; no action to take,\n+\t\t\t * but avoid printing \"modify/delete\" conflict notice\n+\t\t\t * since the contents were not modified.\n+\t\t\t */\n+\t\t} else {\n+\t\t\tpath_msg(opt, path, 0,\n+\t\t\t\t _(\"CONFLICT (modify/delete): %s deleted in %s \"\n+\t\t\t\t \"and modified in %s. Version %s of %s left \"\n+\t\t\t\t \"in tree.\"),\n+\t\t\t\t path, delete_branch, modify_branch,\n+\t\t\t\t modify_branch, path);\n+\t\t}\n \t} else if (ci->filemask == 2 || ci->filemask == 4) {\n \t\t/* Added on one side */\n \t\tint side = (ci->filemask == 4) ? 2 : 1;\n-- \ngitgitgadget\n\n"},{"id":"412182","messageId":"ff09ddb9caf73632c9792c07f1f7499a75a09606.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 11/11] merge-ort: add implementation of type-changed rename handling","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:40Z","receivedAt":"2020-12-14T16:25:57Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement cases where renames are involved in type changes (i.e. the\nside of history that didn't rename the file changed its type from a\nregular file to a symlink or submodule). There was some code to handle\nthis in merge-recursive but only in the special case when the renamed\nfile had no content changes. The code here works differently -- it\nknows process_entry() can handle mode conflicts, so it does a few\nminimal tweaks to ensure process_entry() can just finish the job as\nneeded.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 33 +++++++++++++++++++++++++++++++--\n 1 file changed, 31 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 9aac33c8e31..11e33f56edf 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -778,7 +778,32 @@ static int process_renames(struct merge_options *opt,\n \t\t\t S_ISREG(newinfo->stages[target_index].mode));\n \t\tif (type_changed && collision) {\n \t\t\t/* special handling so later blocks can handle this */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\t/*\n+\t\t\t * if type_changed && collision are both true, then this\n+\t\t\t * was really a double rename, but one side wasn't\n+\t\t\t * detected due to lack of break detection. I.e.\n+\t\t\t * something like\n+\t\t\t * orig: has normal file 'foo'\n+\t\t\t * side1: renames 'foo' to 'bar', adds 'foo' symlink\n+\t\t\t * side2: renames 'foo' to 'bar'\n+\t\t\t * In this case, the foo->bar rename on side1 won't be\n+\t\t\t * detected because the new symlink named 'foo' is\n+\t\t\t * there and we don't do break detection. But we detect\n+\t\t\t * this here because we don't want to merge the content\n+\t\t\t * of the foo symlink with the foo->bar file, so we\n+\t\t\t * have some logic to handle this special case. The\n+\t\t\t * easiest way to do that is make 'bar' on side1 not\n+\t\t\t * be considered a colliding file but the other part\n+\t\t\t * of a normal rename. If the file is very different,\n+\t\t\t * well we're going to get content merge conflicts\n+\t\t\t * anyway so it doesn't hurt. And if the colliding\n+\t\t\t * file also has a different type, that'll be handled\n+\t\t\t * by the content merge logic in process_entry() too.\n+\t\t\t *\n+\t\t\t * See also t6430, 'rename vs. rename/symlink'\n+\t\t\t */\n+\t\t\tcollision = 0;\n+\t\t}\n \t\tif (source_deleted) {\n \t\t\tif (target_index == 1) {\n \t\t\t\trename_branch = opt->branch1;\n@@ -858,7 +883,11 @@ static int process_renames(struct merge_options *opt,\n \t\t\tnewinfo->pathnames[0] = oldpath;\n \t\t\tif (type_changed) {\n \t\t\t\t/* rename vs. typechange */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\t/* Mark the original as resolved by removal */\n+\t\t\t\tmemcpy(&oldinfo->stages[0].oid, &null_oid,\n+\t\t\t\t sizeof(oldinfo->stages[0].oid));\n+\t\t\t\toldinfo->stages[0].mode = 0;\n+\t\t\t\toldinfo->filemask &= 0x06;\n \t\t\t} else if (source_deleted) {\n \t\t\t\t/* rename/delete */\n \t\t\t\tnewinfo->path_conflict = 1;\n-- \ngitgitgadget\n"},{"id":"412184","messageId":"d00e26be784d7915b2d476b0c86f37aa366a4274.1607962900.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v2 07/11] merge-ort: add implementation of both sides renaming differently","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-14T16:21:36Z","receivedAt":"2020-12-14T16:26:15Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(1to2) handling, i.e. both sides of history\nrenaming a file and rename it differently. This code replaces the\nfollowing from merge-recurisve.c:\n\n * all the 1to2 code in process_renames()\n * the RENAME_ONE_FILE_TO_TWO case of process_entry()\n * handle_rename_rename_1to2()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_file_collision()\n * setup_rename_conflict_info()\n\nThe consolidation of five separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nTo be fair, there is a _slight_ tweak to process_entry() here to make\nsure that the two different paths aren't marked as clean but are left in\na conflicted state. So process_renames() and process_entry() aren't\nquite entirely orthogonal, but they are pretty close.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++---\n 1 file changed, 54 insertions(+), 3 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 4034ffcf501..19477cfae60 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -707,7 +707,58 @@ static int process_renames(struct merge_options *opt,\n \t\t\t}\n \n \t\t\t/* This is a rename/rename(1to2) */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\tclean_merge = handle_content_merge(opt,\n+\t\t\t\t\t\t\t pair->one->path,\n+\t\t\t\t\t\t\t &base->stages[0],\n+\t\t\t\t\t\t\t &side1->stages[1],\n+\t\t\t\t\t\t\t &side2->stages[2],\n+\t\t\t\t\t\t\t pathnames,\n+\t\t\t\t\t\t\t 1 + 2 * opt->priv->call_depth,\n+\t\t\t\t\t\t\t &merged);\n+\t\t\tif (!clean_merge &&\n+\t\t\t merged.mode == side1->stages[1].mode &&\n+\t\t\t oideq(&merged.oid, &side1->stages[1].oid))\n+\t\t\t\twas_binary_blob = 1;\n+\t\t\tmemcpy(&side1->stages[1], &merged, sizeof(merged));\n+\t\t\tif (was_binary_blob) {\n+\t\t\t\t/*\n+\t\t\t\t * Getting here means we were attempting to\n+\t\t\t\t * merge a binary blob.\n+\t\t\t\t *\n+\t\t\t\t * Since we can't merge binaries,\n+\t\t\t\t * handle_content_merge() just takes one\n+\t\t\t\t * side. But we don't want to copy the\n+\t\t\t\t * contents of one side to both paths. We\n+\t\t\t\t * used the contents of side1 above for\n+\t\t\t\t * side1->stages, let's use the contents of\n+\t\t\t\t * side2 for side2->stages below.\n+\t\t\t\t */\n+\t\t\t\toidcpy(&merged.oid, &side2->stages[2].oid);\n+\t\t\t\tmerged.mode = side2->stages[2].mode;\n+\t\t\t}\n+\t\t\tmemcpy(&side2->stages[2], &merged, sizeof(merged));\n+\n+\t\t\tside1->path_conflict = 1;\n+\t\t\tside2->path_conflict = 1;\n+\t\t\t/*\n+\t\t\t * TODO: For renames we normally remove the path at the\n+\t\t\t * old name. It would thus seem consistent to do the\n+\t\t\t * same for rename/rename(1to2) cases, but we haven't\n+\t\t\t * done so traditionally and a number of the regression\n+\t\t\t * tests now encode an expectation that the file is\n+\t\t\t * left there at stage 1. If we ever decide to change\n+\t\t\t * this, add the following two lines here:\n+\t\t\t * base->merged.is_null = 1;\n+\t\t\t * base->merged.clean = 1;\n+\t\t\t * and remove the setting of base->path_conflict to 1.\n+\t\t\t */\n+\t\t\tbase->path_conflict = 1;\n+\t\t\tpath_msg(opt, oldpath, 0,\n+\t\t\t\t _(\"CONFLICT (rename/rename): %s renamed to \"\n+\t\t\t\t \"%s in %s and to %s in %s.\"),\n+\t\t\t\t pathnames[0],\n+\t\t\t\t pathnames[1], opt->branch1,\n+\t\t\t\t pathnames[2], opt->branch2);\n \n \t\t\ti++; /* We handled both renames, i.e. i+1 handled */\n \t\t\tcontinue;\n@@ -1292,13 +1343,13 @@ static void process_entry(struct merge_options *opt,\n \t\tint side = (ci->filemask == 4) ? 2 : 1;\n \t\tci->merged.result.mode = ci->stages[side].mode;\n \t\toidcpy(&ci->merged.result.oid, &ci->stages[side].oid);\n-\t\tci->merged.clean = !ci->df_conflict;\n+\t\tci->merged.clean = !ci->df_conflict && !ci->path_conflict;\n \t} else if (ci->filemask == 1) {\n \t\t/* Deleted on both sides */\n \t\tci->merged.is_null = 1;\n \t\tci->merged.result.mode = 0;\n \t\toidcpy(&ci->merged.result.oid, &null_oid);\n-\t\tci->merged.clean = 1;\n+\t\tci->merged.clean = !ci->path_conflict;\n \t}\n \n \t/*\n-- \ngitgitgadget\n\n"},{"id":"412186","messageId":"nycvar.QRO.7.76.6.2012141747400.25979@tvgsbejvaqbjf.bet","threadId":"54806","inReplyTo":"CABPp-BHyNuhNtc6JQgQRBsQFbobzYyS6fs=nopRuv8c1+9zW_w@mail.gmail.com","subject":"Re: [PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-12-14T16:50:18Z","receivedAt":"2020-12-14T16:52:27Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Elijah,\n\nOn Mon, 14 Dec 2020, Elijah Newren wrote:\n\n> On Mon, Dec 14, 2020 at 7:42 AM Johannes Schindelin\n> wrote:\n> >\n> > Hi Elijah & Stolee,\n> >\n> > On Mon, 14 Dec 2020, Derrick Stolee wrote:\n> >\n> > > On 12/13/2020 2:47 AM, Elijah Newren wrote:\n> > > >\n> > > > Sorry for two different email responses to the same email...\n> > > >\n> > > > Addressing the comments on this patchset mean re-submitting\n> > > > en/merge-ort-impl, and causing conflicts in en/merge-ort-2 and this\n> > > > series en/merge-ort-3. Since gitgitgadget will not allow me to submit\n> > > > patches against a series that isn't published by Junio, I'll need to\n> > > > ask Junio to temporarily drop both of these series, then later\n> > > > resubmit en/merge-ort-2 after he publishes my updates to\n> > > > en/merge-ort-impl. Then when he publishes my updates to\n> > > > en/merge-ort-2, I'll be able to submit my already-rebased patches for\n> > > > en/merge-ort-3.\n> > >\n> > > Let's chat privately about perhaps creatin\n> >\n> > Yes, I am totally willing to push up temporary branches if that helps you,\n> > or even giving you push permissions to do that.\n> >\n> > Ciao,\n> > Dscho\n>\n> Given the amount of changes left to push up, I suspect there'll be\n> more cases where it'd be useful. If I could get push permissions, and\n> a suggested namespace to use for such temporary branches, that'd help.\n\nI invited you into the GitGitGadget organization.\n\nRecently, I pushed up `hanwen/libreftable` to have a proper base branch\nfor myself, but I think that was probably not a good idea. I should have\nopened a namespace like `temp/` or some such.\n\n> In this particular case, though, one of my two fears was already\n> realized -- Junio jumped in and did the work of rebasing and conflict\n> resolving for en/merge-ort-2 and en/merge-ort-3. I didn't want to\n> burden him with that extra work, but what he pushed up for\n> en/merge-ort-2 is identical to what I have. So, all I have to do is\n> push en/merge-ort-3 with the extra changes I have in it. So this\n> particular time is taken care of.\n\nUnderstood.\n\nThanks,\nDscho\n"},{"id":"412188","messageId":"CABPp-BG4smguUNX5fz3q8ffFvv3tPt4ysBe2dzJ_4cAATTaYLg@mail.gmail.com","threadId":"54806","inReplyTo":"7b0aafae-cd57-d4f7-ac85-238471428d24@gmail.com","subject":"Re: [PATCH 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-14T17:35:23Z","receivedAt":"2020-12-14T19:06:59Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Dec 14, 2020 at 6:33 AM Derrick Stolee wrote:\n>\n> On 12/13/2020 2:47 AM, Elijah Newren wrote:\n> > Hi,\n> >\n> > Sorry for two different email responses to the same email...\n> >\n> > Addressing the comments on this patchset mean re-submitting\n> > en/merge-ort-impl, and causing conflicts in en/merge-ort-2 and this\n> > series en/merge-ort-3. Since gitgitgadget will not allow me to submit\n> > patches against a series that isn't published by Junio, I'll need to\n> > ask Junio to temporarily drop both of these series, then later\n> > resubmit en/merge-ort-2 after he publishes my updates to\n> > en/merge-ort-impl. Then when he publishes my updates to\n> > en/merge-ort-2, I'll be able to submit my already-rebased patches for\n> > en/merge-ort-3.\n>\n> Let's chat privately about perhaps creatin\n>\n> > A couple extra comments below...\n>\n>\n> >>> + int s, clean = 1;\n> >>> +\n> >>> + memset(&combined, 0, sizeof(combined));\n> >>> +\n> >>> + detect_regular_renames(opt, merge_base, side1, 1);\n> >>> + detect_regular_renames(opt, merge_base, side2, 2);\n> >>\n> >> Find the renames in each side's diff.\n> >>\n> >> I think the use of \"1\" and \"2\" here might be better situated\n> >> for an enum. Perhaps:\n> >>\n> >> enum merge_side {\n> >> MERGE_SIDE1 = 0,\n> >> MERGE_SIDE2 = 1,\n> >> };\n> >>\n> >> (Note, I shift these values to 0 and 1, respectively, allowing\n> >> us to truncate the pairs array to two entries while still\n> >> being mentally clear.)\n> >\n> > So, after mulling it over for a while, I created a\n> >\n> > enum merge_side {\n> > MERGE_BASE = 0,\n> > MERGE_SIDE1 = 1,\n> > MERGE_SIDE2 = 2\n> > };\n> >\n> > and I made use of it in several places. I just avoided going to an\n> > extreme with it (e.g. adding another enum for masks or changing all\n> > possibly relevant variables from ints to enum merge_side), and used it\n> > more as a document-when-values-are-meant-to-refer-to-sides-of-the-merge\n> > kind of thing. Of course, this affects two previous patchsets and not\n> > just this one, so I'll have to post a _lot_ of new patches... :-)\n>\n> I appreciate using names for the meaning behind a numerical constant.\n> You mentioned in the other thread that this will eventually expand to\n> a list of 10 entries, which is particularly frightening if we don't\n> get some control over it now.\n>\n> I generally prefer using types to convey meaning as well, but I'm\n> willing to relax on this because I believe C won't complain if you\n> pass a literal int into an enum-typed parameter, so the compiler\n> doesn't help enough in that sense.\n\nYeah, I went through my 'ort' branch with all 10 entries and did a\nregex search for \\b[12]\\b throughout merge-ort.c, then considered each\none in turn, updating to the new enum where it made sense. Then\nbackported the changes across en/merge-ort-impl and en/merge-ort-3\n(and I just /submit-ted the en/merge-ort-3 updates to the list). Took\nquite a while, of course, but I feel it's in good shape.\n\nSo, take a look at the new sets of series and let me know what you think.\n\n> > Something I missed in my reply yesterday...\n> >\n> > Note that mi->clean is NOT from struct merge_result. It is from\n> > struct merged_info, and in that struct it IS defined as \"unsigned\n> > clean:1\", i.e. it is a true boolean. The merged_info.clean field is\n> > used to determine whether a specific path merged cleanly.\n> >\n> > \"clean\" from struct merge_result is whether the entirety of the merge\n> > was clean or not. It's almost a boolean, but allows for a\n> > \"catastrophic problem encountered\" value. I added the following\n> > comment:\n> > /*\n> > * Whether the merge is clean; possible values:\n> > * 1: clean\n> > * 0: not clean (merge conflicts)\n> > * <0: operation aborted prematurely. (object database\n> > * unreadable, disk full, etc.) Worktree may be left in an\n> > * inconsistent state if operation failed near the end.\n> > */\n> >\n> > This also means that I either abort and return a negative value, or I\n> > can continue treating merge_result's \"clean\" field as a boolean.\n>\n> Having this comment helps a lot!\n>\n> > But again, this isn't new to this patchset; it affects the patchset\n> > before the patchset before this one.\n>\n> Right, when I had the current change checked out, I don't see the\n> patch that introduced the 'clean' member (though, I _could_ have\n> blamed to find out). Instead, I just got confused and thought it\n> worth a question. Your comment prevents this question in the future.\n\nYeah, definitely worth the question. I've been buried in\nmerge-recursive.c & related areas so long that I've forgotten that\ncertain things are weird or surprising on first look. The more of\nthose we can flag and document, the better.\n"},{"id":"412257","messageId":"48fc65a5-12ba-be67-2908-27a446b1f813@gmail.com","threadId":"54806","inReplyTo":"edd610321a053b431def5b06bb2983c7f4a84547.1607962900.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 08/11] merge-ort: add implementation of rename collisions","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-15T14:09:18Z","receivedAt":"2020-12-15T14:10:38Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Implement rename/rename(2to1) and rename/add handling, i.e. a file is\n> renamed into a location where another file is added (with that other\n> file either being a plain add or itself coming from a rename). Note\n> that rename collisions can also have a special case stacked on top: the\n> file being renamed on one side of history is deleted on the other\n> (yielding either a rename/add/delete conflict or perhaps a\n> rename/rename(2to1)/delete[/delete]) conflict.\n> \n> One thing to note here is that when there is a double rename, the code\n> in question only handles one of them at a time; a later iteration\n> through the loop will handle the other. After they've both been\n> handled, process_entry()'s normal add/add code can handle the collision.\n> \n> This code replaces the following from merge-recurisve.c:\n> \n> * all the 2to1 code in process_renames()\n> * the RENAME_TWO_FILES_TO_ONE case of process_entry()\n> * handle_rename_rename_2to1()\n> * handle_rename_add()\n> \n> Also, there is some shared code from merge-recursive.c for multiple\n> different rename cases which we will no longer need for this case (or\n> other rename cases):\n> \n> * handle_file_collision()\n> * setup_rename_conflict_info()\n> \n> The consolidation of six separate codepaths into one is made possible\n> by a change in design: process_renames() tweaks the conflict_info\n> entries within opt->priv->paths such that process_entry() can then\n> handle all the non-rename conflict types (directory/file, modify/delete,\n> etc.) orthogonally. This means we're much less likely to miss special\n> implementation of some kind of combination of conflict types (see\n> commits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n> 2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\n> directory/file conflict handling in ort\", 2020-10-26) for more details).\n> That, together with letting worktree/index updating be handled\n> orthogonally in the merge_switch_to_result() function, dramatically\n> simplifies the code for various special rename cases.\n\nI'm really happy that you broke out the cases earlier, and describe\nthem so well in the message. It makes this hunk of code really easy\nto understand:\n\n> +\t\t\tconst char *pathnames[3];\n> +\t\t\tstruct version_info merged;\n> +\n> +\t\t\tstruct conflict_info *base, *side1, *side2;\n> +\t\t\tunsigned clean;\n> +\n> +\t\t\tpathnames[0] = oldpath;\n> +\t\t\tpathnames[other_source_index] = oldpath;\n> +\t\t\tpathnames[target_index] = newpath;\n> +\n> +\t\t\tbase = strmap_get(&opt->priv->paths, pathnames[0]);\n> +\t\t\tside1 = strmap_get(&opt->priv->paths, pathnames[1]);\n> +\t\t\tside2 = strmap_get(&opt->priv->paths, pathnames[2]);\n> +\n> +\t\t\tVERIFY_CI(base);\n> +\t\t\tVERIFY_CI(side1);\n> +\t\t\tVERIFY_CI(side2);\n> +\n> +\t\t\tclean = handle_content_merge(opt, pair->one->path,\n> +\t\t\t\t\t\t &base->stages[0],\n> +\t\t\t\t\t\t &side1->stages[1],\n> +\t\t\t\t\t\t &side2->stages[2],\n> +\t\t\t\t\t\t pathnames,\n> +\t\t\t\t\t\t 1 + 2*opt->priv->call_depth,\n\nnit: \" * \"\n\n> +\t\t\t\t\t\t &merged);\n> +\n> +\t\t\tmemcpy(&newinfo->stages[target_index], &merged,\n> +\t\t\t sizeof(merged));\n> +\t\t\tif (!clean) {\n> +\t\t\t\tpath_msg(opt, newpath, 0,\n> +\t\t\t\t\t _(\"CONFLICT (rename involved in \"\n> +\t\t\t\t\t \"collision): rename of %s -> %s has \"\n> +\t\t\t\t\t \"content conflicts AND collides \"\n> +\t\t\t\t\t \"with another path; this may result \"\n> +\t\t\t\t\t \"in nested conflict markers.\"),\n> +\t\t\t\t\t oldpath, newpath);\n\nI was briefly taken aback by \"AND collides with another path\" wondering if\nthat wording helps users understand the type of conflict here. But I can't\nthink of anything better, so *shrug*.\n\n> +\t\t\t}\n> \t\t} else if (collision && source_deleted) {\n> -\t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n> -\t\t\tdie(\"Not yet implemented\");\n> +\t\t\t/*\n> +\t\t\t * rename/add/delete or rename/rename(2to1)/delete:\n> +\t\t\t * since oldpath was deleted on the side that didn't\n> +\t\t\t * do the rename, there's not much of a content merge\n> +\t\t\t * we can do for the rename. oldinfo->merged.is_null\n> +\t\t\t * was already set, so we just leave things as-is so\n> +\t\t\t * they look like an add/add conflict.\n> +\t\t\t */\n> +\n> +\t\t\tnewinfo->path_conflict = 1;\n> +\t\t\tpath_msg(opt, newpath, 0,\n> +\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed \"\n> +\t\t\t\t \"to %s in %s, but deleted in %s.\"),\n> +\t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n\nI think this branch is added in the wrong patch. My compiler is complaining\nthat 'rename_branch' and 'delete_branch' are not declared (yet).\n\nThanks,\n-Stolee\n"},{"id":"412258","messageId":"05f52d13-d9ec-226d-370a-140937b12e7a@gmail.com","threadId":"54806","inReplyTo":"f017534243c967caa0137e6899f4e1c69ff02c2e.1607962900.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 09/11] merge-ort: add implementation of rename/delete conflicts","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-15T14:23:24Z","receivedAt":"2020-12-15T14:25:10Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Implement rename/delete conflicts, i.e. one side renames a file and the\n> other deletes the file. This code replaces the following from\n> merge-recurisve.c:\n> \n> * the code relevant to RENAME_DELETE in process_renames()\n> * the RENAME_DELETE case of process_entry()\n> * handle_rename_delete()\n> \n> Also, there is some shared code from merge-recursive.c for multiple\n> different rename cases which we will no longer need for this case (or\n> other rename cases):\n> \n> * handle_change_delete()\n> * setup_rename_conflict_info()\n> \n> The consolidation of five separate codepaths into one is made possible\n> by a change in design: process_renames() tweaks the conflict_info\n> entries within opt->priv->paths such that process_entry() can then\n> handle all the non-rename conflict types (directory/file, modify/delete,\n> etc.) orthogonally. This means we're much less likely to miss special\n> implementation of some kind of combination of conflict types (see\n> commits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n> 2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\n> directory/file conflict handling in ort\", 2020-10-26) for more details).\n> That, together with letting worktree/index updating be handled\n> orthogonally in the merge_switch_to_result() function, dramatically\n> simplifies the code for various special rename cases.\n> \n> To be fair, there is a _slight_ tweak to process_entry() here, because\n> rename/delete cases will also trigger the modify/delete codepath.\n> However, we only want a modify/delete message to be printed for a\n> rename/delete conflict if there is a content change in the renamed file\n> in addition to the rename. So process_renames() and process_entry()\n> aren't quite fully orthogonal, but they are pretty close.\n\nThanks for adding this warning about the change to process_entry().\n\n> @@ -657,6 +657,7 @@ static int process_renames(struct merge_options *opt,\n> \t\tunsigned int old_sidemask;\n> \t\tint target_index, other_source_index;\n> \t\tint source_deleted, collision, type_changed;\n> +\t\tconst char *rename_branch = NULL, *delete_branch = NULL;\n\nAh, here they are!\n\n> +\t\tif (source_deleted) {\n> +\t\t\tif (target_index == 1) {\n> +\t\t\t\trename_branch = opt->branch1;\n> +\t\t\t\tdelete_branch = opt->branch2;\n> +\t\t\t} else {\n> +\t\t\t\trename_branch = opt->branch2;\n> +\t\t\t\tdelete_branch = opt->branch1;\n> +\t\t\t}\n> \t\t}\n> \n> \t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n> @@ -838,13 +847,26 @@ static int process_renames(struct merge_options *opt,\n> \t\t\t\t \"to %s in %s, but deleted in %s.\"),\n> \t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n\nThis context line is the previous use of rename_branch and delete_branch.\nPerhaps the declarations, initialization, and first-use here are worth\ntheir own patch?\n\n> \t\t} else {\n> +\t\t\t/*\n> +\t\t\t * a few different cases...start by copying the\n> +\t\t\t * existing stage(s) from oldinfo over the newinfo\n> +\t\t\t * and update the pathname(s).\n> +\t\t\t */\n> +\t\t\tmemcpy(&newinfo->stages[0], &oldinfo->stages[0],\n> +\t\t\t sizeof(newinfo->stages[0]));\n> +\t\t\tnewinfo->filemask |= (1 << MERGE_BASE);\n> +\t\t\tnewinfo->pathnames[0] = oldpath;\n> \t\t\tif (type_changed) {\n> \t\t\t\t/* rename vs. typechange */\n> \t\t\t\tdie(\"Not yet implemented\");\n> \t\t\t} else if (source_deleted) {\n> \t\t\t\t/* rename/delete */\n> +\t\t\t\tnewinfo->path_conflict = 1;\n> +\t\t\t\tpath_msg(opt, newpath, 0,\n> +\t\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed\"\n> +\t\t\t\t\t \" to %s in %s, but deleted in %s.\"),\n> +\t\t\t\t\t oldpath, newpath,\n> +\t\t\t\t\t rename_branch, delete_branch);\n\nSince the primary purpose of rename_branch and delete_branch appears to\nbe for these error messages, then likely the previous error message about\na rename/delete should just be promoted into this patch instead of the\nprevious.\n\nIn fact, the error messages are the exact same, but with slightly different\nlines due to wrapping:\n\n\t\t\tpath_msg(opt, newpath, 0,\n\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed \"\n\t\t\t\t \"to %s in %s, but deleted in %s.\"),\n\t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n\nand\n\n\t\t\t\tpath_msg(opt, newpath, 0,\n\t\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed\"\n\t\t\t\t\t \" to %s in %s, but deleted in %s.\"),\n\t\t\t\t\t oldpath, newpath,\n\t\t\t\t\t rename_branch, delete_branch);\n\nI wonder if there is a way to group these together? Perhaps the nested\nif/else if/else blocks could store a \"conflict state\" value that says\nwhich CONFLICT message to print after the complicated branching is done.\n\nAlternatively, this message appears to be written in the following case:\n\n\tsource_deleted && !type_changed\n\nyour if/else if/else block could be rearranged as follows:\n\n\tif (collision && !source_deleted)\n\t\t/* collision: rename/add or rename/rename(2to1) */\n\telse if (!type_change && source_deleted)\n\t\t/* rename/delete or rename/add/delete or rename/rename(2to1)/delete */\n\telse if (!collision)\n\t\t/* a few different cases */\n\nOf course, the thing I am missing is that copy of oldinfo->stages[0] into\nnewinfo->stages[0] along with changes to the filemask and pathnames! That\nis likely why you need the two different markers, because the cases truly\nare different in that subtle way.\n\n> \t\t\t\t/* normal rename */\n> \t\t\t\tdie(\"Not yet implemented\");\n> @@ -1380,12 +1402,21 @@ static void process_entry(struct merge_options *opt,\n> \t\tmodify_branch = (side == 1) ? opt->branch1 : opt->branch2;\n> \t\tdelete_branch = (side == 1) ? opt->branch2 : opt->branch1;\n> \n> -\t\tpath_msg(opt, path, 0,\n> -\t\t\t _(\"CONFLICT (modify/delete): %s deleted in %s \"\n> -\t\t\t \"and modified in %s. Version %s of %s left \"\n> -\t\t\t \"in tree.\"),\n> -\t\t\t path, delete_branch, modify_branch,\n> -\t\t\t modify_branch, path);\n> +\t\tif (ci->path_conflict &&\n> +\t\t oideq(&ci->stages[0].oid, &ci->stages[side].oid)) {\n> +\t\t\t/*\n> +\t\t\t * This came from a rename/delete; no action to take,\n> +\t\t\t * but avoid printing \"modify/delete\" conflict notice\n> +\t\t\t * since the contents were not modified.\n> +\t\t\t */\n> +\t\t} else {\n> +\t\t\tpath_msg(opt, path, 0,\n> +\t\t\t\t _(\"CONFLICT (modify/delete): %s deleted in %s \"\n> +\t\t\t\t \"and modified in %s. Version %s of %s left \"\n> +\t\t\t\t \"in tree.\"),\n> +\t\t\t\t path, delete_branch, modify_branch,\n> +\t\t\t\t modify_branch, path);\n> +\t\t}\n\nThanks,\n-Stolee\n"},{"id":"412259","messageId":"217853ab-35c3-e5c9-f29e-2e1470386feb@gmail.com","threadId":"54806","inReplyTo":"f017534243c967caa0137e6899f4e1c69ff02c2e.1607962900.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 09/11] merge-ort: add implementation of rename/delete conflicts","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-15T14:27:17Z","receivedAt":"2020-12-15T14:28:06Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> \t\tif (type_changed && collision) {\n> \t\t\t/* special handling so later blocks can handle this */\n> \t\t\tdie(\"Not yet implemented\");\n> +\t\tif (source_deleted) {\n\nI didn't catch this in my earlier message, but the opening brace\nof the if (type_changed && collision) gets squashed here, causing\na compiler break.\n\nThanks,\n-Stolee\n"},{"id":"412260","messageId":"e7ce7008-0968-895c-f6e2-ee2609fd2556@gmail.com","threadId":"54806","inReplyTo":"22cb71102614f9d1fdd41a4b9524f11e08afa241.1607962900.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 10/11] merge-ort: add implementation of normal rename handling","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-15T14:27:54Z","receivedAt":"2020-12-15T14:28:52Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Implement handling of normal renames. This code replaces the following\n> from merge-recurisve.c:\n> \n> * the code relevant to RENAME_NORMAL in process_renames()\n> * the RENAME_NORMAL case of process_entry()\n> \n> Also, there is some shared code from merge-recursive.c for multiple\n> different rename cases which we will no longer need for this case (or\n> other rename cases):\n> \n> * handle_rename_normal()\n> * setup_rename_conflict_info()\n> \n> The consolidation of four separate codepaths into one is made possible\n> by a change in design: process_renames() tweaks the conflict_info\n> entries within opt->priv->paths such that process_entry() can then\n> handle all the non-rename conflict types (directory/file, modify/delete,\n> etc.) orthogonally. This means we're much less likely to miss special\n> implementation of some kind of combination of conflict types (see\n> commits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n> 2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\n> directory/file conflict handling in ort\", 2020-10-26) for more details).\n> That, together with letting worktree/index updating be handled\n> orthogonally in the merge_switch_to_result() function, dramatically\n> simplifies the code for various special rename cases.\n> \n> (To be fair, the code for handling normal renames wasn't all that\n> complicated beforehand, but it's still much simpler now.)\n\nDefinitely looks simple.\n\n> +\t\t\t\tmemcpy(&newinfo->stages[other_source_index],\n> +\t\t\t\t &oldinfo->stages[other_source_index],\n> +\t\t\t\t sizeof(newinfo->stages[0]));\n> +\t\t\t\tnewinfo->filemask |= (1 << other_source_index);\n> +\t\t\t\tnewinfo->pathnames[other_source_index] = oldpath;\n\nI'm happy your organization brought us to a clean place.\n\nThanks,\n-Stolee\n"},{"id":"412261","messageId":"fcc6b18e-c5eb-27a1-b6b5-4dcdd97e2b90@gmail.com","threadId":"54806","inReplyTo":"ff09ddb9caf73632c9792c07f1f7499a75a09606.1607962900.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 11/11] merge-ort: add implementation of type-changed rename handling","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-15T14:31:13Z","receivedAt":"2020-12-15T14:32:17Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Implement cases where renames are involved in type changes (i.e. the\n> side of history that didn't rename the file changed its type from a\n> regular file to a symlink or submodule). There was some code to handle\n> this in merge-recursive but only in the special case when the renamed\n> file had no content changes. The code here works differently -- it\n> knows process_entry() can handle mode conflicts, so it does a few\n> minimal tweaks to ensure process_entry() can just finish the job as\n> needed.\n> \n> Signed-off-by: Elijah Newren \n> ---\n> merge-ort.c | 33 +++++++++++++++++++++++++++++++--\n> 1 file changed, 31 insertions(+), 2 deletions(-)\n> \n> diff --git a/merge-ort.c b/merge-ort.c\n> index 9aac33c8e31..11e33f56edf 100644\n> --- a/merge-ort.c\n> +++ b/merge-ort.c\n> @@ -778,7 +778,32 @@ static int process_renames(struct merge_options *opt,\n> \t\t\t S_ISREG(newinfo->stages[target_index].mode));\n> \t\tif (type_changed && collision) {\n> \t\t\t/* special handling so later blocks can handle this */\n\nPerhaps drop this comment, or incorporate it into the lower one?\n\n> -\t\t\tdie(\"Not yet implemented\");\n> +\t\t\t/*\n> +\t\t\t * if type_changed && collision are both true, then this\n> +\t\t\t * was really a double rename, but one side wasn't\n> +\t\t\t * detected due to lack of break detection. I.e.\n> +\t\t\t * something like\n> +\t\t\t * orig: has normal file 'foo'\n> +\t\t\t * side1: renames 'foo' to 'bar', adds 'foo' symlink\n> +\t\t\t * side2: renames 'foo' to 'bar'\n> +\t\t\t * In this case, the foo->bar rename on side1 won't be\n> +\t\t\t * detected because the new symlink named 'foo' is\n> +\t\t\t * there and we don't do break detection. But we detect\n> +\t\t\t * this here because we don't want to merge the content\n> +\t\t\t * of the foo symlink with the foo->bar file, so we\n> +\t\t\t * have some logic to handle this special case. The\n> +\t\t\t * easiest way to do that is make 'bar' on side1 not\n> +\t\t\t * be considered a colliding file but the other part\n> +\t\t\t * of a normal rename. If the file is very different,\n> +\t\t\t * well we're going to get content merge conflicts\n> +\t\t\t * anyway so it doesn't hurt. And if the colliding\n> +\t\t\t * file also has a different type, that'll be handled\n> +\t\t\t * by the content merge logic in process_entry() too.\n> +\t\t\t *\n> +\t\t\t * See also t6430, 'rename vs. rename/symlink'\n\nI appreciate the callout to a test that exercises this behavior.\n\n> +\t\t\t */\n> +\t\t\tcollision = 0;\n> +\t\t}\n\nHere, we regain that closing curly brace, fixing the compiler errors from\nearlier.\n\n> \t\tif (source_deleted) {\n> \t\t\tif (target_index == 1) {\n> \t\t\t\trename_branch = opt->branch1;\n> @@ -858,7 +883,11 @@ static int process_renames(struct merge_options *opt,\n> \t\t\tnewinfo->pathnames[0] = oldpath;\n> \t\t\tif (type_changed) {\n> \t\t\t\t/* rename vs. typechange */\n> -\t\t\t\tdie(\"Not yet implemented\");\n> +\t\t\t\t/* Mark the original as resolved by removal */\n> +\t\t\t\tmemcpy(&oldinfo->stages[0].oid, &null_oid,\n> +\t\t\t\t sizeof(oldinfo->stages[0].oid));\n> +\t\t\t\toldinfo->stages[0].mode = 0;\n> +\t\t\t\toldinfo->filemask &= 0x06;\n\nThis matches your explanation in the comment above. I wonder if 0x06\ncould be less magical, but we are really deep in the weeds here already.\n\nThanks,\n-Stolee\n\n"},{"id":"412262","messageId":"6448806f-791b-610a-06de-7be5650334e6@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 00/11] merge-ort: add basic rename detection","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-12-15T14:34:01Z","receivedAt":"2020-12-15T14:34:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> This series builds on en/merge-ort-2 and adds basic rename detection to\n> merge-ort.\n\nI have now finished a full pass through this series. I find it to be\nwell organized with a satisfying conclusion. My comments are mostly\nabout nits. I tried to recommend some better organization, but failed\nto find a clear way to do so, which shows that the organization here\nis sound.\n\nMy only complaint is that some of the patches break compilation, so\nthat should definitely be fixed to assist with bisects in the future\n(likely for bisects unrelated to this feature).\n\nThanks,\n-Stolee\n\n"},{"id":"412267","messageId":"CABPp-BHiwFgTT+y35WEo3v44y6Ypo49di1P6+6A9DaLMTFK7yg@mail.gmail.com","threadId":"54806","inReplyTo":"48fc65a5-12ba-be67-2908-27a446b1f813@gmail.com","subject":"Re: [PATCH v2 08/11] merge-ort: add implementation of rename collisions","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-15T16:56:51Z","receivedAt":"2020-12-15T16:59:05Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Dec 15, 2020 at 6:09 AM Derrick Stolee wrote:\n>\n> On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > Implement rename/rename(2to1) and rename/add handling, i.e. a file is\n> > renamed into a location where another file is added (with that other\n> > file either being a plain add or itself coming from a rename). Note\n> > that rename collisions can also have a special case stacked on top: the\n> > file being renamed on one side of history is deleted on the other\n> > (yielding either a rename/add/delete conflict or perhaps a\n> > rename/rename(2to1)/delete[/delete]) conflict.\n> >\n> > One thing to note here is that when there is a double rename, the code\n> > in question only handles one of them at a time; a later iteration\n> > through the loop will handle the other. After they've both been\n> > handled, process_entry()'s normal add/add code can handle the collision.\n> >\n> > This code replaces the following from merge-recurisve.c:\n> >\n> > * all the 2to1 code in process_renames()\n> > * the RENAME_TWO_FILES_TO_ONE case of process_entry()\n> > * handle_rename_rename_2to1()\n> > * handle_rename_add()\n> >\n> > Also, there is some shared code from merge-recursive.c for multiple\n> > different rename cases which we will no longer need for this case (or\n> > other rename cases):\n> >\n> > * handle_file_collision()\n> > * setup_rename_conflict_info()\n> >\n> > The consolidation of six separate codepaths into one is made possible\n> > by a change in design: process_renames() tweaks the conflict_info\n> > entries within opt->priv->paths such that process_entry() can then\n> > handle all the non-rename conflict types (directory/file, modify/delete,\n> > etc.) orthogonally. This means we're much less likely to miss special\n> > implementation of some kind of combination of conflict types (see\n> > commits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n> > 2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\n> > directory/file conflict handling in ort\", 2020-10-26) for more details).\n> > That, together with letting worktree/index updating be handled\n> > orthogonally in the merge_switch_to_result() function, dramatically\n> > simplifies the code for various special rename cases.\n>\n> I'm really happy that you broke out the cases earlier, and describe\n> them so well in the message. It makes this hunk of code really easy\n> to understand:\n>\n> > + const char *pathnames[3];\n> > + struct version_info merged;\n> > +\n> > + struct conflict_info *base, *side1, *side2;\n> > + unsigned clean;\n> > +\n> > + pathnames[0] = oldpath;\n> > + pathnames[other_source_index] = oldpath;\n> > + pathnames[target_index] = newpath;\n> > +\n> > + base = strmap_get(&opt->priv->paths, pathnames[0]);\n> > + side1 = strmap_get(&opt->priv->paths, pathnames[1]);\n> > + side2 = strmap_get(&opt->priv->paths, pathnames[2]);\n> > +\n> > + VERIFY_CI(base);\n> > + VERIFY_CI(side1);\n> > + VERIFY_CI(side2);\n> > +\n> > + clean = handle_content_merge(opt, pair->one->path,\n> > + &base->stages[0],\n> > + &side1->stages[1],\n> > + &side2->stages[2],\n> > + pathnames,\n> > + 1 + 2*opt->priv->call_depth,\n>\n> nit: \" * \"\n\nWill fix.\n\n> > + &merged);\n> > +\n> > + memcpy(&newinfo->stages[target_index], &merged,\n> > + sizeof(merged));\n> > + if (!clean) {\n> > + path_msg(opt, newpath, 0,\n> > + _(\"CONFLICT (rename involved in \"\n> > + \"collision): rename of %s -> %s has \"\n> > + \"content conflicts AND collides \"\n> > + \"with another path; this may result \"\n> > + \"in nested conflict markers.\"),\n> > + oldpath, newpath);\n>\n> I was briefly taken aback by \"AND collides with another path\" wondering if\n> that wording helps users understand the type of conflict here. But I can't\n> think of anything better, so *shrug*.\n>\n> > + }\n> > } else if (collision && source_deleted) {\n> > - /* rename/add/delete or rename/rename(2to1)/delete */\n> > - die(\"Not yet implemented\");\n> > + /*\n> > + * rename/add/delete or rename/rename(2to1)/delete:\n> > + * since oldpath was deleted on the side that didn't\n> > + * do the rename, there's not much of a content merge\n> > + * we can do for the rename. oldinfo->merged.is_null\n> > + * was already set, so we just leave things as-is so\n> > + * they look like an add/add conflict.\n> > + */\n> > +\n> > + newinfo->path_conflict = 1;\n> > + path_msg(opt, newpath, 0,\n> > + _(\"CONFLICT (rename/delete): %s renamed \"\n> > + \"to %s in %s, but deleted in %s.\"),\n> > + oldpath, newpath, rename_branch, delete_branch);\n>\n> I think this branch is added in the wrong patch. My compiler is complaining\n> that 'rename_branch' and 'delete_branch' are not declared (yet).\n\nWhoops. This used to be separate patches, with the second half coming\nafter one of the later patches in the series. But in the commit\nmessage seemed most natural to talk about \"rename collisions\" which\nthen means both types of conflicts here. So I squashed them...and\nbroke the build. I'll rearrange this one to come after the\nrename/delete patch so that rename_branch and delete_branch will be\ndefined.\n"},{"id":"412268","messageId":"CABPp-BFbiszLAbWgF_qgHCxRgw4CBeiGnAHE8ErNYm4iAjiVXw@mail.gmail.com","threadId":"54806","inReplyTo":"05f52d13-d9ec-226d-370a-140937b12e7a@gmail.com","subject":"Re: [PATCH v2 09/11] merge-ort: add implementation of rename/delete conflicts","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-15T17:07:36Z","receivedAt":"2020-12-15T17:08:57Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Dec 15, 2020 at 6:23 AM Derrick Stolee wrote:\n>\n> On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > Implement rename/delete conflicts, i.e. one side renames a file and the\n> > other deletes the file. This code replaces the following from\n> > merge-recurisve.c:\n> >\n> > * the code relevant to RENAME_DELETE in process_renames()\n> > * the RENAME_DELETE case of process_entry()\n> > * handle_rename_delete()\n> >\n> > Also, there is some shared code from merge-recursive.c for multiple\n> > different rename cases which we will no longer need for this case (or\n> > other rename cases):\n> >\n> > * handle_change_delete()\n> > * setup_rename_conflict_info()\n> >\n> > The consolidation of five separate codepaths into one is made possible\n> > by a change in design: process_renames() tweaks the conflict_info\n> > entries within opt->priv->paths such that process_entry() can then\n> > handle all the non-rename conflict types (directory/file, modify/delete,\n> > etc.) orthogonally. This means we're much less likely to miss special\n> > implementation of some kind of combination of conflict types (see\n> > commits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n> > 2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\n> > directory/file conflict handling in ort\", 2020-10-26) for more details).\n> > That, together with letting worktree/index updating be handled\n> > orthogonally in the merge_switch_to_result() function, dramatically\n> > simplifies the code for various special rename cases.\n> >\n> > To be fair, there is a _slight_ tweak to process_entry() here, because\n> > rename/delete cases will also trigger the modify/delete codepath.\n> > However, we only want a modify/delete message to be printed for a\n> > rename/delete conflict if there is a content change in the renamed file\n> > in addition to the rename. So process_renames() and process_entry()\n> > aren't quite fully orthogonal, but they are pretty close.\n>\n> Thanks for adding this warning about the change to process_entry().\n>\n> > @@ -657,6 +657,7 @@ static int process_renames(struct merge_options *opt,\n> > unsigned int old_sidemask;\n> > int target_index, other_source_index;\n> > int source_deleted, collision, type_changed;\n> > + const char *rename_branch = NULL, *delete_branch = NULL;\n>\n> Ah, here they are!\n>\n> > + if (source_deleted) {\n> > + if (target_index == 1) {\n> > + rename_branch = opt->branch1;\n> > + delete_branch = opt->branch2;\n> > + } else {\n> > + rename_branch = opt->branch2;\n> > + delete_branch = opt->branch1;\n> > + }\n> > }\n> >\n> > assert(source_deleted || oldinfo->filemask & old_sidemask);\n> > @@ -838,13 +847,26 @@ static int process_renames(struct merge_options *opt,\n> > \"to %s in %s, but deleted in %s.\"),\n> > oldpath, newpath, rename_branch, delete_branch);\n>\n> This context line is the previous use of rename_branch and delete_branch.\n> Perhaps the declarations, initialization, and first-use here are worth\n> their own patch?\n>\n> > } else {\n> > + /*\n> > + * a few different cases...start by copying the\n> > + * existing stage(s) from oldinfo over the newinfo\n> > + * and update the pathname(s).\n> > + */\n> > + memcpy(&newinfo->stages[0], &oldinfo->stages[0],\n> > + sizeof(newinfo->stages[0]));\n> > + newinfo->filemask |= (1 << MERGE_BASE);\n> > + newinfo->pathnames[0] = oldpath;\n> > if (type_changed) {\n> > /* rename vs. typechange */\n> > die(\"Not yet implemented\");\n> > } else if (source_deleted) {\n> > /* rename/delete */\n> > + newinfo->path_conflict = 1;\n> > + path_msg(opt, newpath, 0,\n> > + _(\"CONFLICT (rename/delete): %s renamed\"\n> > + \" to %s in %s, but deleted in %s.\"),\n> > + oldpath, newpath,\n> > + rename_branch, delete_branch);\n>\n> Since the primary purpose of rename_branch and delete_branch appears to\n> be for these error messages, then likely the previous error message about\n> a rename/delete should just be promoted into this patch instead of the\n> previous.\n>\n> In fact, the error messages are the exact same, but with slightly different\n> lines due to wrapping:\n>\n> path_msg(opt, newpath, 0,\n> _(\"CONFLICT (rename/delete): %s renamed \"\n> \"to %s in %s, but deleted in %s.\"),\n> oldpath, newpath, rename_branch, delete_branch);\n>\n> and\n>\n> path_msg(opt, newpath, 0,\n> _(\"CONFLICT (rename/delete): %s renamed\"\n> \" to %s in %s, but deleted in %s.\"),\n> oldpath, newpath,\n> rename_branch, delete_branch);\n>\n> I wonder if there is a way to group these together? Perhaps the nested\n> if/else if/else blocks could store a \"conflict state\" value that says\n> which CONFLICT message to print after the complicated branching is done.\n>\n> Alternatively, this message appears to be written in the following case:\n>\n> source_deleted && !type_changed\n>\n> your if/else if/else block could be rearranged as follows:\n>\n> if (collision && !source_deleted)\n> /* collision: rename/add or rename/rename(2to1) */\n> else if (!type_change && source_deleted)\n> /* rename/delete or rename/add/delete or rename/rename(2to1)/delete */\n> else if (!collision)\n> /* a few different cases */\n>\n> Of course, the thing I am missing is that copy of oldinfo->stages[0] into\n> newinfo->stages[0] along with changes to the filemask and pathnames! That\n> is likely why you need the two different markers, because the cases truly\n> are different in that subtle way.\n\nYeah, there is that subtlety and another one -- the rename/add/delete\ncase will also later trigger the \"add/add\" conflict type within\nprocess_entries() for this same path, whereas the rename/delete case\nfrom this patch won't. The combination is enough of a difference that\nI'm worried that trying to make both types run through the same code\nblock might blur the differences and pose a landmine for future folks\ncoming to edit the code; it'd make it too easy to break one or the\nother conflict type.\n\nIf the sharing was done a different way, namely saving the basic\nmessage in some variable before either if-block and then both places\njust pass that string to path_msg() instead of both having it typed\nout, then that'd probably make sense, but then we're not really saving\nmuch.\n\n> > /* normal rename */\n> > die(\"Not yet implemented\");\n> > @@ -1380,12 +1402,21 @@ static void process_entry(struct merge_options *opt,\n> > modify_branch = (side == 1) ? opt->branch1 : opt->branch2;\n> > delete_branch = (side == 1) ? opt->branch2 : opt->branch1;\n> >\n> > - path_msg(opt, path, 0,\n> > - _(\"CONFLICT (modify/delete): %s deleted in %s \"\n> > - \"and modified in %s. Version %s of %s left \"\n> > - \"in tree.\"),\n> > - path, delete_branch, modify_branch,\n> > - modify_branch, path);\n> > + if (ci->path_conflict &&\n> > + oideq(&ci->stages[0].oid, &ci->stages[side].oid)) {\n> > + /*\n> > + * This came from a rename/delete; no action to take,\n> > + * but avoid printing \"modify/delete\" conflict notice\n> > + * since the contents were not modified.\n> > + */\n> > + } else {\n> > + path_msg(opt, path, 0,\n> > + _(\"CONFLICT (modify/delete): %s deleted in %s \"\n> > + \"and modified in %s. Version %s of %s left \"\n> > + \"in tree.\"),\n> > + path, delete_branch, modify_branch,\n> > + modify_branch, path);\n> > + }\n>\n> Thanks,\n> -Stolee\n"},{"id":"412269","messageId":"CABPp-BGSCeffT-VnniGXM8D8BC=T6YWHr4ei2ij+1-+0ghg8OA@mail.gmail.com","threadId":"54806","inReplyTo":"fcc6b18e-c5eb-27a1-b6b5-4dcdd97e2b90@gmail.com","subject":"Re: [PATCH v2 11/11] merge-ort: add implementation of type-changed rename handling","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-12-15T17:11:04Z","receivedAt":"2020-12-15T17:13:14Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Dec 15, 2020 at 6:31 AM Derrick Stolee wrote:\n>\n> On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n> > From: Elijah Newren \n> >\n> > Implement cases where renames are involved in type changes (i.e. the\n> > side of history that didn't rename the file changed its type from a\n> > regular file to a symlink or submodule). There was some code to handle\n> > this in merge-recursive but only in the special case when the renamed\n> > file had no content changes. The code here works differently -- it\n> > knows process_entry() can handle mode conflicts, so it does a few\n> > minimal tweaks to ensure process_entry() can just finish the job as\n> > needed.\n> >\n> > Signed-off-by: Elijah Newren \n> > ---\n> > merge-ort.c | 33 +++++++++++++++++++++++++++++++--\n> > 1 file changed, 31 insertions(+), 2 deletions(-)\n> >\n> > diff --git a/merge-ort.c b/merge-ort.c\n> > index 9aac33c8e31..11e33f56edf 100644\n> > --- a/merge-ort.c\n> > +++ b/merge-ort.c\n> > @@ -778,7 +778,32 @@ static int process_renames(struct merge_options *opt,\n> > S_ISREG(newinfo->stages[target_index].mode));\n> > if (type_changed && collision) {\n> > /* special handling so later blocks can handle this */\n>\n> Perhaps drop this comment, or incorporate it into the lower one?\n\nWill do.\n\n> > - die(\"Not yet implemented\");\n> > + /*\n> > + * if type_changed && collision are both true, then this\n> > + * was really a double rename, but one side wasn't\n> > + * detected due to lack of break detection. I.e.\n> > + * something like\n> > + * orig: has normal file 'foo'\n> > + * side1: renames 'foo' to 'bar', adds 'foo' symlink\n> > + * side2: renames 'foo' to 'bar'\n> > + * In this case, the foo->bar rename on side1 won't be\n> > + * detected because the new symlink named 'foo' is\n> > + * there and we don't do break detection. But we detect\n> > + * this here because we don't want to merge the content\n> > + * of the foo symlink with the foo->bar file, so we\n> > + * have some logic to handle this special case. The\n> > + * easiest way to do that is make 'bar' on side1 not\n> > + * be considered a colliding file but the other part\n> > + * of a normal rename. If the file is very different,\n> > + * well we're going to get content merge conflicts\n> > + * anyway so it doesn't hurt. And if the colliding\n> > + * file also has a different type, that'll be handled\n> > + * by the content merge logic in process_entry() too.\n> > + *\n> > + * See also t6430, 'rename vs. rename/symlink'\n>\n> I appreciate the callout to a test that exercises this behavior.\n>\n> > + */\n> > + collision = 0;\n> > + }\n>\n> Here, we regain that closing curly brace, fixing the compiler errors from\n> earlier.\n\nSo embarrassing. I was pretty sure I tested the individual patches,\nbut maybe I somehow missed this series?? Anyway, yeah, I'll fix it\nup.\n\n>\n> > if (source_deleted) {\n> > if (target_index == 1) {\n> > rename_branch = opt->branch1;\n> > @@ -858,7 +883,11 @@ static int process_renames(struct merge_options *opt,\n> > newinfo->pathnames[0] = oldpath;\n> > if (type_changed) {\n> > /* rename vs. typechange */\n> > - die(\"Not yet implemented\");\n> > + /* Mark the original as resolved by removal */\n> > + memcpy(&oldinfo->stages[0].oid, &null_oid,\n> > + sizeof(oldinfo->stages[0].oid));\n> > + oldinfo->stages[0].mode = 0;\n> > + oldinfo->filemask &= 0x06;\n>\n> This matches your explanation in the comment above. I wonder if 0x06\n> could be less magical, but we are really deep in the weeds here already.\n>\n> Thanks,\n> -Stolee\n>\n"},{"id":"412278","messageId":"78621ca07887e39a41ad9cff7b8dd5dcc9b33fbc.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 01/11] merge-ort: add basic data structures for handling renames","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:27:56Z","receivedAt":"2020-12-15T18:28:52Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nThis will grow later, but we only need a few fields for basic rename\nhandling.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 24 ++++++++++++++++++++++++\n 1 file changed, 24 insertions(+)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 414e7b7eeac..1c1a7fa4bf1 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -46,6 +46,25 @@ enum merge_side {\n \tMERGE_SIDE2 = 2\n };\n \n+struct rename_info {\n+\t/*\n+\t * pairs: pairing of filenames from diffcore_rename()\n+\t *\n+\t * Index 1 and 2 correspond to sides 1 & 2 as used in\n+\t * conflict_info.stages. Index 0 unused.\n+\t */\n+\tstruct diff_queue_struct pairs[3];\n+\n+\t/*\n+\t * needed_limit: value needed for inexact rename detection to run\n+\t *\n+\t * If the current rename limit wasn't high enough for inexact\n+\t * rename detection to run, this records the limit needed. Otherwise,\n+\t * this value remains 0.\n+\t */\n+\tint needed_limit;\n+};\n+\n struct merge_options_internal {\n \t/*\n \t * paths: primary data structure in all of merge ort.\n@@ -113,6 +132,11 @@ struct merge_options_internal {\n \t */\n \tstruct strmap output;\n \n+\t/*\n+\t * renames: various data relating to rename detection\n+\t */\n+\tstruct rename_info renames;\n+\n \t/*\n \t * current_dir_name: temporary var used in collect_merge_info_callback()\n \t *\n-- \ngitgitgadget\n\n"},{"id":"412279","messageId":"d846decf40be829590b88bb61fb19ae6f64c3162.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 02/11] merge-ort: add initial outline for basic rename detection","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:27:57Z","receivedAt":"2020-12-15T18:28:57Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++-------\n 1 file changed, 60 insertions(+), 8 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 1c1a7fa4bf1..8552f5e2318 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -644,20 +644,72 @@ static int handle_content_merge(struct merge_options *opt,\n \n /*** Function Grouping: functions related to regular rename detection ***/\n \n+static int process_renames(struct merge_options *opt,\n+\t\t\t struct diff_queue_struct *renames)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+static int compare_pairs(const void *a_, const void *b_)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+/* Call diffcore_rename() to compute which files have changed on given side */\n+static void detect_regular_renames(struct merge_options *opt,\n+\t\t\t\t struct tree *merge_base,\n+\t\t\t\t struct tree *side,\n+\t\t\t\t unsigned side_index)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n+/*\n+ * Get information of all renames which occurred in 'side_pairs', discarding\n+ * non-renames.\n+ */\n+static int collect_renames(struct merge_options *opt,\n+\t\t\t struct diff_queue_struct *result,\n+\t\t\t unsigned side_index)\n+{\n+\tdie(\"Not yet implemented.\");\n+}\n+\n static int detect_and_process_renames(struct merge_options *opt,\n \t\t\t\t struct tree *merge_base,\n \t\t\t\t struct tree *side1,\n \t\t\t\t struct tree *side2)\n {\n-\tint clean = 1;\n+\tstruct diff_queue_struct combined;\n+\tstruct rename_info *renames = &opt->priv->renames;\n+\tint s, clean = 1;\n+\n+\tmemset(&combined, 0, sizeof(combined));\n+\n+\tdetect_regular_renames(opt, merge_base, side1, MERGE_SIDE1);\n+\tdetect_regular_renames(opt, merge_base, side2, MERGE_SIDE2);\n+\n+\tALLOC_GROW(combined.queue,\n+\t\t renames->pairs[1].nr + renames->pairs[2].nr,\n+\t\t combined.alloc);\n+\tclean &= collect_renames(opt, &combined, MERGE_SIDE1);\n+\tclean &= collect_renames(opt, &combined, MERGE_SIDE2);\n+\tQSORT(combined.queue, combined.nr, compare_pairs);\n+\n+\tclean &= process_renames(opt, &combined);\n+\n+\t/* Free memory for renames->pairs[] and combined */\n+\tfor (s = MERGE_SIDE1; s <= MERGE_SIDE2; s++) {\n+\t\tfree(renames->pairs[s].queue);\n+\t\tDIFF_QUEUE_CLEAR(&renames->pairs[s]);\n+\t}\n+\tif (combined.nr) {\n+\t\tint i;\n+\t\tfor (i = 0; i < combined.nr; i++)\n+\t\t\tdiff_free_filepair(combined.queue[i]);\n+\t\tfree(combined.queue);\n+\t}\n \n-\t/*\n-\t * Rename detection works by detecting file similarity. Here we use\n-\t * a really easy-to-implement scheme: files are similar IFF they have\n-\t * the same filename. Therefore, by this scheme, there are no renames.\n-\t *\n-\t * TODO: Actually implement a real rename detection scheme.\n-\t */\n \treturn clean;\n }\n \n-- \ngitgitgadget\n\n"},{"id":"412280","messageId":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v2.git.1607962900.gitgitgadget@gmail.com","subject":"[PATCH v3 00/11] merge-ort: add basic rename detection","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:27:55Z","receivedAt":"2020-12-15T18:28:57Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This series builds on en/merge-ort-2 and adds basic rename detection to\nmerge-ort.\n\nChanges since v2 (all due to feedback from Stolee's reviews):\n\n * reordered two of the patches (one depended on vars declared in another)\n * a few other adjustments to make patches individually compile (I usually\n check this; so embarrassing that I somehow missed it)\n\nElijah Newren (11):\n merge-ort: add basic data structures for handling renames\n merge-ort: add initial outline for basic rename detection\n merge-ort: implement detect_regular_renames()\n merge-ort: implement compare_pairs() and collect_renames()\n merge-ort: add basic outline for process_renames()\n merge-ort: add implementation of both sides renaming identically\n merge-ort: add implementation of both sides renaming differently\n merge-ort: add implementation of rename/delete conflicts\n merge-ort: add implementation of rename collisions\n merge-ort: add implementation of normal rename handling\n merge-ort: add implementation of type-changed rename handling\n\n merge-ort.c | 446 ++++++++++++++++++++++++++++++++++++++++++++++++++--\n 1 file changed, 430 insertions(+), 16 deletions(-)\n\n\nbase-commit: c5a6f65527aa3b6f5d7cf25437a88d8727ab0646\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-812%2Fnewren%2Fort-renames-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-812/newren/ort-renames-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/812\n\nRange-diff vs v2:\n\n 1: 78621ca0788 = 1: 78621ca0788 merge-ort: add basic data structures for handling renames\n 2: d846decf40b = 2: d846decf40b merge-ort: add initial outline for basic rename detection\n 3: 620fc64032d = 3: 620fc64032d merge-ort: implement detect_regular_renames()\n 4: 9382dc4d50b = 4: 9382dc4d50b merge-ort: implement compare_pairs() and collect_renames()\n 5: d20fab8d403 = 5: d20fab8d403 merge-ort: add basic outline for process_renames()\n 6: 15fff3dd0c4 ! 6: 7ec51feb418 merge-ort: add implementation of both sides renaming identically\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n \t\t\tconst char *pathnames[3];\n +\t\t\tstruct version_info merged;\n +\t\t\tstruct conflict_info *base, *side1, *side2;\n -+\t\t\tunsigned was_binary_blob = 0;\n \n \t\t\tpathnames[0] = oldpath;\n \t\t\tpathnames[1] = newpath;\n 7: d00e26be784 ! 7: d37e2626c30 merge-ort: add implementation of both sides renaming differently\n @@ Commit message\n Signed-off-by: Elijah Newren \n \n ## merge-ort.c ##\n +@@ merge-ort.c: static int process_renames(struct merge_options *opt,\n + \t\t\tconst char *pathnames[3];\n + \t\t\tstruct version_info merged;\n + \t\t\tstruct conflict_info *base, *side1, *side2;\n ++\t\t\tunsigned was_binary_blob = 0;\n + \n + \t\t\tpathnames[0] = oldpath;\n + \t\t\tpathnames[1] = newpath;\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n \t\t\t}\n \n 9: f017534243c ! 8: 6b79da5e8a4 merge-ort: add implementation of rename/delete conflicts\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n \t\told_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n \t\toldpath = old_ent->key;\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n - \t\tif (type_changed && collision) {\n \t\t\t/* special handling so later blocks can handle this */\n \t\t\tdie(\"Not yet implemented\");\n + \t\t}\n +\t\tif (source_deleted) {\n +\t\t\tif (target_index == 1) {\n +\t\t\t\trename_branch = opt->branch1;\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n +\t\t\t\trename_branch = opt->branch2;\n +\t\t\t\tdelete_branch = opt->branch1;\n +\t\t\t}\n - \t\t}\n ++\t\t}\n \n \t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n + \n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n - \t\t\t\t \"to %s in %s, but deleted in %s.\"),\n - \t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n + \t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n + \t\t\tdie(\"Not yet implemented\");\n \t\t} else {\n -\t\t\t/* a few different cases... */\n +\t\t\t/*\n 8: edd610321a0 ! 9: 065fc0396dc merge-ort: add implementation of rename collisions\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n +\t\t\t\t\t\t &side1->stages[1],\n +\t\t\t\t\t\t &side2->stages[2],\n +\t\t\t\t\t\t pathnames,\n -+\t\t\t\t\t\t 1 + 2*opt->priv->call_depth,\n ++\t\t\t\t\t\t 1 + 2 * opt->priv->call_depth,\n +\t\t\t\t\t\t &merged);\n +\n +\t\t\tmemcpy(&newinfo->stages[target_index], &merged,\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n +\t\t\t\t \"to %s in %s, but deleted in %s.\"),\n +\t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n \t\t} else {\n - \t\t\t/* a few different cases... */\n - \t\t\tif (type_changed) {\n + \t\t\t/*\n + \t\t\t * a few different cases...start by copying the\n 10: 22cb7110261 = 10: 73426c16687 merge-ort: add implementation of normal rename handling\n 11: ff09ddb9caf ! 11: 8f4662398ab merge-ort: add implementation of type-changed rename handling\n @@ Commit message\n \n ## merge-ort.c ##\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n + \t\t\t(S_ISREG(oldinfo->stages[other_source_index].mode) !=\n \t\t\t S_ISREG(newinfo->stages[target_index].mode));\n \t\tif (type_changed && collision) {\n - \t\t\t/* special handling so later blocks can handle this */\n +-\t\t\t/* special handling so later blocks can handle this */\n -\t\t\tdie(\"Not yet implemented\");\n +\t\t\t/*\n ++\t\t\t * special handling so later blocks can handle this...\n ++\t\t\t *\n +\t\t\t * if type_changed && collision are both true, then this\n +\t\t\t * was really a double rename, but one side wasn't\n +\t\t\t * detected due to lack of break detection. I.e.\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n +\t\t\t * See also t6430, 'rename vs. rename/symlink'\n +\t\t\t */\n +\t\t\tcollision = 0;\n -+\t\t}\n + \t\t}\n \t\tif (source_deleted) {\n \t\t\tif (target_index == 1) {\n - \t\t\t\trename_branch = opt->branch1;\n @@ merge-ort.c: static int process_renames(struct merge_options *opt,\n \t\t\tnewinfo->pathnames[0] = oldpath;\n \t\t\tif (type_changed) {\n\n-- \ngitgitgadget\n"},{"id":"412281","messageId":"620fc64032d26e374b2fb52a044564725e023f0f.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 03/11] merge-ort: implement detect_regular_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:27:58Z","receivedAt":"2020-12-15T18:29:19Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nBased heavily on merge-recursive's get_diffpairs() function, and also\nincludes the necessary paired call to diff_warn_rename_limit() so that\nusers will be warned if merge.renameLimit is not sufficiently large for\nrename detection to run.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 32 +++++++++++++++++++++++++++++++-\n 1 file changed, 31 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 8552f5e2318..66f84d39b43 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -661,7 +661,33 @@ static void detect_regular_renames(struct merge_options *opt,\n \t\t\t\t struct tree *side,\n \t\t\t\t unsigned side_index)\n {\n-\tdie(\"Not yet implemented.\");\n+\tstruct diff_options diff_opts;\n+\tstruct rename_info *renames = &opt->priv->renames;\n+\n+\trepo_diff_setup(opt->repo, &diff_opts);\n+\tdiff_opts.flags.recursive = 1;\n+\tdiff_opts.flags.rename_empty = 0;\n+\tdiff_opts.detect_rename = DIFF_DETECT_RENAME;\n+\tdiff_opts.rename_limit = opt->rename_limit;\n+\tif (opt->rename_limit <= 0)\n+\t\tdiff_opts.rename_limit = 1000;\n+\tdiff_opts.rename_score = opt->rename_score;\n+\tdiff_opts.show_rename_progress = opt->show_rename_progress;\n+\tdiff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n+\tdiff_setup_done(&diff_opts);\n+\tdiff_tree_oid(&merge_base->object.oid, &side->object.oid, \"\",\n+\t\t &diff_opts);\n+\tdiffcore_std(&diff_opts);\n+\n+\tif (diff_opts.needed_rename_limit > renames->needed_limit)\n+\t\trenames->needed_limit = diff_opts.needed_rename_limit;\n+\n+\trenames->pairs[side_index] = diff_queued_diff;\n+\n+\tdiff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;\n+\tdiff_queued_diff.nr = 0;\n+\tdiff_queued_diff.queue = NULL;\n+\tdiff_flush(&diff_opts);\n }\n \n /*\n@@ -1406,6 +1432,10 @@ void merge_switch_to_result(struct merge_options *opt,\n \t\t\tprintf(\"%s\", sb->buf);\n \t\t}\n \t\tstring_list_clear(&olist, 0);\n+\n+\t\t/* Also include needed rename limit adjustment now */\n+\t\tdiff_warn_rename_limit(\"merge.renamelimit\",\n+\t\t\t\t opti->renames.needed_limit, 0);\n \t}\n \n \tmerge_finalize(opt, result);\n-- \ngitgitgadget\n\n"},{"id":"412282","messageId":"d20fab8d403851ff9247d8b6bb950927f72b2729.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 05/11] merge-ort: add basic outline for process_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:28:00Z","receivedAt":"2020-12-15T18:29:29Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nAdd code which determines which kind of special rename case each rename\ncorresponds to, but leave the handling of each type unimplemented for\nnow. Future commits will implement each one.\n\nThere is some tenuous resemblance to merge-recursive's\nprocess_renames(), but comparing the two is very unlikely to yield any\ninsights. merge-ort's process_renames() is a bit complex and I would\nprefer if I could simplify it more, but it is far easier to grok than\nmerge-recursive's function of the same name in my opinion. Plus,\nmerge-ort handles more rename conflict types than merge-recursive does.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 97 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 10550c542b8..ebe275ef73c 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -647,7 +647,103 @@ static int handle_content_merge(struct merge_options *opt,\n static int process_renames(struct merge_options *opt,\n \t\t\t struct diff_queue_struct *renames)\n {\n-\tdie(\"Not yet implemented.\");\n+\tint clean_merge = 1, i;\n+\n+\tfor (i = 0; i < renames->nr; ++i) {\n+\t\tconst char *oldpath = NULL, *newpath;\n+\t\tstruct diff_filepair *pair = renames->queue[i];\n+\t\tstruct conflict_info *oldinfo = NULL, *newinfo = NULL;\n+\t\tstruct strmap_entry *old_ent, *new_ent;\n+\t\tunsigned int old_sidemask;\n+\t\tint target_index, other_source_index;\n+\t\tint source_deleted, collision, type_changed;\n+\n+\t\told_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n+\t\toldpath = old_ent->key;\n+\t\toldinfo = old_ent->value;\n+\n+\t\tnew_ent = strmap_get_entry(&opt->priv->paths, pair->two->path);\n+\t\tnewpath = new_ent->key;\n+\t\tnewinfo = new_ent->value;\n+\n+\t\t/*\n+\t\t * diff_filepairs have copies of pathnames, thus we have to\n+\t\t * use standard 'strcmp()' (negated) instead of '=='.\n+\t\t */\n+\t\tif (i + 1 < renames->nr &&\n+\t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n+\t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n+\t\t\tconst char *pathnames[3];\n+\n+\t\t\tpathnames[0] = oldpath;\n+\t\t\tpathnames[1] = newpath;\n+\t\t\tpathnames[2] = renames->queue[i+1]->two->path;\n+\n+\t\t\tif (!strcmp(pathnames[1], pathnames[2])) {\n+\t\t\t\t/* Both sides renamed the same way. */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\n+\t\t\t\t/* We handled both renames, i.e. i+1 handled */\n+\t\t\t\ti++;\n+\t\t\t\t/* Move to next rename */\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\t/* This is a rename/rename(1to2) */\n+\t\t\tdie(\"Not yet implemented\");\n+\n+\t\t\ti++; /* We handled both renames, i.e. i+1 handled */\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tVERIFY_CI(oldinfo);\n+\t\tVERIFY_CI(newinfo);\n+\t\ttarget_index = pair->score; /* from collect_renames() */\n+\t\tassert(target_index == 1 || target_index == 2);\n+\t\tother_source_index = 3 - target_index;\n+\t\told_sidemask = (1 << other_source_index); /* 2 or 4 */\n+\t\tsource_deleted = (oldinfo->filemask == 1);\n+\t\tcollision = ((newinfo->filemask & old_sidemask) != 0);\n+\t\ttype_changed = !source_deleted &&\n+\t\t\t(S_ISREG(oldinfo->stages[other_source_index].mode) !=\n+\t\t\t S_ISREG(newinfo->stages[target_index].mode));\n+\t\tif (type_changed && collision) {\n+\t\t\t/* special handling so later blocks can handle this */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t}\n+\n+\t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n+\n+\t\t/* Need to check for special types of rename conflicts... */\n+\t\tif (collision && !source_deleted) {\n+\t\t\t/* collision: rename/add or rename/rename(2to1) */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t} else if (collision && source_deleted) {\n+\t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n+\t\t\tdie(\"Not yet implemented\");\n+\t\t} else {\n+\t\t\t/* a few different cases... */\n+\t\t\tif (type_changed) {\n+\t\t\t\t/* rename vs. typechange */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t} else if (source_deleted) {\n+\t\t\t\t/* rename/delete */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t} else {\n+\t\t\t\t/* normal rename */\n+\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!type_changed) {\n+\t\t\t/* Mark the original as resolved by removal */\n+\t\t\toldinfo->merged.is_null = 1;\n+\t\t\toldinfo->merged.clean = 1;\n+\t\t}\n+\n+\t}\n+\n+\treturn clean_merge;\n }\n \n static int compare_pairs(const void *a_, const void *b_)\n-- \ngitgitgadget\n\n"},{"id":"412283","messageId":"9382dc4d50b3cf675d607cf5d39be08542d8147e.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 04/11] merge-ort: implement compare_pairs() and collect_renames()","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:27:59Z","receivedAt":"2020-12-15T18:29:35Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 35 +++++++++++++++++++++++++++++++++--\n 1 file changed, 33 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 66f84d39b43..10550c542b8 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -652,7 +652,10 @@ static int process_renames(struct merge_options *opt,\n \n static int compare_pairs(const void *a_, const void *b_)\n {\n-\tdie(\"Not yet implemented.\");\n+\tconst struct diff_filepair *a = *((const struct diff_filepair **)a_);\n+\tconst struct diff_filepair *b = *((const struct diff_filepair **)b_);\n+\n+\treturn strcmp(a->one->path, b->one->path);\n }\n \n /* Call diffcore_rename() to compute which files have changed on given side */\n@@ -698,7 +701,35 @@ static int collect_renames(struct merge_options *opt,\n \t\t\t struct diff_queue_struct *result,\n \t\t\t unsigned side_index)\n {\n-\tdie(\"Not yet implemented.\");\n+\tint i, clean = 1;\n+\tstruct diff_queue_struct *side_pairs;\n+\tstruct rename_info *renames = &opt->priv->renames;\n+\n+\tside_pairs = &renames->pairs[side_index];\n+\n+\tfor (i = 0; i < side_pairs->nr; ++i) {\n+\t\tstruct diff_filepair *p = side_pairs->queue[i];\n+\n+\t\tif (p->status != 'R') {\n+\t\t\tdiff_free_filepair(p);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/*\n+\t\t * p->score comes back from diffcore_rename_extended() with\n+\t\t * the similarity of the renamed file. The similarity is\n+\t\t * was used to determine that the two files were related\n+\t\t * and are a rename, which we have already used, but beyond\n+\t\t * that we have no use for the similarity. So p->score is\n+\t\t * now irrelevant. However, process_renames() will need to\n+\t\t * know which side of the merge this rename was associated\n+\t\t * with, so overwrite p->score with that value.\n+\t\t */\n+\t\tp->score = side_index;\n+\t\tresult->queue[result->nr++] = p;\n+\t}\n+\n+\treturn clean;\n }\n \n static int detect_and_process_renames(struct merge_options *opt,\n-- \ngitgitgadget\n\n"},{"id":"412284","messageId":"8f4662398ab4421a466b88b8984bd047528f31f1.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 11/11] merge-ort: add implementation of type-changed rename handling","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:28:06Z","receivedAt":"2020-12-15T18:30:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement cases where renames are involved in type changes (i.e. the\nside of history that didn't rename the file changed its type from a\nregular file to a symlink or submodule). There was some code to handle\nthis in merge-recursive but only in the special case when the renamed\nfile had no content changes. The code here works differently -- it\nknows process_entry() can handle mode conflicts, so it does a few\nminimal tweaks to ensure process_entry() can just finish the job as\nneeded.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 35 ++++++++++++++++++++++++++++++++---\n 1 file changed, 32 insertions(+), 3 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 26f357e524f..677c6a878c5 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -777,8 +777,33 @@ static int process_renames(struct merge_options *opt,\n \t\t\t(S_ISREG(oldinfo->stages[other_source_index].mode) !=\n \t\t\t S_ISREG(newinfo->stages[target_index].mode));\n \t\tif (type_changed && collision) {\n-\t\t\t/* special handling so later blocks can handle this */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\t/*\n+\t\t\t * special handling so later blocks can handle this...\n+\t\t\t *\n+\t\t\t * if type_changed && collision are both true, then this\n+\t\t\t * was really a double rename, but one side wasn't\n+\t\t\t * detected due to lack of break detection. I.e.\n+\t\t\t * something like\n+\t\t\t * orig: has normal file 'foo'\n+\t\t\t * side1: renames 'foo' to 'bar', adds 'foo' symlink\n+\t\t\t * side2: renames 'foo' to 'bar'\n+\t\t\t * In this case, the foo->bar rename on side1 won't be\n+\t\t\t * detected because the new symlink named 'foo' is\n+\t\t\t * there and we don't do break detection. But we detect\n+\t\t\t * this here because we don't want to merge the content\n+\t\t\t * of the foo symlink with the foo->bar file, so we\n+\t\t\t * have some logic to handle this special case. The\n+\t\t\t * easiest way to do that is make 'bar' on side1 not\n+\t\t\t * be considered a colliding file but the other part\n+\t\t\t * of a normal rename. If the file is very different,\n+\t\t\t * well we're going to get content merge conflicts\n+\t\t\t * anyway so it doesn't hurt. And if the colliding\n+\t\t\t * file also has a different type, that'll be handled\n+\t\t\t * by the content merge logic in process_entry() too.\n+\t\t\t *\n+\t\t\t * See also t6430, 'rename vs. rename/symlink'\n+\t\t\t */\n+\t\t\tcollision = 0;\n \t\t}\n \t\tif (source_deleted) {\n \t\t\tif (target_index == 1) {\n@@ -859,7 +884,11 @@ static int process_renames(struct merge_options *opt,\n \t\t\tnewinfo->pathnames[0] = oldpath;\n \t\t\tif (type_changed) {\n \t\t\t\t/* rename vs. typechange */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\t/* Mark the original as resolved by removal */\n+\t\t\t\tmemcpy(&oldinfo->stages[0].oid, &null_oid,\n+\t\t\t\t sizeof(oldinfo->stages[0].oid));\n+\t\t\t\toldinfo->stages[0].mode = 0;\n+\t\t\t\toldinfo->filemask &= 0x06;\n \t\t\t} else if (source_deleted) {\n \t\t\t\t/* rename/delete */\n \t\t\t\tnewinfo->path_conflict = 1;\n-- \ngitgitgadget\n"},{"id":"412285","messageId":"73426c166877194d3c53a2ea250751318d223d21.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 10/11] merge-ort: add implementation of normal rename handling","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:28:05Z","receivedAt":"2020-12-15T18:31:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement handling of normal renames. This code replaces the following\nfrom merge-recurisve.c:\n\n * the code relevant to RENAME_NORMAL in process_renames()\n * the RENAME_NORMAL case of process_entry()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_rename_normal()\n * setup_rename_conflict_info()\n\nThe consolidation of four separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\n(To be fair, the code for handling normal renames wasn't all that\ncomplicated beforehand, but it's still much simpler now.)\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 6 +++++-\n 1 file changed, 5 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 1c5b2f7e3b9..26f357e524f 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -870,7 +870,11 @@ static int process_renames(struct merge_options *opt,\n \t\t\t\t\t rename_branch, delete_branch);\n \t\t\t} else {\n \t\t\t\t/* normal rename */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\tmemcpy(&newinfo->stages[other_source_index],\n+\t\t\t\t &oldinfo->stages[other_source_index],\n+\t\t\t\t sizeof(newinfo->stages[0]));\n+\t\t\t\tnewinfo->filemask |= (1 << other_source_index);\n+\t\t\t\tnewinfo->pathnames[other_source_index] = oldpath;\n \t\t\t}\n \t\t}\n \n-- \ngitgitgadget\n\n"},{"id":"412286","messageId":"065fc0396dcb0dac080fd921209f77b824eee633.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 09/11] merge-ort: add implementation of rename collisions","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:28:04Z","receivedAt":"2020-12-15T18:31:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(2to1) and rename/add handling, i.e. a file is\nrenamed into a location where another file is added (with that other\nfile either being a plain add or itself coming from a rename). Note\nthat rename collisions can also have a special case stacked on top: the\nfile being renamed on one side of history is deleted on the other\n(yielding either a rename/add/delete conflict or perhaps a\nrename/rename(2to1)/delete[/delete]) conflict.\n\nOne thing to note here is that when there is a double rename, the code\nin question only handles one of them at a time; a later iteration\nthrough the loop will handle the other. After they've both been\nhandled, process_entry()'s normal add/add code can handle the collision.\n\nThis code replaces the following from merge-recurisve.c:\n\n * all the 2to1 code in process_renames()\n * the RENAME_TWO_FILES_TO_ONE case of process_entry()\n * handle_rename_rename_2to1()\n * handle_rename_add()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_file_collision()\n * setup_rename_conflict_info()\n\nThe consolidation of six separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++---\n 1 file changed, 51 insertions(+), 3 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex a10c3f5046f..1c5b2f7e3b9 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -795,10 +795,58 @@ static int process_renames(struct merge_options *opt,\n \t\t/* Need to check for special types of rename conflicts... */\n \t\tif (collision && !source_deleted) {\n \t\t\t/* collision: rename/add or rename/rename(2to1) */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\tconst char *pathnames[3];\n+\t\t\tstruct version_info merged;\n+\n+\t\t\tstruct conflict_info *base, *side1, *side2;\n+\t\t\tunsigned clean;\n+\n+\t\t\tpathnames[0] = oldpath;\n+\t\t\tpathnames[other_source_index] = oldpath;\n+\t\t\tpathnames[target_index] = newpath;\n+\n+\t\t\tbase = strmap_get(&opt->priv->paths, pathnames[0]);\n+\t\t\tside1 = strmap_get(&opt->priv->paths, pathnames[1]);\n+\t\t\tside2 = strmap_get(&opt->priv->paths, pathnames[2]);\n+\n+\t\t\tVERIFY_CI(base);\n+\t\t\tVERIFY_CI(side1);\n+\t\t\tVERIFY_CI(side2);\n+\n+\t\t\tclean = handle_content_merge(opt, pair->one->path,\n+\t\t\t\t\t\t &base->stages[0],\n+\t\t\t\t\t\t &side1->stages[1],\n+\t\t\t\t\t\t &side2->stages[2],\n+\t\t\t\t\t\t pathnames,\n+\t\t\t\t\t\t 1 + 2 * opt->priv->call_depth,\n+\t\t\t\t\t\t &merged);\n+\n+\t\t\tmemcpy(&newinfo->stages[target_index], &merged,\n+\t\t\t sizeof(merged));\n+\t\t\tif (!clean) {\n+\t\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t\t _(\"CONFLICT (rename involved in \"\n+\t\t\t\t\t \"collision): rename of %s -> %s has \"\n+\t\t\t\t\t \"content conflicts AND collides \"\n+\t\t\t\t\t \"with another path; this may result \"\n+\t\t\t\t\t \"in nested conflict markers.\"),\n+\t\t\t\t\t oldpath, newpath);\n+\t\t\t}\n \t\t} else if (collision && source_deleted) {\n-\t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\t/*\n+\t\t\t * rename/add/delete or rename/rename(2to1)/delete:\n+\t\t\t * since oldpath was deleted on the side that didn't\n+\t\t\t * do the rename, there's not much of a content merge\n+\t\t\t * we can do for the rename. oldinfo->merged.is_null\n+\t\t\t * was already set, so we just leave things as-is so\n+\t\t\t * they look like an add/add conflict.\n+\t\t\t */\n+\n+\t\t\tnewinfo->path_conflict = 1;\n+\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed \"\n+\t\t\t\t \"to %s in %s, but deleted in %s.\"),\n+\t\t\t\t oldpath, newpath, rename_branch, delete_branch);\n \t\t} else {\n \t\t\t/*\n \t\t\t * a few different cases...start by copying the\n-- \ngitgitgadget\n\n"},{"id":"412287","messageId":"d37e2626c30b03ecd5d091e0e1860144a1c0283b.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 07/11] merge-ort: add implementation of both sides renaming differently","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:28:02Z","receivedAt":"2020-12-15T18:33:04Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(1to2) handling, i.e. both sides of history\nrenaming a file and rename it differently. This code replaces the\nfollowing from merge-recurisve.c:\n\n * all the 1to2 code in process_renames()\n * the RENAME_ONE_FILE_TO_TWO case of process_entry()\n * handle_rename_rename_1to2()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_file_collision()\n * setup_rename_conflict_info()\n\nThe consolidation of five separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nTo be fair, there is a _slight_ tweak to process_entry() here to make\nsure that the two different paths aren't marked as clean but are left in\na conflicted state. So process_renames() and process_entry() aren't\nquite entirely orthogonal, but they are pretty close.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++---\n 1 file changed, 55 insertions(+), 3 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex da3715baa63..19477cfae60 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -676,6 +676,7 @@ static int process_renames(struct merge_options *opt,\n \t\t\tconst char *pathnames[3];\n \t\t\tstruct version_info merged;\n \t\t\tstruct conflict_info *base, *side1, *side2;\n+\t\t\tunsigned was_binary_blob = 0;\n \n \t\t\tpathnames[0] = oldpath;\n \t\t\tpathnames[1] = newpath;\n@@ -706,7 +707,58 @@ static int process_renames(struct merge_options *opt,\n \t\t\t}\n \n \t\t\t/* This is a rename/rename(1to2) */\n-\t\t\tdie(\"Not yet implemented\");\n+\t\t\tclean_merge = handle_content_merge(opt,\n+\t\t\t\t\t\t\t pair->one->path,\n+\t\t\t\t\t\t\t &base->stages[0],\n+\t\t\t\t\t\t\t &side1->stages[1],\n+\t\t\t\t\t\t\t &side2->stages[2],\n+\t\t\t\t\t\t\t pathnames,\n+\t\t\t\t\t\t\t 1 + 2 * opt->priv->call_depth,\n+\t\t\t\t\t\t\t &merged);\n+\t\t\tif (!clean_merge &&\n+\t\t\t merged.mode == side1->stages[1].mode &&\n+\t\t\t oideq(&merged.oid, &side1->stages[1].oid))\n+\t\t\t\twas_binary_blob = 1;\n+\t\t\tmemcpy(&side1->stages[1], &merged, sizeof(merged));\n+\t\t\tif (was_binary_blob) {\n+\t\t\t\t/*\n+\t\t\t\t * Getting here means we were attempting to\n+\t\t\t\t * merge a binary blob.\n+\t\t\t\t *\n+\t\t\t\t * Since we can't merge binaries,\n+\t\t\t\t * handle_content_merge() just takes one\n+\t\t\t\t * side. But we don't want to copy the\n+\t\t\t\t * contents of one side to both paths. We\n+\t\t\t\t * used the contents of side1 above for\n+\t\t\t\t * side1->stages, let's use the contents of\n+\t\t\t\t * side2 for side2->stages below.\n+\t\t\t\t */\n+\t\t\t\toidcpy(&merged.oid, &side2->stages[2].oid);\n+\t\t\t\tmerged.mode = side2->stages[2].mode;\n+\t\t\t}\n+\t\t\tmemcpy(&side2->stages[2], &merged, sizeof(merged));\n+\n+\t\t\tside1->path_conflict = 1;\n+\t\t\tside2->path_conflict = 1;\n+\t\t\t/*\n+\t\t\t * TODO: For renames we normally remove the path at the\n+\t\t\t * old name. It would thus seem consistent to do the\n+\t\t\t * same for rename/rename(1to2) cases, but we haven't\n+\t\t\t * done so traditionally and a number of the regression\n+\t\t\t * tests now encode an expectation that the file is\n+\t\t\t * left there at stage 1. If we ever decide to change\n+\t\t\t * this, add the following two lines here:\n+\t\t\t * base->merged.is_null = 1;\n+\t\t\t * base->merged.clean = 1;\n+\t\t\t * and remove the setting of base->path_conflict to 1.\n+\t\t\t */\n+\t\t\tbase->path_conflict = 1;\n+\t\t\tpath_msg(opt, oldpath, 0,\n+\t\t\t\t _(\"CONFLICT (rename/rename): %s renamed to \"\n+\t\t\t\t \"%s in %s and to %s in %s.\"),\n+\t\t\t\t pathnames[0],\n+\t\t\t\t pathnames[1], opt->branch1,\n+\t\t\t\t pathnames[2], opt->branch2);\n \n \t\t\ti++; /* We handled both renames, i.e. i+1 handled */\n \t\t\tcontinue;\n@@ -1291,13 +1343,13 @@ static void process_entry(struct merge_options *opt,\n \t\tint side = (ci->filemask == 4) ? 2 : 1;\n \t\tci->merged.result.mode = ci->stages[side].mode;\n \t\toidcpy(&ci->merged.result.oid, &ci->stages[side].oid);\n-\t\tci->merged.clean = !ci->df_conflict;\n+\t\tci->merged.clean = !ci->df_conflict && !ci->path_conflict;\n \t} else if (ci->filemask == 1) {\n \t\t/* Deleted on both sides */\n \t\tci->merged.is_null = 1;\n \t\tci->merged.result.mode = 0;\n \t\toidcpy(&ci->merged.result.oid, &null_oid);\n-\t\tci->merged.clean = 1;\n+\t\tci->merged.clean = !ci->path_conflict;\n \t}\n \n \t/*\n-- \ngitgitgadget\n\n"},{"id":"412288","messageId":"6b79da5e8a40e768b91d0742aff2919a0095a3f9.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 08/11] merge-ort: add implementation of rename/delete conflicts","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:28:03Z","receivedAt":"2020-12-15T18:33:04Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/delete conflicts, i.e. one side renames a file and the\nother deletes the file. This code replaces the following from\nmerge-recurisve.c:\n\n * the code relevant to RENAME_DELETE in process_renames()\n * the RENAME_DELETE case of process_entry()\n * handle_rename_delete()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_change_delete()\n * setup_rename_conflict_info()\n\nThe consolidation of five separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nTo be fair, there is a _slight_ tweak to process_entry() here, because\nrename/delete cases will also trigger the modify/delete codepath.\nHowever, we only want a modify/delete message to be printed for a\nrename/delete conflict if there is a content change in the renamed file\nin addition to the rename. So process_renames() and process_entry()\naren't quite fully orthogonal, but they are pretty close.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 48 ++++++++++++++++++++++++++++++++++++++++--------\n 1 file changed, 40 insertions(+), 8 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex 19477cfae60..a10c3f5046f 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -657,6 +657,7 @@ static int process_renames(struct merge_options *opt,\n \t\tunsigned int old_sidemask;\n \t\tint target_index, other_source_index;\n \t\tint source_deleted, collision, type_changed;\n+\t\tconst char *rename_branch = NULL, *delete_branch = NULL;\n \n \t\told_ent = strmap_get_entry(&opt->priv->paths, pair->one->path);\n \t\toldpath = old_ent->key;\n@@ -779,6 +780,15 @@ static int process_renames(struct merge_options *opt,\n \t\t\t/* special handling so later blocks can handle this */\n \t\t\tdie(\"Not yet implemented\");\n \t\t}\n+\t\tif (source_deleted) {\n+\t\t\tif (target_index == 1) {\n+\t\t\t\trename_branch = opt->branch1;\n+\t\t\t\tdelete_branch = opt->branch2;\n+\t\t\t} else {\n+\t\t\t\trename_branch = opt->branch2;\n+\t\t\t\tdelete_branch = opt->branch1;\n+\t\t\t}\n+\t\t}\n \n \t\tassert(source_deleted || oldinfo->filemask & old_sidemask);\n \n@@ -790,13 +800,26 @@ static int process_renames(struct merge_options *opt,\n \t\t\t/* rename/add/delete or rename/rename(2to1)/delete */\n \t\t\tdie(\"Not yet implemented\");\n \t\t} else {\n-\t\t\t/* a few different cases... */\n+\t\t\t/*\n+\t\t\t * a few different cases...start by copying the\n+\t\t\t * existing stage(s) from oldinfo over the newinfo\n+\t\t\t * and update the pathname(s).\n+\t\t\t */\n+\t\t\tmemcpy(&newinfo->stages[0], &oldinfo->stages[0],\n+\t\t\t sizeof(newinfo->stages[0]));\n+\t\t\tnewinfo->filemask |= (1 << MERGE_BASE);\n+\t\t\tnewinfo->pathnames[0] = oldpath;\n \t\t\tif (type_changed) {\n \t\t\t\t/* rename vs. typechange */\n \t\t\t\tdie(\"Not yet implemented\");\n \t\t\t} else if (source_deleted) {\n \t\t\t\t/* rename/delete */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\tnewinfo->path_conflict = 1;\n+\t\t\t\tpath_msg(opt, newpath, 0,\n+\t\t\t\t\t _(\"CONFLICT (rename/delete): %s renamed\"\n+\t\t\t\t\t \" to %s in %s, but deleted in %s.\"),\n+\t\t\t\t\t oldpath, newpath,\n+\t\t\t\t\t rename_branch, delete_branch);\n \t\t\t} else {\n \t\t\t\t/* normal rename */\n \t\t\t\tdie(\"Not yet implemented\");\n@@ -1332,12 +1355,21 @@ static void process_entry(struct merge_options *opt,\n \t\tmodify_branch = (side == 1) ? opt->branch1 : opt->branch2;\n \t\tdelete_branch = (side == 1) ? opt->branch2 : opt->branch1;\n \n-\t\tpath_msg(opt, path, 0,\n-\t\t\t _(\"CONFLICT (modify/delete): %s deleted in %s \"\n-\t\t\t \"and modified in %s. Version %s of %s left \"\n-\t\t\t \"in tree.\"),\n-\t\t\t path, delete_branch, modify_branch,\n-\t\t\t modify_branch, path);\n+\t\tif (ci->path_conflict &&\n+\t\t oideq(&ci->stages[0].oid, &ci->stages[side].oid)) {\n+\t\t\t/*\n+\t\t\t * This came from a rename/delete; no action to take,\n+\t\t\t * but avoid printing \"modify/delete\" conflict notice\n+\t\t\t * since the contents were not modified.\n+\t\t\t */\n+\t\t} else {\n+\t\t\tpath_msg(opt, path, 0,\n+\t\t\t\t _(\"CONFLICT (modify/delete): %s deleted in %s \"\n+\t\t\t\t \"and modified in %s. Version %s of %s left \"\n+\t\t\t\t \"in tree.\"),\n+\t\t\t\t path, delete_branch, modify_branch,\n+\t\t\t\t modify_branch, path);\n+\t\t}\n \t} else if (ci->filemask == 2 || ci->filemask == 4) {\n \t\t/* Added on one side */\n \t\tint side = (ci->filemask == 4) ? 2 : 1;\n-- \ngitgitgadget\n\n"},{"id":"412289","messageId":"7ec51feb4189209bab7912d457a2f54fd0a3cc0e.1608056886.git.gitgitgadget@gmail.com","threadId":"54806","inReplyTo":"pull.812.v3.git.1608056886.gitgitgadget@gmail.com","subject":"[PATCH v3 06/11] merge-ort: add implementation of both sides renaming identically","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-12-15T18:28:01Z","receivedAt":"2020-12-15T18:33:59Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nImplement rename/rename(1to1) handling, i.e. both sides of history\nrenaming a file but renaming the same way. This code replaces the\nfollowing from merge-recurisve.c:\n\n * all the 1to1 code in process_renames()\n * the RENAME_ONE_FILE_TO_ONE case of process_entry()\n\nAlso, there is some shared code from merge-recursive.c for multiple\ndifferent rename cases which we will no longer need for this case (or\nother rename cases):\n\n * handle_rename_normal()\n * setup_rename_conflict_info()\n\nThe consolidation of four separate codepaths into one is made possible\nby a change in design: process_renames() tweaks the conflict_info\nentries within opt->priv->paths such that process_entry() can then\nhandle all the non-rename conflict types (directory/file, modify/delete,\netc.) orthogonally. This means we're much less likely to miss special\nimplementation of some kind of combination of conflict types (see\ncommits brought in by 66c62eaec6 (\"Merge branch 'en/merge-tests'\",\n2020-11-18), especially commit ef52778708 (\"merge tests: expect improved\ndirectory/file conflict handling in ort\", 2020-10-26) for more details).\nThat, together with letting worktree/index updating be handled\northogonally in the merge_switch_to_result() function, dramatically\nsimplifies the code for various special rename cases.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort.c | 20 ++++++++++++++++++--\n 1 file changed, 18 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex ebe275ef73c..da3715baa63 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -674,14 +674,30 @@ static int process_renames(struct merge_options *opt,\n \t\t !strcmp(oldpath, renames->queue[i+1]->one->path)) {\n \t\t\t/* Handle rename/rename(1to2) or rename/rename(1to1) */\n \t\t\tconst char *pathnames[3];\n+\t\t\tstruct version_info merged;\n+\t\t\tstruct conflict_info *base, *side1, *side2;\n \n \t\t\tpathnames[0] = oldpath;\n \t\t\tpathnames[1] = newpath;\n \t\t\tpathnames[2] = renames->queue[i+1]->two->path;\n \n+\t\t\tbase = strmap_get(&opt->priv->paths, pathnames[0]);\n+\t\t\tside1 = strmap_get(&opt->priv->paths, pathnames[1]);\n+\t\t\tside2 = strmap_get(&opt->priv->paths, pathnames[2]);\n+\n+\t\t\tVERIFY_CI(base);\n+\t\t\tVERIFY_CI(side1);\n+\t\t\tVERIFY_CI(side2);\n+\n \t\t\tif (!strcmp(pathnames[1], pathnames[2])) {\n-\t\t\t\t/* Both sides renamed the same way. */\n-\t\t\t\tdie(\"Not yet implemented\");\n+\t\t\t\t/* Both sides renamed the same way */\n+\t\t\t\tassert(side1 == side2);\n+\t\t\t\tmemcpy(&side1->stages[0], &base->stages[0],\n+\t\t\t\t sizeof(merged));\n+\t\t\t\tside1->filemask |= (1 << MERGE_BASE);\n+\t\t\t\t/* Mark base as resolved by removal */\n+\t\t\t\tbase->merged.is_null = 1;\n+\t\t\t\tbase->merged.clean = 1;\n \n \t\t\t\t/* We handled both renames, i.e. i+1 handled */\n \t\t\t\ti++;\n-- \ngitgitgadget\n\n"},{"id":"412304","messageId":"xmqqh7om7mdc.fsf@gitster.c.googlers.com","threadId":"54806","inReplyTo":"6448806f-791b-610a-06de-7be5650334e6@gmail.com","subject":"Re: [PATCH v2 00/11] merge-ort: add basic rename detection","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-12-15T22:09:03Z","receivedAt":"2020-12-15T22:10:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 12/14/2020 11:21 AM, Elijah Newren via GitGitGadget wrote:\n>> This series builds on en/merge-ort-2 and adds basic rename detection to\n>> merge-ort.\n>\n> I have now finished a full pass through this series. I find it to be\n> well organized with a satisfying conclusion. My comments are mostly\n> about nits. I tried to recommend some better organization, but failed\n> to find a clear way to do so, which shows that the organization here\n> is sound.\n>\n> My only complaint is that some of the patches break compilation, so\n> that should definitely be fixed to assist with bisects in the future\n> (likely for bisects unrelated to this feature).\n\nThanks.\n"}]}