{"thread":{"id":"57250","subject":"[PATCH] merge-ort: avoid assuming all renames detected","startedAt":"2022-01-15T02:09:32Z","lastAt":"2022-07-01T05:21:55Z","messageCount":12,"participants":["Elijah Newren via GitGitGadget","Junio C Hamano","Elijah Newren","Taylor Blau","SZEDER Gábor"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"446284","messageId":"pull.1194.git.git.1642212566346.gitgitgadget@gmail.com","threadId":"57250","inReplyTo":null,"subject":"[PATCH] merge-ort: avoid assuming all renames detected","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-15T02:09:26Z","receivedAt":"2022-01-15T02:09:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nIn commit 8b09a900a1 (\"merge-ort: restart merge with cached renames to\nreduce process entry cost\", 2021-07-16), we noted that in the merge-ort\nsteps of\n    collect_merge_info()\n    detect_and_process_renames()\n    process_entries()\nthat process_entries() was expensive, and we could often make it cheaper\nby changing this to\n    collect_merge_info()\n    detect_and_process_renames()\n    <cache all the renames, and restart>\n    collect_merge_info()\n    detect_and_process_renames()\n    process_entries()\nbecause the second collect_merge_info() would be cheaper (we could avoid\ntraversing into some directories), the second\ndetect_and_process_renames() would be free since we had already detected\nall renames, and then process_entries() has far fewer entries to handle.\n\nHowever, this was built on the assumption that the first\ndetect_and_process_renames() actually detected all potential renames.\nIf someone has merge.renameLimit set to some small value, that\nassumption is violated which manifests later with the following message:\n\n    $ git -c merge.renameLimit=1 rebase upstream\n    ...\n    git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion\n    `renames->cached_pairs_valid_side == 0' failed.\n\nTurn off this cache-renames-and-restart whenever we cannot detect all\nrenames, and add a testcase that would have caught this problem.\n\nReported-by: Taylor Blau <me@ttaylorr.com>\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n    merge-ort: avoid assuming all renames detected\n    \n    Fixes https://lore.kernel.org/git/YeHTIfEutLYM4TIU@nand.local/\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1194%2Fnewren%2Favoid-assertion-assuming-renames-found-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1194/newren/avoid-assertion-assuming-renames-found-v1\nPull-Request: https://github.com/git/git/pull/1194\n\n merge-ort.c                              |  4 ++\n t/t6429-merge-sequence-rename-caching.sh | 67 ++++++++++++++++++++++++\n 2 files changed, 71 insertions(+)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex c3197970219..143d274f117 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -3060,6 +3060,10 @@ static int detect_and_process_renames(struct merge_options *opt,\n \ttrace2_region_enter(\"merge\", \"regular renames\", opt->repo);\n \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE1);\n \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE2);\n+\tif (renames->needed_limit != 0) {\n+\t\trenames->cached_pairs_valid_side = 0;\n+\t\trenames->redo_after_renames = 0;\n+\t}\n \tif (renames->redo_after_renames && detection_run) {\n \t\tint i, side;\n \t\tstruct diff_filepair *p;\ndiff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh\nindex 035edc40b1e..f2bc8a7d2a2 100755\n--- a/t/t6429-merge-sequence-rename-caching.sh\n+++ b/t/t6429-merge-sequence-rename-caching.sh\n@@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '\n \t)\n '\n \n+#\n+# The following testcase just creates two simple renames (slightly modified\n+# on both sides but without conflicting changes), and a directory full of\n+# files that are otherwise uninteresting.  The setup is as follows:\n+#\n+#   base:     unrelated/<BUNCH OF FILES>\n+#             numbers\n+#             values\n+#   upstream: modify: numbers\n+#             modify: values\n+#   topic:    add: unrelated/foo\n+#             modify: numbers\n+#             modify: values\n+#             rename: numbers -> sequence\n+#             rename: values -> progression\n+#\n+# This is a trivial rename case, but we're curious what happens with a very\n+# low renameLimit interacting with the restart optimization trying to notice\n+# that unrelated/ looks like a trivial merge candidate.\n+#\n+test_expect_success 'avoid assuming we detected renames' '\n+\tgit init redo-weirdness &&\n+\t(\n+\t\tcd redo-weirdness &&\n+\n+\t\tmkdir unrelated &&\n+\t\tfor i in $(test_seq 1 10)\n+\t\tdo\n+\t\t\t>unrelated/$i\n+\t\tdone &&\n+\t\ttest_seq  2 10 >numbers &&\n+\t\ttest_seq 12 20 >values &&\n+\t\tgit add numbers values unrelated/ &&\n+\t\tgit commit -m orig &&\n+\n+\t\tgit branch upstream &&\n+\t\tgit branch topic &&\n+\n+\t\tgit switch upstream &&\n+\t\ttest_seq  1 10 >numbers &&\n+\t\ttest_seq 11 20 >values &&\n+\t\tgit add numbers &&\n+\t\tgit commit -m \"Some tweaks\" &&\n+\n+\t\tgit switch topic &&\n+\n+\t\t>unrelated/foo &&\n+\t\ttest_seq  2 12 >numbers &&\n+\t\ttest_seq 12 22 >values &&\n+\t\tgit add numbers values unrelated/ &&\n+\t\tgit mv numbers sequence &&\n+\t\tgit mv values progression &&\n+\t\tgit commit -m A &&\n+\n+\t\t#\n+\t\t# Actual testing\n+\t\t#\n+\n+\t\tgit switch --detach topic^0 &&\n+\n+\t\ttest_must_fail git -c merge.renameLimit=1 rebase upstream &&\n+\n+\t\tgit ls-files -u >actual &&\n+\t\t! test_file_is_empty actual\n+\t)\n+'\n+\n test_done\n\nbase-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2\n-- \ngitgitgadget\n"},{"id":"446329","messageId":"xmqqczkr1hhb.fsf@gitster.g","threadId":"57250","inReplyTo":"pull.1194.git.git.1642212566346.gitgitgadget@gmail.com","subject":"Re: [PATCH] merge-ort: avoid assuming all renames detected","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-16T21:46:40Z","receivedAt":"2022-01-16T21:46:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Elijah Newren <newren@gmail.com>\n>\n> In commit 8b09a900a1 (\"merge-ort: restart merge with cached renames to\n> reduce process entry cost\", 2021-07-16), we noted that in the merge-ort\n> steps of\n>     collect_merge_info()\n>     detect_and_process_renames()\n>     process_entries()\n> that process_entries() was expensive, and we could often make it cheaper\n> by changing this to\n>     collect_merge_info()\n>     detect_and_process_renames()\n>     <cache all the renames, and restart>\n>     collect_merge_info()\n>     detect_and_process_renames()\n>     process_entries()\n> because the second collect_merge_info() would be cheaper (we could avoid\n> traversing into some directories), the second\n> detect_and_process_renames() would be free since we had already detected\n> all renames, and then process_entries() has far fewer entries to handle.\n>\n> However, this was built on the assumption that the first\n> detect_and_process_renames() actually detected all potential renames.\n> If someone has merge.renameLimit set to some small value, that\n> assumption is violated which manifests later with the following message:\n>\n>     $ git -c merge.renameLimit=1 rebase upstream\n>     ...\n>     git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion\n>     `renames->cached_pairs_valid_side == 0' failed.\n>\n> Turn off this cache-renames-and-restart whenever we cannot detect all\n> renames, and add a testcase that would have caught this problem.\n>\n> Reported-by: Taylor Blau <me@ttaylorr.com>\n> Signed-off-by: Elijah Newren <newren@gmail.com>\n> ---\n\nThanks.  Will fork from maint-2.33 and park on 'seen' for now.\n"},{"id":"446332","messageId":"xmqqwnizxo8g.fsf@gitster.g","threadId":"57250","inReplyTo":"pull.1194.git.git.1642212566346.gitgitgadget@gmail.com","subject":"Re: [PATCH] merge-ort: avoid assuming all renames detected","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-16T23:19:43Z","receivedAt":"2022-01-16T23:19:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> diff --git a/merge-ort.c b/merge-ort.c\n> index c3197970219..143d274f117 100644\n> --- a/merge-ort.c\n> +++ b/merge-ort.c\n> @@ -3060,6 +3060,10 @@ static int detect_and_process_renames(struct merge_options *opt,\n>  \ttrace2_region_enter(\"merge\", \"regular renames\", opt->repo);\n>  \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE1);\n>  \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE2);\n> +\tif (renames->needed_limit != 0) {\n\nDon't compare with NULL or 0, i.e.\n\n\tif (renames->needed_limit) {\n\n> +\t\trenames->cached_pairs_valid_side = 0;\n> +\t\trenames->redo_after_renames = 0;\n> +\t}\n>  \tif (renames->redo_after_renames && detection_run) {\n>  \t\tint i, side;\n>  \t\tstruct diff_filepair *p;\n\n"},{"id":"446369","messageId":"pull.1194.v2.git.git.1642443955836.gitgitgadget@gmail.com","threadId":"57250","inReplyTo":"pull.1194.git.git.1642212566346.gitgitgadget@gmail.com","subject":"[PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-01-17T18:25:55Z","receivedAt":"2022-01-17T18:26:02Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren <newren@gmail.com>\n\nIn commit 8b09a900a1 (\"merge-ort: restart merge with cached renames to\nreduce process entry cost\", 2021-07-16), we noted that in the merge-ort\nsteps of\n    collect_merge_info()\n    detect_and_process_renames()\n    process_entries()\nthat process_entries() was expensive, and we could often make it cheaper\nby changing this to\n    collect_merge_info()\n    detect_and_process_renames()\n    <cache all the renames, and restart>\n    collect_merge_info()\n    detect_and_process_renames()\n    process_entries()\nbecause the second collect_merge_info() would be cheaper (we could avoid\ntraversing into some directories), the second\ndetect_and_process_renames() would be free since we had already detected\nall renames, and then process_entries() has far fewer entries to handle.\n\nHowever, this was built on the assumption that the first\ndetect_and_process_renames() actually detected all potential renames.\nIf someone has merge.renameLimit set to some small value, that\nassumption is violated which manifests later with the following message:\n\n    $ git -c merge.renameLimit=1 rebase upstream\n    ...\n    git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion\n    `renames->cached_pairs_valid_side == 0' failed.\n\nTurn off this cache-renames-and-restart whenever we cannot detect all\nrenames, and add a testcase that would have caught this problem.\n\nReported-by: Taylor Blau <me@ttaylorr.com>\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n    merge-ort: avoid assuming all renames detected\n    \n    Fixes https://lore.kernel.org/git/YeHTIfEutLYM4TIU@nand.local/\n    \n    Changes since v1:\n    \n     * Fixed a small style issue\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1194%2Fnewren%2Favoid-assertion-assuming-renames-found-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1194/newren/avoid-assertion-assuming-renames-found-v2\nPull-Request: https://github.com/git/git/pull/1194\n\nRange-diff vs v1:\n\n 1:  f1e9901ae67 ! 1:  239d3ba08c1 merge-ort: avoid assuming all renames detected\n     @@ merge-ort.c: static int detect_and_process_renames(struct merge_options *opt,\n       \ttrace2_region_enter(\"merge\", \"regular renames\", opt->repo);\n       \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE1);\n       \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE2);\n     -+\tif (renames->needed_limit != 0) {\n     ++\tif (renames->needed_limit) {\n      +\t\trenames->cached_pairs_valid_side = 0;\n      +\t\trenames->redo_after_renames = 0;\n      +\t}\n\n\n merge-ort.c                              |  4 ++\n t/t6429-merge-sequence-rename-caching.sh | 67 ++++++++++++++++++++++++\n 2 files changed, 71 insertions(+)\n\ndiff --git a/merge-ort.c b/merge-ort.c\nindex c3197970219..b0ff9a72879 100644\n--- a/merge-ort.c\n+++ b/merge-ort.c\n@@ -3060,6 +3060,10 @@ static int detect_and_process_renames(struct merge_options *opt,\n \ttrace2_region_enter(\"merge\", \"regular renames\", opt->repo);\n \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE1);\n \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE2);\n+\tif (renames->needed_limit) {\n+\t\trenames->cached_pairs_valid_side = 0;\n+\t\trenames->redo_after_renames = 0;\n+\t}\n \tif (renames->redo_after_renames && detection_run) {\n \t\tint i, side;\n \t\tstruct diff_filepair *p;\ndiff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh\nindex 035edc40b1e..f2bc8a7d2a2 100755\n--- a/t/t6429-merge-sequence-rename-caching.sh\n+++ b/t/t6429-merge-sequence-rename-caching.sh\n@@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '\n \t)\n '\n \n+#\n+# The following testcase just creates two simple renames (slightly modified\n+# on both sides but without conflicting changes), and a directory full of\n+# files that are otherwise uninteresting.  The setup is as follows:\n+#\n+#   base:     unrelated/<BUNCH OF FILES>\n+#             numbers\n+#             values\n+#   upstream: modify: numbers\n+#             modify: values\n+#   topic:    add: unrelated/foo\n+#             modify: numbers\n+#             modify: values\n+#             rename: numbers -> sequence\n+#             rename: values -> progression\n+#\n+# This is a trivial rename case, but we're curious what happens with a very\n+# low renameLimit interacting with the restart optimization trying to notice\n+# that unrelated/ looks like a trivial merge candidate.\n+#\n+test_expect_success 'avoid assuming we detected renames' '\n+\tgit init redo-weirdness &&\n+\t(\n+\t\tcd redo-weirdness &&\n+\n+\t\tmkdir unrelated &&\n+\t\tfor i in $(test_seq 1 10)\n+\t\tdo\n+\t\t\t>unrelated/$i\n+\t\tdone &&\n+\t\ttest_seq  2 10 >numbers &&\n+\t\ttest_seq 12 20 >values &&\n+\t\tgit add numbers values unrelated/ &&\n+\t\tgit commit -m orig &&\n+\n+\t\tgit branch upstream &&\n+\t\tgit branch topic &&\n+\n+\t\tgit switch upstream &&\n+\t\ttest_seq  1 10 >numbers &&\n+\t\ttest_seq 11 20 >values &&\n+\t\tgit add numbers &&\n+\t\tgit commit -m \"Some tweaks\" &&\n+\n+\t\tgit switch topic &&\n+\n+\t\t>unrelated/foo &&\n+\t\ttest_seq  2 12 >numbers &&\n+\t\ttest_seq 12 22 >values &&\n+\t\tgit add numbers values unrelated/ &&\n+\t\tgit mv numbers sequence &&\n+\t\tgit mv values progression &&\n+\t\tgit commit -m A &&\n+\n+\t\t#\n+\t\t# Actual testing\n+\t\t#\n+\n+\t\tgit switch --detach topic^0 &&\n+\n+\t\ttest_must_fail git -c merge.renameLimit=1 rebase upstream &&\n+\n+\t\tgit ls-files -u >actual &&\n+\t\t! test_file_is_empty actual\n+\t)\n+'\n+\n test_done\n\nbase-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2\n-- \ngitgitgadget\n"},{"id":"446378","messageId":"xmqqh7a2uphk.fsf@gitster.g","threadId":"57250","inReplyTo":"pull.1194.v2.git.git.1642443955836.gitgitgadget@gmail.com","subject":"Re: [PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-17T19:33:11Z","receivedAt":"2022-01-17T19:33:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Elijah Newren <newren@gmail.com>\n>\n> In commit 8b09a900a1 (\"merge-ort: restart merge with cached renames to\n> reduce process entry cost\", 2021-07-16), we noted that in the merge-ort\n> steps of\n>     collect_merge_info()\n>     detect_and_process_renames()\n>     process_entries()\n> that process_entries() was expensive, and we could often make it cheaper\n> by changing this to\n>     collect_merge_info()\n>     detect_and_process_renames()\n>     <cache all the renames, and restart>\n>     collect_merge_info()\n>     detect_and_process_renames()\n>     process_entries()\n> because the second collect_merge_info() would be cheaper (we could avoid\n> traversing into some directories), the second\n> detect_and_process_renames() would be free since we had already detected\n> all renames, and then process_entries() has far fewer entries to handle.\n>\n> However, this was built on the assumption that the first\n> detect_and_process_renames() actually detected all potential renames.\n> If someone has merge.renameLimit set to some small value, that\n> assumption is violated which manifests later with the following message:\n>\n>     $ git -c merge.renameLimit=1 rebase upstream\n>     ...\n>     git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion\n>     `renames->cached_pairs_valid_side == 0' failed.\n>\n> Turn off this cache-renames-and-restart whenever we cannot detect all\n> renames, and add a testcase that would have caught this problem.\n>\n> Reported-by: Taylor Blau <me@ttaylorr.com>\n> Signed-off-by: Elijah Newren <newren@gmail.com>\n> ---\n\nThanks.  An Ack?\n\n>     merge-ort: avoid assuming all renames detected\n>     \n>     Fixes https://lore.kernel.org/git/YeHTIfEutLYM4TIU@nand.local/\n>     \n>     Changes since v1:\n>     \n>      * Fixed a small style issue\n>\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1194%2Fnewren%2Favoid-assertion-assuming-renames-found-v2\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1194/newren/avoid-assertion-assuming-renames-found-v2\n> Pull-Request: https://github.com/git/git/pull/1194\n>\n> Range-diff vs v1:\n>\n>  1:  f1e9901ae67 ! 1:  239d3ba08c1 merge-ort: avoid assuming all renames detected\n>      @@ merge-ort.c: static int detect_and_process_renames(struct merge_options *opt,\n>        \ttrace2_region_enter(\"merge\", \"regular renames\", opt->repo);\n>        \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE1);\n>        \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE2);\n>      -+\tif (renames->needed_limit != 0) {\n>      ++\tif (renames->needed_limit) {\n>       +\t\trenames->cached_pairs_valid_side = 0;\n>       +\t\trenames->redo_after_renames = 0;\n>       +\t}\n>\n>\n>  merge-ort.c                              |  4 ++\n>  t/t6429-merge-sequence-rename-caching.sh | 67 ++++++++++++++++++++++++\n>  2 files changed, 71 insertions(+)\n>\n> diff --git a/merge-ort.c b/merge-ort.c\n> index c3197970219..b0ff9a72879 100644\n> --- a/merge-ort.c\n> +++ b/merge-ort.c\n> @@ -3060,6 +3060,10 @@ static int detect_and_process_renames(struct merge_options *opt,\n>  \ttrace2_region_enter(\"merge\", \"regular renames\", opt->repo);\n>  \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE1);\n>  \tdetection_run |= detect_regular_renames(opt, MERGE_SIDE2);\n> +\tif (renames->needed_limit) {\n> +\t\trenames->cached_pairs_valid_side = 0;\n> +\t\trenames->redo_after_renames = 0;\n> +\t}\n>  \tif (renames->redo_after_renames && detection_run) {\n>  \t\tint i, side;\n>  \t\tstruct diff_filepair *p;\n> diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh\n> index 035edc40b1e..f2bc8a7d2a2 100755\n> --- a/t/t6429-merge-sequence-rename-caching.sh\n> +++ b/t/t6429-merge-sequence-rename-caching.sh\n> @@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '\n>  \t)\n>  '\n>  \n> +#\n> +# The following testcase just creates two simple renames (slightly modified\n> +# on both sides but without conflicting changes), and a directory full of\n> +# files that are otherwise uninteresting.  The setup is as follows:\n> +#\n> +#   base:     unrelated/<BUNCH OF FILES>\n> +#             numbers\n> +#             values\n> +#   upstream: modify: numbers\n> +#             modify: values\n> +#   topic:    add: unrelated/foo\n> +#             modify: numbers\n> +#             modify: values\n> +#             rename: numbers -> sequence\n> +#             rename: values -> progression\n> +#\n> +# This is a trivial rename case, but we're curious what happens with a very\n> +# low renameLimit interacting with the restart optimization trying to notice\n> +# that unrelated/ looks like a trivial merge candidate.\n> +#\n> +test_expect_success 'avoid assuming we detected renames' '\n> +\tgit init redo-weirdness &&\n> +\t(\n> +\t\tcd redo-weirdness &&\n> +\n> +\t\tmkdir unrelated &&\n> +\t\tfor i in $(test_seq 1 10)\n> +\t\tdo\n> +\t\t\t>unrelated/$i\n> +\t\tdone &&\n> +\t\ttest_seq  2 10 >numbers &&\n> +\t\ttest_seq 12 20 >values &&\n> +\t\tgit add numbers values unrelated/ &&\n> +\t\tgit commit -m orig &&\n> +\n> +\t\tgit branch upstream &&\n> +\t\tgit branch topic &&\n> +\n> +\t\tgit switch upstream &&\n> +\t\ttest_seq  1 10 >numbers &&\n> +\t\ttest_seq 11 20 >values &&\n> +\t\tgit add numbers &&\n> +\t\tgit commit -m \"Some tweaks\" &&\n> +\n> +\t\tgit switch topic &&\n> +\n> +\t\t>unrelated/foo &&\n> +\t\ttest_seq  2 12 >numbers &&\n> +\t\ttest_seq 12 22 >values &&\n> +\t\tgit add numbers values unrelated/ &&\n> +\t\tgit mv numbers sequence &&\n> +\t\tgit mv values progression &&\n> +\t\tgit commit -m A &&\n> +\n> +\t\t#\n> +\t\t# Actual testing\n> +\t\t#\n> +\n> +\t\tgit switch --detach topic^0 &&\n> +\n> +\t\ttest_must_fail git -c merge.renameLimit=1 rebase upstream &&\n> +\n> +\t\tgit ls-files -u >actual &&\n> +\t\t! test_file_is_empty actual\n> +\t)\n> +'\n> +\n>  test_done\n>\n> base-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2\n"},{"id":"446381","messageId":"CABPp-BF+eUGJ7FgMRG9vOUh8X5ExU+jkR3WHZeFGnKH80SO7gQ@mail.gmail.com","threadId":"57250","inReplyTo":"xmqqh7a2uphk.fsf@gitster.g","subject":"Re: [PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-01-17T21:21:11Z","receivedAt":"2022-01-17T21:21:26Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Jan 17, 2022 at 11:33 AM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> \"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n>\n> > From: Elijah Newren <newren@gmail.com>\n> >\n> > In commit 8b09a900a1 (\"merge-ort: restart merge with cached renames to\n> > reduce process entry cost\", 2021-07-16), we noted that in the merge-ort\n> > steps of\n> >     collect_merge_info()\n> >     detect_and_process_renames()\n> >     process_entries()\n> > that process_entries() was expensive, and we could often make it cheaper\n> > by changing this to\n> >     collect_merge_info()\n> >     detect_and_process_renames()\n> >     <cache all the renames, and restart>\n> >     collect_merge_info()\n> >     detect_and_process_renames()\n> >     process_entries()\n> > because the second collect_merge_info() would be cheaper (we could avoid\n> > traversing into some directories), the second\n> > detect_and_process_renames() would be free since we had already detected\n> > all renames, and then process_entries() has far fewer entries to handle.\n> >\n> > However, this was built on the assumption that the first\n> > detect_and_process_renames() actually detected all potential renames.\n> > If someone has merge.renameLimit set to some small value, that\n> > assumption is violated which manifests later with the following message:\n> >\n> >     $ git -c merge.renameLimit=1 rebase upstream\n> >     ...\n> >     git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion\n> >     `renames->cached_pairs_valid_side == 0' failed.\n> >\n> > Turn off this cache-renames-and-restart whenever we cannot detect all\n> > renames, and add a testcase that would have caught this problem.\n> >\n> > Reported-by: Taylor Blau <me@ttaylorr.com>\n> > Signed-off-by: Elijah Newren <newren@gmail.com>\n> > ---\n>\n> Thanks.  An Ack?\n\nTaylor told me the code change fixed his case, and that he'd review my\nfull patch with the testcase when I posted it.  Let's wait to hear\nfrom him.\n"},{"id":"446385","messageId":"YeXotaZmqhkJaQr6@nand.local","threadId":"57250","inReplyTo":"CABPp-BF+eUGJ7FgMRG9vOUh8X5ExU+jkR3WHZeFGnKH80SO7gQ@mail.gmail.com","subject":"Re: [PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-01-17T22:07:49Z","receivedAt":"2022-01-17T22:07:55Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Jan 17, 2022 at 01:21:11PM -0800, Elijah Newren wrote:\n> On Mon, Jan 17, 2022 at 11:33 AM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> > Thanks.  An Ack?\n>\n> Taylor told me the code change fixed his case, and that he'd review my\n> full patch with the testcase when I posted it.  Let's wait to hear\n> from him.\n\nAck. I can't vouch for the ort-specific details, but I trust Elijah's\njudgement (obviously). Running a version of Git with this patch applied\nfixes the issue I originally reported.\n\nThanks, Elijah!\n"},{"id":"446386","messageId":"xmqq35lmuhxy.fsf@gitster.g","threadId":"57250","inReplyTo":"CABPp-BF+eUGJ7FgMRG9vOUh8X5ExU+jkR3WHZeFGnKH80SO7gQ@mail.gmail.com","subject":"Re: [PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-17T22:16:09Z","receivedAt":"2022-01-17T22:16:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren <newren@gmail.com> writes:\n\n> On Mon, Jan 17, 2022 at 11:33 AM Junio C Hamano <gitster@pobox.com> wrote:\n>>\n>> \"Elijah Newren via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n>>\n>> > From: Elijah Newren <newren@gmail.com>\n>> >\n>> > In commit 8b09a900a1 (\"merge-ort: restart merge with cached renames to\n>> > reduce process entry cost\", 2021-07-16), we noted that in the merge-ort\n>> > steps of\n>> >     collect_merge_info()\n>> >     detect_and_process_renames()\n>> >     process_entries()\n>> > that process_entries() was expensive, and we could often make it cheaper\n>> > by changing this to\n>> >     collect_merge_info()\n>> >     detect_and_process_renames()\n>> >     <cache all the renames, and restart>\n>> >     collect_merge_info()\n>> >     detect_and_process_renames()\n>> >     process_entries()\n>> > because the second collect_merge_info() would be cheaper (we could avoid\n>> > traversing into some directories), the second\n>> > detect_and_process_renames() would be free since we had already detected\n>> > all renames, and then process_entries() has far fewer entries to handle.\n>> >\n>> > However, this was built on the assumption that the first\n>> > detect_and_process_renames() actually detected all potential renames.\n>> > If someone has merge.renameLimit set to some small value, that\n>> > assumption is violated which manifests later with the following message:\n>> >\n>> >     $ git -c merge.renameLimit=1 rebase upstream\n>> >     ...\n>> >     git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion\n>> >     `renames->cached_pairs_valid_side == 0' failed.\n>> >\n>> > Turn off this cache-renames-and-restart whenever we cannot detect all\n>> > renames, and add a testcase that would have caught this problem.\n>> >\n>> > Reported-by: Taylor Blau <me@ttaylorr.com>\n>> > Signed-off-by: Elijah Newren <newren@gmail.com>\n>> > ---\n>>\n>> Thanks.  An Ack?\n>\n> Taylor told me the code change fixed his case, and that he'd review my\n> full patch with the testcase when I posted it.  Let's wait to hear\n> from him.\n\nYes, I am waiting (notice who is on To: and not Cc: on the message\nyou are responding to ;-).\n\nThanks.\n"},{"id":"446387","messageId":"xmqqy23et30s.fsf@gitster.g","threadId":"57250","inReplyTo":"YeXotaZmqhkJaQr6@nand.local","subject":"Re: [PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-01-17T22:23:47Z","receivedAt":"2022-01-17T22:23:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau <me@ttaylorr.com> writes:\n\n> On Mon, Jan 17, 2022 at 01:21:11PM -0800, Elijah Newren wrote:\n>> On Mon, Jan 17, 2022 at 11:33 AM Junio C Hamano <gitster@pobox.com> wrote:\n>>\n>> > Thanks.  An Ack?\n>>\n>> Taylor told me the code change fixed his case, and that he'd review my\n>> full patch with the testcase when I posted it.  Let's wait to hear\n>> from him.\n>\n> Ack. I can't vouch for the ort-specific details, but I trust Elijah's\n> judgement (obviously). Running a version of Git with this patch applied\n> fixes the issue I originally reported.\n>\n> Thanks, Elijah!\n\nThanks.  Will queue with your Tested-by, then.\n\nThank you very much, both of you.\n"},{"id":"458173","messageId":"20220630095357.GA2123@szeder.dev","threadId":"57250","inReplyTo":"pull.1194.v2.git.git.1642443955836.gitgitgadget@gmail.com","subject":"Re: [PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2022-06-30T09:53:57Z","receivedAt":"2022-06-30T09:54:05Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Mon, Jan 17, 2022 at 06:25:55PM +0000, Elijah Newren via GitGitGadget wrote:\n> diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh\n> index 035edc40b1e..f2bc8a7d2a2 100755\n> --- a/t/t6429-merge-sequence-rename-caching.sh\n> +++ b/t/t6429-merge-sequence-rename-caching.sh\n> @@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '\n>  \t)\n>  '\n>  \n> +#\n> +# The following testcase just creates two simple renames (slightly modified\n> +# on both sides but without conflicting changes), and a directory full of\n> +# files that are otherwise uninteresting.  The setup is as follows:\n> +#\n> +#   base:     unrelated/<BUNCH OF FILES>\n> +#             numbers\n> +#             values\n> +#   upstream: modify: numbers\n> +#             modify: values\n> +#   topic:    add: unrelated/foo\n> +#             modify: numbers\n> +#             modify: values\n> +#             rename: numbers -> sequence\n> +#             rename: values -> progression\n> +#\n> +# This is a trivial rename case, but we're curious what happens with a very\n> +# low renameLimit interacting with the restart optimization trying to notice\n> +# that unrelated/ looks like a trivial merge candidate.\n> +#\n> +test_expect_success 'avoid assuming we detected renames' '\n> +\tgit init redo-weirdness &&\n> +\t(\n> +\t\tcd redo-weirdness &&\n> +\n> +\t\tmkdir unrelated &&\n> +\t\tfor i in $(test_seq 1 10)\n> +\t\tdo\n> +\t\t\t>unrelated/$i\n> +\t\tdone &&\n> +\t\ttest_seq  2 10 >numbers &&\n> +\t\ttest_seq 12 20 >values &&\n> +\t\tgit add numbers values unrelated/ &&\n> +\t\tgit commit -m orig &&\n> +\n> +\t\tgit branch upstream &&\n> +\t\tgit branch topic &&\n> +\n> +\t\tgit switch upstream &&\n> +\t\ttest_seq  1 10 >numbers &&\n> +\t\ttest_seq 11 20 >values &&\n> +\t\tgit add numbers &&\n> +\t\tgit commit -m \"Some tweaks\" &&\n> +\n> +\t\tgit switch topic &&\n> +\n> +\t\t>unrelated/foo &&\n> +\t\ttest_seq  2 12 >numbers &&\n> +\t\ttest_seq 12 22 >values &&\n> +\t\tgit add numbers values unrelated/ &&\n> +\t\tgit mv numbers sequence &&\n> +\t\tgit mv values progression &&\n> +\t\tgit commit -m A &&\n> +\n> +\t\t#\n> +\t\t# Actual testing\n> +\t\t#\n> +\n> +\t\tgit switch --detach topic^0 &&\n> +\n> +\t\ttest_must_fail git -c merge.renameLimit=1 rebase upstream &&\n> +\n> +\t\tgit ls-files -u >actual &&\n> +\t\t! test_file_is_empty actual\n\nThere is no 'test_file_is_empty' function, but because of the ! at the\nbeginning of the line it didn't fail the test.\n\nThe minimal fix would be to use 'test_file_not_empty' instead, but I\nwonder whether we should use 'test_line_count = 2' instead for a tad\ntighter check.\n\n> +\t)\n> +'\n> +\n>  test_done\n> \n> base-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2\n> -- \n> gitgitgadget\n"},{"id":"458326","messageId":"CABPp-BG1MARvcqgPYBNQ0P1M9jZ2b8vt9btPG1oQ5_h_YM78TQ@mail.gmail.com","threadId":"57250","inReplyTo":"20220630095357.GA2123@szeder.dev","subject":"Re: [PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-01T02:30:39Z","receivedAt":"2022-07-01T02:30:59Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Jun 30, 2022 at 2:54 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:\n>\n> On Mon, Jan 17, 2022 at 06:25:55PM +0000, Elijah Newren via GitGitGadget wrote:\n> > diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh\n> > index 035edc40b1e..f2bc8a7d2a2 100755\n> > --- a/t/t6429-merge-sequence-rename-caching.sh\n> > +++ b/t/t6429-merge-sequence-rename-caching.sh\n> > @@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '\n> >       )\n> >  '\n> >\n> > +#\n> > +# The following testcase just creates two simple renames (slightly modified\n> > +# on both sides but without conflicting changes), and a directory full of\n> > +# files that are otherwise uninteresting.  The setup is as follows:\n> > +#\n> > +#   base:     unrelated/<BUNCH OF FILES>\n> > +#             numbers\n> > +#             values\n> > +#   upstream: modify: numbers\n> > +#             modify: values\n> > +#   topic:    add: unrelated/foo\n> > +#             modify: numbers\n> > +#             modify: values\n> > +#             rename: numbers -> sequence\n> > +#             rename: values -> progression\n> > +#\n> > +# This is a trivial rename case, but we're curious what happens with a very\n> > +# low renameLimit interacting with the restart optimization trying to notice\n> > +# that unrelated/ looks like a trivial merge candidate.\n> > +#\n> > +test_expect_success 'avoid assuming we detected renames' '\n> > +     git init redo-weirdness &&\n> > +     (\n> > +             cd redo-weirdness &&\n> > +\n> > +             mkdir unrelated &&\n> > +             for i in $(test_seq 1 10)\n> > +             do\n> > +                     >unrelated/$i\n> > +             done &&\n> > +             test_seq  2 10 >numbers &&\n> > +             test_seq 12 20 >values &&\n> > +             git add numbers values unrelated/ &&\n> > +             git commit -m orig &&\n> > +\n> > +             git branch upstream &&\n> > +             git branch topic &&\n> > +\n> > +             git switch upstream &&\n> > +             test_seq  1 10 >numbers &&\n> > +             test_seq 11 20 >values &&\n> > +             git add numbers &&\n> > +             git commit -m \"Some tweaks\" &&\n> > +\n> > +             git switch topic &&\n> > +\n> > +             >unrelated/foo &&\n> > +             test_seq  2 12 >numbers &&\n> > +             test_seq 12 22 >values &&\n> > +             git add numbers values unrelated/ &&\n> > +             git mv numbers sequence &&\n> > +             git mv values progression &&\n> > +             git commit -m A &&\n> > +\n> > +             #\n> > +             # Actual testing\n> > +             #\n> > +\n> > +             git switch --detach topic^0 &&\n> > +\n> > +             test_must_fail git -c merge.renameLimit=1 rebase upstream &&\n> > +\n> > +             git ls-files -u >actual &&\n> > +             ! test_file_is_empty actual\n>\n> There is no 'test_file_is_empty' function, but because of the ! at the\n> beginning of the line it didn't fail the test.\n\nOops, looks like I meant test_must_be_empty.\n\n> The minimal fix would be to use 'test_file_not_empty' instead, but I\n> wonder whether we should use 'test_line_count = 2' instead for a tad\n> tighter check.\n\nMakes sense; since this merged about half a year ago, I'll submit a\nnew patch to fix this.  Thanks for catching and pointing it out!\n\n\n\n>\n> > +     )\n> > +'\n> > +\n> >  test_done\n> >\n> > base-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2\n> > --\n> > gitgitgadget\n"},{"id":"458337","messageId":"CABPp-BHSJqt9+cutx5qppdT6vuQFUqCoBsmfqdGj3vzh4Z_sVg@mail.gmail.com","threadId":"57250","inReplyTo":"CABPp-BG1MARvcqgPYBNQ0P1M9jZ2b8vt9btPG1oQ5_h_YM78TQ@mail.gmail.com","subject":"Re: [PATCH v2] merge-ort: avoid assuming all renames detected","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-01T05:21:39Z","receivedAt":"2022-07-01T05:21:55Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Jun 30, 2022 at 7:30 PM Elijah Newren <newren@gmail.com> wrote:\n>\n> On Thu, Jun 30, 2022 at 2:54 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:\n> >\n> > On Mon, Jan 17, 2022 at 06:25:55PM +0000, Elijah Newren via GitGitGadget wrote:\n> > > diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh\n> > > index 035edc40b1e..f2bc8a7d2a2 100755\n> > > --- a/t/t6429-merge-sequence-rename-caching.sh\n> > > +++ b/t/t6429-merge-sequence-rename-caching.sh\n[...]\n> > > +             git ls-files -u >actual &&\n> > > +             ! test_file_is_empty actual\n> >\n> > There is no 'test_file_is_empty' function, but because of the ! at the\n> > beginning of the line it didn't fail the test.\n>\n> Oops, looks like I meant test_must_be_empty.\n>\n> > The minimal fix would be to use 'test_file_not_empty' instead, but I\n> > wonder whether we should use 'test_line_count = 2' instead for a tad\n> > tighter check.\n>\n> Makes sense; since this merged about half a year ago, I'll submit a\n> new patch to fix this.  Thanks for catching and pointing it out!\n\nSubmitted over here:\nhttps://lore.kernel.org/git/pull.1276.git.1656652799863.gitgitgadget@gmail.com/\n"}]}