{"thread":{"id":"57898","subject":"[PATCH 1/2] merge: remove unused variable","startedAt":"2022-05-19T16:26:31Z","lastAt":"2023-03-06T22:19:31Z","messageCount":87,"participants":["Elijah Newren via GitGitGadget","Junio C Hamano","Elijah Newren","ZheNing Hu","Junio C Hamano via GitGitGadget","Eric Sunshine","Ævar Arnfjörð Bjarmason","Ben Humphreys","Rudy Rigot"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"455495","messageId":"042d624b8159364229e95d35e9309f12b67f8173.1652977582.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.git.1652977582.gitgitgadget@gmail.com","subject":"[PATCH 1/2] merge: remove unused variable","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-05-19T16:26:21Z","receivedAt":"2022-05-19T16:26:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nrestore_state() had a local variable sb that is not used, and in fact,\nwas never used even in the original commit that introduced it,\n1c7b76be7d (\"Build in merge\", 2008-07-07). Remove it.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 2 --\n 1 file changed, 2 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex f178f5a3ee1..00de224a2da 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -375,7 +375,6 @@ static void reset_hard(const struct object_id *oid, int verbose)\n static void restore_state(const struct object_id *head,\n \t\t\t const struct object_id *stash)\n {\n-\tstruct strbuf sb = STRBUF_INIT;\n \tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n \n \tif (is_null_oid(stash))\n@@ -391,7 +390,6 @@ static void restore_state(const struct object_id *head,\n \t */\n \trun_command_v_opt(args, RUN_GIT_CMD);\n \n-\tstrbuf_release(&sb);\n \trefresh_cache(REFRESH_QUIET);\n }\n \n-- \ngitgitgadget\n\n"},{"id":"455496","messageId":"pull.1231.git.1652977582.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":null,"subject":"[PATCH 0/2] Fix merge restore state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-05-19T16:26:20Z","receivedAt":"2022-05-19T16:26:35Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"A simple series to fix restore_state() in builtin/merge.c, fixing the issue\nreported by ZheNing Hu over here:\nhttps://lore.kernel.org/git/CAOLTT8R7QmpvaFPTRs3xTpxr7eiuxF-ZWtvUUSC0-JOo9Y+SqA@mail.gmail.com/\n\nElijah Newren (2):\n merge: remove unused variable\n merge: make restore_state() do as its name says\n\n builtin/merge.c | 12 ++++++------\n t/t7607-merge-state.sh | 25 +++++++++++++++++++++++++\n 2 files changed, 31 insertions(+), 6 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\n\nbase-commit: 6cd33dceed60949e2dbc32e3f0f5e67c4c882e1e\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1231%2Fnewren%2Ffix-merge-restore-state-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1231/newren/fix-merge-restore-state-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1231\n-- \ngitgitgadget\n"},{"id":"455497","messageId":"88bdca72a780d70e156e22e1ab96dedd368c761b.1652977582.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.git.1652977582.gitgitgadget@gmail.com","subject":"[PATCH 2/2] merge: make restore_state() do as its name says","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-05-19T16:26:22Z","receivedAt":"2022-05-19T16:26:37Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nPreviously, if the user:\n\n* Had no local changes before starting the merge\n* A merge strategy makes changes to the working tree/index but returns\n with exit status 2\n\nThen we'd call restore_state() to clean up the changes and either let\nthe next merge strategy run (if there is one), or exit telling the user\nthat no merge strategy could handle the merge. Unfortunately,\nrestore_state() did not clean up the changes as expected; that function\nwas a no-op if the stash was a null, and the stash would be null if\nthere were no local changes before starting the merge. So, instead of\n\"Rewinding the tree to pristine...\" as the code claimed, restore_state()\nwould leave garbage around in the index and working tree (possibly\nincluding conflicts) for either the next merge strategy or for the user\nafter aborting the merge. And in the case of aborting the merge, the\nuser would be unable to run \"git merge --abort\" to get rid of the\nunintended leftover conflicts, because the merge control files were not\nwritten as it was presumed that we had restored to a clean state\nalready.\n\nFix the main problem by making sure that restore_state() only skips the\nstash application if the stash is null rather than skipping the whole\nfunction.\n\nHowever, there is a secondary problem -- since merge.c forks\nsubprocesses to do the cleanup, the in-memory index is left out-of-sync.\nWhile there was a refresh_cache(REFRESH_QUIET) call that attempted to\ncorrect that, that function would not handle cases where the previous\nmerge strategy added conflicted entries. We need to drop the index and\nre-read it to handle such cases.\n\n(Alternatively, we could stop forking subprocesses and instead call some\nappropriate function to do the work which would update the in-memory\nindex automatically. For now, just do the simple fix.)\n\nReported-by: ZheNing Hu \nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 10 ++++++----\n t/t7607-merge-state.sh | 25 +++++++++++++++++++++++++\n 2 files changed, 31 insertions(+), 4 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 00de224a2da..ae3ee3a996b 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -377,11 +377,11 @@ static void restore_state(const struct object_id *head,\n {\n \tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n \n-\tif (is_null_oid(stash))\n-\t\treturn;\n-\n \treset_hard(head, 1);\n \n+\tif (is_null_oid(stash))\n+\t\tgoto refresh_cache;\n+\n \targs[2] = oid_to_hex(stash);\n \n \t/*\n@@ -390,7 +390,9 @@ static void restore_state(const struct object_id *head,\n \t */\n \trun_command_v_opt(args, RUN_GIT_CMD);\n \n-\trefresh_cache(REFRESH_QUIET);\n+refresh_cache:\n+\tif (discard_cache() < 0 || read_cache() < 0)\n+\t\tdie(_(\"could not read index\"));\n }\n \n /* This is called when no merge was necessary. */\ndiff --git a/t/t7607-merge-state.sh b/t/t7607-merge-state.sh\nnew file mode 100755\nindex 00000000000..655478cd0b3\n--- /dev/null\n+++ b/t/t7607-merge-state.sh\n@@ -0,0 +1,25 @@\n+#!/bin/sh\n+\n+test_description=\"Test that merge state is as expected after failed merge\"\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+. ./test-lib.sh\n+\n+test_expect_success 'set up custom strategy' '\n+\ttest_commit --no-tag \"Initial\" base base &&\n+git show-ref &&\n+\n+\tfor b in branch1 branch2 branch3\n+\tdo\n+\t\tgit checkout -b $b main &&\n+\t\ttest_commit --no-tag \"Change on $b\" base $b\n+\tdone &&\n+\n+\tgit checkout branch1 &&\n+\ttest_must_fail git merge branch2 branch3 &&\n+\tgit diff --exit-code --name-status &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n+test_done\n-- \ngitgitgadget\n"},{"id":"455500","messageId":"xmqq5ym1pgby.fsf@gitster.g","threadId":"57898","inReplyTo":"88bdca72a780d70e156e22e1ab96dedd368c761b.1652977582.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/2] merge: make restore_state() do as its name says","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-05-19T17:44:49Z","receivedAt":"2022-05-19T17:44:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 00de224a2da..ae3ee3a996b 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -377,11 +377,11 @@ static void restore_state(const struct object_id *head,\n> {\n> \tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n> \n> -\tif (is_null_oid(stash))\n> -\t\treturn;\n> -\n> \treset_hard(head, 1);\n\nwhen there is only one strategy to be tried, save_state() will never\nbe called. Removing the above safety means the hard-reset is\ndiscarding a local change that is not saved anywhere. The reason\nwhy the merge stopped may be because such a local change has crashed\nwith the change the merge wanted to bring in, no?\n\n> +\tif (is_null_oid(stash))\n> +\t\tgoto refresh_cache;\n> +\n> +test_description=\"Test that merge state is as expected after failed merge\"\n> +\n> +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n> +. ./test-lib.sh\n> +\n> +test_expect_success 'set up custom strategy' '\n> +\ttest_commit --no-tag \"Initial\" base base &&\n> +git show-ref &&\n> +\n> +\tfor b in branch1 branch2 branch3\n> +\tdo\n> +\t\tgit checkout -b $b main &&\n> +\t\ttest_commit --no-tag \"Change on $b\" base $b\n> +\tdone &&\n> +\n> +\tgit checkout branch1 &&\n\nHere, perhaps we can make two additional test cases, that try with\nlocal change that (1) overlaps with the changes branch2 and branch3\nbring in and that (2) does not overlap. I am worried about the case\n(2) losing the local change due to the call to reset_hard().\n\n> +\ttest_must_fail git merge branch2 branch3 &&\n> +\tgit diff --exit-code --name-status &&\n> +\ttest_path_is_missing .git/MERGE_HEAD\n> +'\n> +\n> +test_done\n"},{"id":"455501","messageId":"xmqq1qwppgbh.fsf@gitster.g","threadId":"57898","inReplyTo":"042d624b8159364229e95d35e9309f12b67f8173.1652977582.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/2] merge: remove unused variable","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-05-19T17:45:06Z","receivedAt":"2022-05-19T17:45:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> From: Elijah Newren \n>\n> restore_state() had a local variable sb that is not used, and in fact,\n> was never used even in the original commit that introduced it,\n> 1c7b76be7d (\"Build in merge\", 2008-07-07). Remove it.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 2 --\n> 1 file changed, 2 deletions(-)\n\nNice. Thanks.\n\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index f178f5a3ee1..00de224a2da 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -375,7 +375,6 @@ static void reset_hard(const struct object_id *oid, int verbose)\n> static void restore_state(const struct object_id *head,\n> \t\t\t const struct object_id *stash)\n> {\n> -\tstruct strbuf sb = STRBUF_INIT;\n> \tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n> \n> \tif (is_null_oid(stash))\n> @@ -391,7 +390,6 @@ static void restore_state(const struct object_id *head,\n> \t */\n> \trun_command_v_opt(args, RUN_GIT_CMD);\n> \n> -\tstrbuf_release(&sb);\n> \trefresh_cache(REFRESH_QUIET);\n> }\n"},{"id":"455524","messageId":"xmqqwnehnzk0.fsf@gitster.g","threadId":"57898","inReplyTo":"xmqq5ym1pgby.fsf@gitster.g","subject":"Re: [PATCH 2/2] merge: make restore_state() do as its name says","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-05-19T18:32:31Z","receivedAt":"2022-05-19T18:32:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n>> +test_expect_success 'set up custom strategy' '\n>> +\ttest_commit --no-tag \"Initial\" base base &&\n>> +git show-ref &&\n>> +\n>> +\tfor b in branch1 branch2 branch3\n>> +\tdo\n>> +\t\tgit checkout -b $b main &&\n>> +\t\ttest_commit --no-tag \"Change on $b\" base $b\n>> +\tdone &&\n>> +\n>> +\tgit checkout branch1 &&\n>\n> Here, perhaps we can make two additional test cases, that try with\n> local change that (1) overlaps with the changes branch2 and branch3\n> bring in and that (2) does not overlap. I am worried about the case\n> (2) losing the local change due to the call to reset_hard().\n\nWe do not need a new test to demonstrate the breakage in the\nproposed patch, I think. Here is one place I found that we already\ntest that merging in a dirty working tree fails. We only need to\nmake sure that we do so without losing local changes.\n\n t/t6424-merge-unrelated-index-changes.sh | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git c/t/t6424-merge-unrelated-index-changes.sh w/t/t6424-merge-unrelated-index-changes.sh\nindex 89dd544f38..88e0b541a0 100755\n--- c/t/t6424-merge-unrelated-index-changes.sh\n+++ w/t/t6424-merge-unrelated-index-changes.sh\n@@ -171,7 +171,8 @@ test_expect_success 'octopus, unrelated file touched' '\n \ttouch random_file && git add random_file &&\n \n \ttest_must_fail git merge C^0 D^0 &&\n-\ttest_path_is_missing .git/MERGE_HEAD\n+\ttest_path_is_missing .git/MERGE_HEAD &&\n+\ttest_path_exists random_file\n '\n \n test_expect_success 'octopus, related file removed' '\n"},{"id":"457079","messageId":"CABPp-BHfdo_JK-NyhMv=_7FpAGNj_vxGwGRoSYXRK==ARzh1Cw@mail.gmail.com","threadId":"57898","inReplyTo":"pull.1231.git.1652977582.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/2] Fix merge restore state","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-06-12T06:58:02Z","receivedAt":"2022-06-12T06:58:18Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, May 19, 2022 at 9:26 AM Elijah Newren via GitGitGadget\n wrote:\n>\n> A simple series to fix restore_state() in builtin/merge.c, fixing the issue\n> reported by ZheNing Hu over here:\n> https://lore.kernel.org/git/CAOLTT8R7QmpvaFPTRs3xTpxr7eiuxF-ZWtvUUSC0-JOo9Y+SqA@mail.gmail.com/\n\nI updated this series a few weeks ago, I believe addressing all the\nfeedback. But I hesitated to type \"/submit\" over at\nhttps://github.com/gitgitgadget/git/pull/1231. I've done virtually no\ndevelopment on \"interesting\" (to me) projects for quite some time; I'm\nonly responding to email questions and review requests on the list.\nAnd the review process takes quite a bit of work at times, and on the\nchance there was more feedback to address, I just didn't have it in me\nto submit a new round even though it _might_ be complete and certainly\nfixes some known issues. I haven't even gotten to review Dscho's\nupdates of my patches this whole week (spent the time I had mostly on\nthe merge-ort issue reported).\n\nDoes anyone else want to take this topic over? There are updated\npatches and even an updated cover letter over at\nhttps://github.com/gitgitgadget/git/pull/1231. It might involve\nnothing more than submitting those patches; they might be good enough\nalready. (Alternatively, I can type \"/submit\" to send them in...if\nsomeone else agrees to respond to any feedback.)\n"},{"id":"457086","messageId":"CAOLTT8RpGGioOyaMw5tkeWXmHpOaBW9UH8JghUvBRQ50ZcDdYQ@mail.gmail.com","threadId":"57898","inReplyTo":"CABPp-BHfdo_JK-NyhMv=_7FpAGNj_vxGwGRoSYXRK==ARzh1Cw@mail.gmail.com","subject":"Re: [PATCH 0/2] Fix merge restore state","fromName":"ZheNing Hu","fromEmail":"adlternative@gmail.com","sentAt":"2022-06-12T08:54:15Z","receivedAt":"2022-06-12T08:54:32Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"Elijah Newren 于2022年6月12日周日 14:58写道：\n>\n> On Thu, May 19, 2022 at 9:26 AM Elijah Newren via GitGitGadget\n> wrote:\n> >\n> > A simple series to fix restore_state() in builtin/merge.c, fixing the issue\n> > reported by ZheNing Hu over here:\n> > https://lore.kernel.org/git/CAOLTT8R7QmpvaFPTRs3xTpxr7eiuxF-ZWtvUUSC0-JOo9Y+SqA@mail.gmail.com/\n>\n> I updated this series a few weeks ago, I believe addressing all the\n> feedback. But I hesitated to type \"/submit\" over at\n> https://github.com/gitgitgadget/git/pull/1231. I've done virtually no\n> development on \"interesting\" (to me) projects for quite some time; I'm\n> only responding to email questions and review requests on the list.\n> And the review process takes quite a bit of work at times, and on the\n> chance there was more feedback to address, I just didn't have it in me\n> to submit a new round even though it _might_ be complete and certainly\n> fixes some known issues. I haven't even gotten to review Dscho's\n> updates of my patches this whole week (spent the time I had mostly on\n> the merge-ort issue reported).\n>\n\nSorry for missing this patch before. I have tried this patch, which\ncan perfectly\nsolve my origin problem. Now it can reset to origin state (Even when there are\nsome files in the work-tree here that have changed)\n\n> Does anyone else want to take this topic over? There are updated\n> patches and even an updated cover letter over at\n> https://github.com/gitgitgadget/git/pull/1231. It might involve\n> nothing more than submitting those patches; they might be good enough\n> already. (Alternatively, I can type \"/submit\" to send them in...if\n> someone else agrees to respond to any feedback.)\n\nYes, I think you can submit it, thanks very much!\n\n--\nZheNing Hu\n"},{"id":"457522","messageId":"pull.1231.v2.git.1655621424.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.git.1652977582.gitgitgadget@gmail.com","subject":"[PATCH v2 0/6] Fix merge restore state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-19T06:50:18Z","receivedAt":"2022-06-19T06:50:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"MAINTAINER NOTE: Rebased on latest main/master. (In particular, needs\nvd/sparse-stash; otherwise, the changes made here regress a\nensure-full-index testcase in t1092). Also, this fixes issues that predate\nthe v2.37 cycle, so this series can wait until v2.38 opens up.\n\nOther note: If this rounds needs updates, ZheNing Hu may be the one to\nrespond and make any necessary updates, as per [1].\n\n----------------------------------------------------------------------------\n\nThis is a simple series to fix restore_state() in builtin/merge.c, fixing\nthe issue reported by ZheNing Hu over here:\nhttps://lore.kernel.org/git/CAOLTT8R7QmpvaFPTRs3xTpxr7eiuxF-ZWtvUUSC0-JOo9Y+SqA@mail.gmail.com/\n\nChanges since v1:\n\n * Rebased\n * Included Junio's patch providing more testcases from\n https://lore.kernel.org/git/xmqqbkvtnyae.fsf@gitster.g/\n * Added three new patches to fix issues highlighted by Junio's testcases,\n in particular to (a) fix stashing with racy-dirty files present, (b) fix\n restoring staged state in restore_state(), and (c) ensure we can restore\n pre-merge state. All three were long-standing issues that we just hadn't\n noticed yet and thus are useful fixes on their own. However, my fix from\n v1 (which still remains as the final patch) does make it easier to notice\n these issues, and in particular that combined with Junio's new testcases\n unearthed those problems.\n\n[1]\nhttps://lore.kernel.org/git/CAOLTT8RpGGioOyaMw5tkeWXmHpOaBW9UH8JghUvBRQ50ZcDdYQ@mail.gmail.com/\n\nElijah Newren (5):\n merge: remove unused variable\n merge: fix save_state() to work when there are racy-dirty files\n merge: make restore_state() restore staged state too\n merge: ensure we can actually restore pre-merge state\n merge: do not exit restore_state() prematurely\n\nJunio C Hamano (1):\n t6424: make sure a failed merge preserves local changes\n\n builtin/merge.c | 32 ++++++++++++++----------\n t/t6424-merge-unrelated-index-changes.sh | 32 ++++++++++++++++++++++--\n t/t7607-merge-state.sh | 25 ++++++++++++++++++\n 3 files changed, 74 insertions(+), 15 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\n\nbase-commit: 8ddf593a250e07d388059f7e3f471078e1d2ed5c\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1231%2Fnewren%2Ffix-merge-restore-state-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1231/newren/fix-merge-restore-state-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/1231\n\nRange-diff vs v1:\n\n -: ----------- > 1: 6147e72c309 t6424: make sure a failed merge preserves local changes\n 1: 042d624b815 = 2: 230d84f09c8 merge: remove unused variable\n -: ----------- > 3: 89e5e633241 merge: fix save_state() to work when there are racy-dirty files\n -: ----------- > 4: 4a8b7c9e06d merge: make restore_state() restore staged state too\n -: ----------- > 5: a03075167c1 merge: ensure we can actually restore pre-merge state\n 2: 88bdca72a78 ! 6: 0783b48c121 merge: make restore_state() do as its name says\n @@ Metadata\n Author: Elijah Newren \n \n ## Commit message ##\n - merge: make restore_state() do as its name says\n + merge: do not exit restore_state() prematurely\n \n Previously, if the user:\n \n @@ Commit message\n ## builtin/merge.c ##\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n {\n - \tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n + \tconst char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n \n -\tif (is_null_oid(stash))\n -\t\treturn;\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n +\tif (is_null_oid(stash))\n +\t\tgoto refresh_cache;\n +\n - \targs[2] = oid_to_hex(stash);\n + \targs[3] = oid_to_hex(stash);\n \n \t/*\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n\n-- \ngitgitgadget\n"},{"id":"457523","messageId":"6147e72c3097fec85d6202a600d06b1ad222b49a.1655621424.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v2.git.1655621424.gitgitgadget@gmail.com","subject":"[PATCH v2 1/6] t6424: make sure a failed merge preserves local changes","fromName":"Junio C Hamano via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-19T06:50:19Z","receivedAt":"2022-06-19T06:50:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"From: Junio C Hamano \n\nWe do make sure that an attempt to merge with various forms of local\nchanges will \"fail\", but the point of stopping the merge is so that\nwe refrain from discarding uncommitted local changes that could be\nprecious. Add a few more checks for each case to make sure the\nlocal changes are left intact.\n\nSigned-off-by: Junio C Hamano \n---\n t/t6424-merge-unrelated-index-changes.sh | 32 ++++++++++++++++++++++--\n 1 file changed, 30 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex 89dd544f388..b6e424a427b 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -71,7 +71,9 @@ test_expect_success 'ff update' '\n \tgit merge E^0 &&\n \n \ttest_must_fail git rev-parse HEAD:random_file &&\n-\ttest \"$(git diff --name-only --cached E)\" = \"random_file\"\n+\ttest \"$(git diff --name-only --cached E)\" = \"random_file\" &&\n+\ttest_path_is_file random_file &&\n+\tgit rev-parse --verify :random_file\n '\n \n test_expect_success 'ff update, important file modified' '\n@@ -83,6 +85,8 @@ test_expect_success 'ff update, important file modified' '\n \tgit add subdir/e &&\n \n \ttest_must_fail git merge E^0 &&\n+\ttest_path_is_file subdir/e &&\n+\tgit rev-parse --verify :subdir/e &&\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n@@ -93,6 +97,8 @@ test_expect_success 'resolve, trivial' '\n \ttouch random_file && git add random_file &&\n \n \ttest_must_fail git merge -s resolve C^0 &&\n+\ttest_path_is_file random_file &&\n+\tgit rev-parse --verify :random_file &&\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n@@ -103,6 +109,8 @@ test_expect_success 'resolve, non-trivial' '\n \ttouch random_file && git add random_file &&\n \n \ttest_must_fail git merge -s resolve D^0 &&\n+\ttest_path_is_file random_file &&\n+\tgit rev-parse --verify :random_file &&\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n@@ -113,6 +121,8 @@ test_expect_success 'recursive' '\n \ttouch random_file && git add random_file &&\n \n \ttest_must_fail git merge -s recursive C^0 &&\n+\ttest_path_is_file random_file &&\n+\tgit rev-parse --verify :random_file &&\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n@@ -145,9 +155,12 @@ test_expect_success 'recursive, when file has staged changes not matching HEAD n\n \tmkdir subdir &&\n \ttest_seq 1 10 >subdir/a &&\n \tgit add subdir/a &&\n+\tgit rev-parse --verify :subdir/a >expect &&\n \n \t# We have staged changes; merge should error out\n \ttest_must_fail git merge -s recursive E^0 2>err &&\n+\tgit rev-parse --verify :subdir/a >actual &&\n+\ttest_cmp expect actual &&\n \ttest_i18ngrep \"changes to the following files would be overwritten\" err\n '\n \n@@ -158,9 +171,12 @@ test_expect_success 'recursive, when file has staged changes matching what a mer\n \tmkdir subdir &&\n \ttest_seq 1 11 >subdir/a &&\n \tgit add subdir/a &&\n+\tgit rev-parse --verify :subdir/a >expect &&\n \n \t# We have staged changes; merge should error out\n \ttest_must_fail git merge -s recursive E^0 2>err &&\n+\tgit rev-parse --verify :subdir/a >actual &&\n+\ttest_cmp expect actual &&\n \ttest_i18ngrep \"changes to the following files would be overwritten\" err\n '\n \n@@ -171,7 +187,9 @@ test_expect_success 'octopus, unrelated file touched' '\n \ttouch random_file && git add random_file &&\n \n \ttest_must_fail git merge C^0 D^0 &&\n-\ttest_path_is_missing .git/MERGE_HEAD\n+\ttest_path_is_missing .git/MERGE_HEAD &&\n+\tgit rev-parse --verify :random_file &&\n+\ttest_path_exists random_file\n '\n \n test_expect_success 'octopus, related file removed' '\n@@ -181,6 +199,8 @@ test_expect_success 'octopus, related file removed' '\n \tgit rm b &&\n \n \ttest_must_fail git merge C^0 D^0 &&\n+\ttest_path_is_missing b &&\n+\ttest_must_fail git rev-parse --verify :b &&\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n@@ -189,8 +209,12 @@ test_expect_success 'octopus, related file modified' '\n \tgit checkout B^0 &&\n \n \techo 12 >>a && git add a &&\n+\tgit rev-parse --verify :a >expect &&\n \n \ttest_must_fail git merge C^0 D^0 &&\n+\ttest_path_is_file a &&\n+\tgit rev-parse --verify :a >actual &&\n+\ttest_cmp expect actual &&\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n@@ -201,6 +225,8 @@ test_expect_success 'ours' '\n \ttouch random_file && git add random_file &&\n \n \ttest_must_fail git merge -s ours C^0 &&\n+\ttest_path_is_file random_file &&\n+\tgit rev-parse --verify :random_file &&\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n@@ -211,6 +237,8 @@ test_expect_success 'subtree' '\n \ttouch random_file && git add random_file &&\n \n \ttest_must_fail git merge -s subtree E^0 &&\n+\ttest_path_is_file random_file &&\n+\tgit rev-parse --verify :random_file &&\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n-- \ngitgitgadget\n\n"},{"id":"457524","messageId":"230d84f09c8b75d15168d5ff8205b543fa8ca91d.1655621424.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v2.git.1655621424.gitgitgadget@gmail.com","subject":"[PATCH v2 2/6] merge: remove unused variable","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-19T06:50:20Z","receivedAt":"2022-06-19T06:50:44Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nrestore_state() had a local variable sb that is not used, and in fact,\nwas never used even in the original commit that introduced it,\n1c7b76be7d (\"Build in merge\", 2008-07-07). Remove it.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 2 --\n 1 file changed, 2 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex f178f5a3ee1..00de224a2da 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -375,7 +375,6 @@ static void reset_hard(const struct object_id *oid, int verbose)\n static void restore_state(const struct object_id *head,\n \t\t\t const struct object_id *stash)\n {\n-\tstruct strbuf sb = STRBUF_INIT;\n \tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n \n \tif (is_null_oid(stash))\n@@ -391,7 +390,6 @@ static void restore_state(const struct object_id *head,\n \t */\n \trun_command_v_opt(args, RUN_GIT_CMD);\n \n-\tstrbuf_release(&sb);\n \trefresh_cache(REFRESH_QUIET);\n }\n \n-- \ngitgitgadget\n\n"},{"id":"457525","messageId":"4a8b7c9e06df36444b94b929b2558f40e3f72e81.1655621424.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v2.git.1655621424.gitgitgadget@gmail.com","subject":"[PATCH v2 4/6] merge: make restore_state() restore staged state too","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-19T06:50:22Z","receivedAt":"2022-06-19T06:50:46Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nmerge can be invoked with uncommitted changes, including staged changes.\nmerge is responsible for restoring this state if some of the merge\nstrategies make changes. However, it was not restoring staged changes\ndue to the lack of the \"--index\" option to \"git stash apply\". Add the\noption to fix this shortcoming.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 8ce4336dd3f..2dc56fab70b 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -383,14 +383,14 @@ static void reset_hard(const struct object_id *oid, int verbose)\n static void restore_state(const struct object_id *head,\n \t\t\t const struct object_id *stash)\n {\n-\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n+\tconst char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n \n \tif (is_null_oid(stash))\n \t\treturn;\n \n \treset_hard(head, 1);\n \n-\targs[2] = oid_to_hex(stash);\n+\targs[3] = oid_to_hex(stash);\n \n \t/*\n \t * It is OK to ignore error here, for example when there was\n-- \ngitgitgadget\n\n"},{"id":"457526","messageId":"89e5e633241e45a0c4b18289ab2fafdaabc8191e.1655621424.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v2.git.1655621424.gitgitgadget@gmail.com","subject":"[PATCH v2 3/6] merge: fix save_state() to work when there are racy-dirty files","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-19T06:50:21Z","receivedAt":"2022-06-19T06:50:46Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nWhen there are racy-dirty files, but no files are modified,\n`git stash create` exits with unsuccessful status. This causes merge\nto fail. Refresh the index first to avoid this problem.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 8 ++++++++\n 1 file changed, 8 insertions(+)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 00de224a2da..8ce4336dd3f 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -313,8 +313,16 @@ static int save_state(struct object_id *stash)\n \tint len;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tstruct strbuf buffer = STRBUF_INIT;\n+\tstruct lock_file lock_file = LOCK_INIT;\n+\tint fd;\n \tint rc = -1;\n \n+\tfd = repo_hold_locked_index(the_repository, &lock_file, 0);\n+\trefresh_cache(REFRESH_QUIET);\n+\tif (0 <= fd)\n+\t\trepo_update_index_if_able(the_repository, &lock_file);\n+\trollback_lock_file(&lock_file);\n+\n \tstrvec_pushl(&cp.args, \"stash\", \"create\", NULL);\n \tcp.out = -1;\n \tcp.git_cmd = 1;\n-- \ngitgitgadget\n\n"},{"id":"457527","messageId":"0783b48c121fe74051c13e7d9118d1a5b7cb9aa9.1655621424.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v2.git.1655621424.gitgitgadget@gmail.com","subject":"[PATCH v2 6/6] merge: do not exit restore_state() prematurely","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-19T06:50:24Z","receivedAt":"2022-06-19T06:50:48Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nPreviously, if the user:\n\n* Had no local changes before starting the merge\n* A merge strategy makes changes to the working tree/index but returns\n with exit status 2\n\nThen we'd call restore_state() to clean up the changes and either let\nthe next merge strategy run (if there is one), or exit telling the user\nthat no merge strategy could handle the merge. Unfortunately,\nrestore_state() did not clean up the changes as expected; that function\nwas a no-op if the stash was a null, and the stash would be null if\nthere were no local changes before starting the merge. So, instead of\n\"Rewinding the tree to pristine...\" as the code claimed, restore_state()\nwould leave garbage around in the index and working tree (possibly\nincluding conflicts) for either the next merge strategy or for the user\nafter aborting the merge. And in the case of aborting the merge, the\nuser would be unable to run \"git merge --abort\" to get rid of the\nunintended leftover conflicts, because the merge control files were not\nwritten as it was presumed that we had restored to a clean state\nalready.\n\nFix the main problem by making sure that restore_state() only skips the\nstash application if the stash is null rather than skipping the whole\nfunction.\n\nHowever, there is a secondary problem -- since merge.c forks\nsubprocesses to do the cleanup, the in-memory index is left out-of-sync.\nWhile there was a refresh_cache(REFRESH_QUIET) call that attempted to\ncorrect that, that function would not handle cases where the previous\nmerge strategy added conflicted entries. We need to drop the index and\nre-read it to handle such cases.\n\n(Alternatively, we could stop forking subprocesses and instead call some\nappropriate function to do the work which would update the in-memory\nindex automatically. For now, just do the simple fix.)\n\nReported-by: ZheNing Hu \nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 10 ++++++----\n t/t7607-merge-state.sh | 25 +++++++++++++++++++++++++\n 2 files changed, 31 insertions(+), 4 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex aaee8f6a553..a21dece1b55 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -385,11 +385,11 @@ static void restore_state(const struct object_id *head,\n {\n \tconst char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n \n-\tif (is_null_oid(stash))\n-\t\treturn;\n-\n \treset_hard(head, 1);\n \n+\tif (is_null_oid(stash))\n+\t\tgoto refresh_cache;\n+\n \targs[3] = oid_to_hex(stash);\n \n \t/*\n@@ -398,7 +398,9 @@ static void restore_state(const struct object_id *head,\n \t */\n \trun_command_v_opt(args, RUN_GIT_CMD);\n \n-\trefresh_cache(REFRESH_QUIET);\n+refresh_cache:\n+\tif (discard_cache() < 0 || read_cache() < 0)\n+\t\tdie(_(\"could not read index\"));\n }\n \n /* This is called when no merge was necessary. */\ndiff --git a/t/t7607-merge-state.sh b/t/t7607-merge-state.sh\nnew file mode 100755\nindex 00000000000..655478cd0b3\n--- /dev/null\n+++ b/t/t7607-merge-state.sh\n@@ -0,0 +1,25 @@\n+#!/bin/sh\n+\n+test_description=\"Test that merge state is as expected after failed merge\"\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+. ./test-lib.sh\n+\n+test_expect_success 'set up custom strategy' '\n+\ttest_commit --no-tag \"Initial\" base base &&\n+git show-ref &&\n+\n+\tfor b in branch1 branch2 branch3\n+\tdo\n+\t\tgit checkout -b $b main &&\n+\t\ttest_commit --no-tag \"Change on $b\" base $b\n+\tdone &&\n+\n+\tgit checkout branch1 &&\n+\ttest_must_fail git merge branch2 branch3 &&\n+\tgit diff --exit-code --name-status &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n+test_done\n-- \ngitgitgadget\n"},{"id":"457528","messageId":"a03075167c1f4410a1b4f415313f11a7e1c3a594.1655621424.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v2.git.1655621424.gitgitgadget@gmail.com","subject":"[PATCH v2 5/6] merge: ensure we can actually restore pre-merge state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-06-19T06:50:23Z","receivedAt":"2022-06-19T06:50:49Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nMerge strategies can fail -- not just have conflicts, but give up and\nsay that they are unable to handle the current type of merge. However,\nthey can also make changes to the index and working tree before giving\nup; merge-octopus does this, for example. Currently, we do not expect\nthe individual strategies to clean up after themselves, but instead\nexpect builtin/merge.c to do so. For it to be able to, it needs to save\nthe state before trying the merge strategy so it can have something to\nrestore to. Therefore, remove the shortcut bypassing the save_state()\ncall.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 8 +++-----\n 1 file changed, 3 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 2dc56fab70b..aaee8f6a553 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -1663,12 +1663,10 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t * tree in the index -- this means that the index must be in\n \t * sync with the head commit. The strategies are responsible\n \t * to ensure this.\n+\t *\n+\t * Stash away the local changes so that we can try more than one.\n \t */\n-\tif (use_strategies_nr == 1 ||\n-\t /*\n-\t * Stash away the local changes so that we can try more than one.\n-\t */\n-\t save_state(&stash))\n+\tif (save_state(&stash))\n \t\toidclr(&stash);\n \n \tfor (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n-- \ngitgitgadget\n\n"},{"id":"459225","messageId":"CAOLTT8THesUrMtov0L=pHKNfdABYvHeue6OzHre-sQD36=2e=g@mail.gmail.com","threadId":"57898","inReplyTo":"89e5e633241e45a0c4b18289ab2fafdaabc8191e.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 3/6] merge: fix save_state() to work when there are racy-dirty files","fromName":"ZheNing Hu","fromEmail":"adlternative@gmail.com","sentAt":"2022-07-17T16:28:06Z","receivedAt":"2022-07-17T16:28:24Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"Elijah Newren via GitGitGadget 于2022年6月19日周日 14:50写道：\n>\n> From: Elijah Newren \n>\n> When there are racy-dirty files, but no files are modified,\n> `git stash create` exits with unsuccessful status. This causes merge\n> to fail. Refresh the index first to avoid this problem.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 8 ++++++++\n> 1 file changed, 8 insertions(+)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 00de224a2da..8ce4336dd3f 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -313,8 +313,16 @@ static int save_state(struct object_id *stash)\n> int len;\n> struct child_process cp = CHILD_PROCESS_INIT;\n> struct strbuf buffer = STRBUF_INIT;\n> + struct lock_file lock_file = LOCK_INIT;\n> + int fd;\n> int rc = -1;\n>\n> + fd = repo_hold_locked_index(the_repository, &lock_file, 0);\n> + refresh_cache(REFRESH_QUIET);\n> + if (0 <= fd)\n> + repo_update_index_if_able(the_repository, &lock_file);\n> + rollback_lock_file(&lock_file);\n> +\n> strvec_pushl(&cp.args, \"stash\", \"create\", NULL);\n> cp.out = -1;\n> cp.git_cmd = 1;\n> --\n> gitgitgadget\n>\n\nI just want to show what sence will meet this errors:\n\n1. touch file\n2. git add file\n3. git stash push (user may do it before git merge)\n4. touch file (update file but not update its content)\n5. git merge (call git stash create and return 1)\n\nSo I have knew about what's the meaning of this patch:\n\nIf user do the git stash manually and update file timestamp\nbefore git merge, it may make next git stash failed. And\nrefresh index can make index entry sync with the work tree\nfile status.\n\nThanks.\n\nZheNing Hu\n"},{"id":"459226","messageId":"CAOLTT8RhMZDYM0tuTR64VQRPaj8ghkU=jJdOnodV1txsXXpUQA@mail.gmail.com","threadId":"57898","inReplyTo":"4a8b7c9e06df36444b94b929b2558f40e3f72e81.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 4/6] merge: make restore_state() restore staged state too","fromName":"ZheNing Hu","fromEmail":"adlternative@gmail.com","sentAt":"2022-07-17T16:37:40Z","receivedAt":"2022-07-17T16:37:54Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"Elijah Newren via GitGitGadget 于2022年6月19日周日 14:50写道：\n>\n> From: Elijah Newren \n>\n> merge can be invoked with uncommitted changes, including staged changes.\n> merge is responsible for restoring this state if some of the merge\n> strategies make changes. However, it was not restoring staged changes\n> due to the lack of the \"--index\" option to \"git stash apply\". Add the\n> option to fix this shortcoming.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 8ce4336dd3f..2dc56fab70b 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -383,14 +383,14 @@ static void reset_hard(const struct object_id *oid, int verbose)\n> static void restore_state(const struct object_id *head,\n> const struct object_id *stash)\n> {\n> - const char *args[] = { \"stash\", \"apply\", NULL, NULL };\n> + const char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n>\n> if (is_null_oid(stash))\n> return;\n>\n> reset_hard(head, 1);\n>\n> - args[2] = oid_to_hex(stash);\n> + args[3] = oid_to_hex(stash);\n>\n> /*\n> * It is OK to ignore error here, for example when there was\n> --\n> gitgitgadget\n>\n\nNow git merge (all strategies) can learn to restore origin index state.\nLGTM.\n\nZheNing Hu\n"},{"id":"459227","messageId":"CAOLTT8TCmDfbbakfXhAoFfx+4Zwn0brcdL0_hEM=u-eSAfjzMQ@mail.gmail.com","threadId":"57898","inReplyTo":"a03075167c1f4410a1b4f415313f11a7e1c3a594.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 5/6] merge: ensure we can actually restore pre-merge state","fromName":"ZheNing Hu","fromEmail":"adlternative@gmail.com","sentAt":"2022-07-17T16:41:30Z","receivedAt":"2022-07-17T16:41:44Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"Elijah Newren via GitGitGadget 于2022年6月19日周日 14:50写道：\n>\n> From: Elijah Newren \n>\n> Merge strategies can fail -- not just have conflicts, but give up and\n> say that they are unable to handle the current type of merge. However,\n> they can also make changes to the index and working tree before giving\n> up; merge-octopus does this, for example. Currently, we do not expect\n> the individual strategies to clean up after themselves, but instead\n> expect builtin/merge.c to do so. For it to be able to, it needs to save\n> the state before trying the merge strategy so it can have something to\n> restore to. Therefore, remove the shortcut bypassing the save_state()\n> call.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 8 +++-----\n> 1 file changed, 3 insertions(+), 5 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 2dc56fab70b..aaee8f6a553 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -1663,12 +1663,10 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> * tree in the index -- this means that the index must be in\n> * sync with the head commit. The strategies are responsible\n> * to ensure this.\n> + *\n> + * Stash away the local changes so that we can try more than one.\n> */\n> - if (use_strategies_nr == 1 ||\n> - /*\n> - * Stash away the local changes so that we can try more than one.\n> - */\n> - save_state(&stash))\n> + if (save_state(&stash))\n> oidclr(&stash);\n>\n> for (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n> --\n> gitgitgadget\n>\n\nSo now we will not make \"stash\" empty even if we are using\nonly one merge strategy (e.g. octopus), so we can reset to\nthe original state correctly.\n\nGood.\n\nZheNing Hu\n"},{"id":"459228","messageId":"CAOLTT8QtqVotu8Y0JYDFdxz2p2MvRitGgrfLJ_yCnT0XZoB40Q@mail.gmail.com","threadId":"57898","inReplyTo":"0783b48c121fe74051c13e7d9118d1a5b7cb9aa9.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 6/6] merge: do not exit restore_state() prematurely","fromName":"ZheNing Hu","fromEmail":"adlternative@gmail.com","sentAt":"2022-07-17T16:44:46Z","receivedAt":"2022-07-17T16:45:02Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"Elijah Newren via GitGitGadget 于2022年6月19日周日 14:50写道：\n>\n> From: Elijah Newren \n>\n> @@ -398,7 +398,9 @@ static void restore_state(const struct object_id *head,\n> */\n> run_command_v_opt(args, RUN_GIT_CMD);\n>\n> - refresh_cache(REFRESH_QUIET);\n> +refresh_cache:\n> + if (discard_cache() < 0 || read_cache() < 0)\n> + die(_(\"could not read index\"));\n> }\n>\n\nWe don't need to check discard_cache() return value,\nit's equal to zero constantly.\n\n> /* This is called when no merge was necessary. */\n> diff --git a/t/t7607-merge-state.sh b/t/t7607-merge-state.sh\n> new file mode 100755\n> index 00000000000..655478cd0b3\n> --- /dev/null\n> +++ b/t/t7607-merge-state.sh\n> @@ -0,0 +1,25 @@\n> +#!/bin/sh\n> +\n> +test_description=\"Test that merge state is as expected after failed merge\"\n> +\n> +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n> +. ./test-lib.sh\n> +\n> +test_expect_success 'set up custom strategy' '\n> + test_commit --no-tag \"Initial\" base base &&\n> +git show-ref &&\n> +\n> + for b in branch1 branch2 branch3\n> + do\n> + git checkout -b $b main &&\n> + test_commit --no-tag \"Change on $b\" base $b\n> + done &&\n> +\n> + git checkout branch1 &&\n> + test_must_fail git merge branch2 branch3 &&\n> + git diff --exit-code --name-status &&\n> + test_path_is_missing .git/MERGE_HEAD\n> +'\n> +\n\nLittle typo: less a small tab before \"git show ref\"?\n\n> +test_done\n> --\n> gitgitgadget\n"},{"id":"459479","messageId":"xmqqsfmw7mgx.fsf@gitster.g","threadId":"57898","inReplyTo":"89e5e633241e45a0c4b18289ab2fafdaabc8191e.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 3/6] merge: fix save_state() to work when there are racy-dirty files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T22:43:10Z","receivedAt":"2022-07-19T22:43:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> From: Elijah Newren \n>\n> When there are racy-dirty files, but no files are modified,\n> `git stash create` exits with unsuccessful status. This causes merge\n> to fail. Refresh the index first to avoid this problem.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 8 ++++++++\n> 1 file changed, 8 insertions(+)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 00de224a2da..8ce4336dd3f 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -313,8 +313,16 @@ static int save_state(struct object_id *stash)\n> \tint len;\n> \tstruct child_process cp = CHILD_PROCESS_INIT;\n> \tstruct strbuf buffer = STRBUF_INIT;\n> +\tstruct lock_file lock_file = LOCK_INIT;\n> +\tint fd;\n> \tint rc = -1;\n> \n> +\tfd = repo_hold_locked_index(the_repository, &lock_file, 0);\n> +\trefresh_cache(REFRESH_QUIET);\n> +\tif (0 <= fd)\n> +\t\trepo_update_index_if_able(the_repository, &lock_file);\n> +\trollback_lock_file(&lock_file);\n\nI might have added \"else\" but rolling back a lock file that was\nalready committed or rolled back is a safe no-op, so this is OK.\nThe pattern already appears elsewhere twice, anyway.\n\nIs it sufficient to be opportunistic? IOW, if we fail to refresh\nthe index or write the refreshed result to disk, can we be silent\nhere and rely on \"stash create\" and things that follow to safely\nfail as necessary, or should we also be detecting errors?\n\n> \tstrvec_pushl(&cp.args, \"stash\", \"create\", NULL);\n> \tcp.out = -1;\n> \tcp.git_cmd = 1;\n"},{"id":"459480","messageId":"xmqqo7xk7m73.fsf@gitster.g","threadId":"57898","inReplyTo":"CAOLTT8THesUrMtov0L=pHKNfdABYvHeue6OzHre-sQD36=2e=g@mail.gmail.com","subject":"Re: [PATCH v2 3/6] merge: fix save_state() to work when there are racy-dirty files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T22:49:04Z","receivedAt":"2022-07-19T22:49:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"ZheNing Hu writes:\n\n> Elijah Newren via GitGitGadget 于2022年6月19日周日 14:50写道：\n>>\n>> From: Elijah Newren \n>>\n>> When there are racy-dirty files, but no files are modified,\n>> `git stash create` exits with unsuccessful status. This causes merge\n>> to fail. Refresh the index first to avoid this problem.\n\nRacily dirty? Or just being stat-dirty is sufficient to cause the\n\"stash create\" to fail?\n\n> I just want to show what sence will meet this errors:\n>\n> 1. touch file\n> 2. git add file\n> 3. git stash push (user may do it before git merge)\n> 4. touch file (update file but not update its content)\n> 5. git merge (call git stash create and return 1)\n\nI think, from the above reproduction recipe, that the breakage does\nnot depend on racily-clean index entries (i.e. file touched within\nthe same timestamp as the last write of the index without changing\ntheir size). So s/racy-dirty/stat-dirty/ (both on the title and the\nbody) would be a sufficient fix.\n\nThanks.\n"},{"id":"459481","messageId":"xmqq7d487lsq.fsf@gitster.g","threadId":"57898","inReplyTo":"a03075167c1f4410a1b4f415313f11a7e1c3a594.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 5/6] merge: ensure we can actually restore pre-merge state","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T22:57:41Z","receivedAt":"2022-07-19T22:57:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> From: Elijah Newren \n>\n> Merge strategies can fail -- not just have conflicts, but give up and\n> say that they are unable to handle the current type of merge. However,\n> they can also make changes to the index and working tree before giving\n> up; merge-octopus does this, for example. Currently, we do not expect\n> the individual strategies to clean up after themselves, but instead\n> expect builtin/merge.c to do so. For it to be able to, it needs to save\n> the state before trying the merge strategy so it can have something to\n> restore to. Therefore, remove the shortcut bypassing the save_state()\n> call.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 8 +++-----\n> 1 file changed, 3 insertions(+), 5 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 2dc56fab70b..aaee8f6a553 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -1663,12 +1663,10 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> \t * tree in the index -- this means that the index must be in\n> \t * sync with the head commit. The strategies are responsible\n> \t * to ensure this.\n> +\t *\n> +\t * Stash away the local changes so that we can try more than one.\n> \t */\n\nThe comment explains why we limited the save_state() to avoid wasted\ncycles and SSD wear and tear by looking at the number of strategies.\nBut because we are removing the restriction (which I am not 100%\nsure is a good idea), \"so that we can try more than one\" no longer\napplies as the reason why we run save_state() here.\n\n> -\tif (use_strategies_nr == 1 ||\n> -\t /*\n> -\t * Stash away the local changes so that we can try more than one.\n> -\t */\n> -\t save_state(&stash))\n> +\tif (save_state(&stash))\n> \t\toidclr(&stash);\n> \n> \tfor (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n"},{"id":"459482","messageId":"xmqqzgh466ia.fsf@gitster.g","threadId":"57898","inReplyTo":"0783b48c121fe74051c13e7d9118d1a5b7cb9aa9.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 6/6] merge: do not exit restore_state() prematurely","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T23:13:17Z","receivedAt":"2022-07-19T23:13:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> Fix the main problem by making sure that restore_state() only skips the\n> stash application if the stash is null rather than skipping the whole\n> function.\n\nOK.\n\n\n> However, there is a secondary problem -- since merge.c forks\n> subprocesses to do the cleanup, the in-memory index is left out-of-sync.\n> While there was a refresh_cache(REFRESH_QUIET) call that attempted to\n> correct that, that function would not handle cases where the previous\n> merge strategy added conflicted entries. We need to drop the index and\n> re-read it to handle such cases.\n\nAbsolutely right.\n\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index aaee8f6a553..a21dece1b55 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -385,11 +385,11 @@ static void restore_state(const struct object_id *head,\n> {\n> \tconst char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n> \n> -\tif (is_null_oid(stash))\n> -\t\treturn;\n> -\n> \treset_hard(head, 1);\n> \n> +\tif (is_null_oid(stash))\n> +\t\tgoto refresh_cache;\n> +\n> \targs[3] = oid_to_hex(stash);\n> \n> \t/*\n> @@ -398,7 +398,9 @@ static void restore_state(const struct object_id *head,\n> \t */\n> \trun_command_v_opt(args, RUN_GIT_CMD);\n> \n> -\trefresh_cache(REFRESH_QUIET);\n> +refresh_cache:\n> +\tif (discard_cache() < 0 || read_cache() < 0)\n> +\t\tdie(_(\"could not read index\"));\n\nDon't we need refresh_cache() after re-reading the on-disk index, or\ndo we have nothing to do further after restore_state() returns and\nthe stat-info being stale does not matter? Given that [3/6] exists,\nI suspect that we do want to make sure the in-core index is refreshed\nbefore we go ahead and run the next merge, no?\n\n> }\n> \n> /* This is called when no merge was necessary. */\n\n> diff --git a/t/t7607-merge-state.sh b/t/t7607-merge-state.sh\n> new file mode 100755\n\nAs long we are adding a brand-new script for new tests, probably we\nshould add tests for other steps (like [4/6]) here, perhaps?\n\n> index 00000000000..655478cd0b3\n> --- /dev/null\n> +++ b/t/t7607-merge-state.sh\n> @@ -0,0 +1,25 @@\n> +#!/bin/sh\n> +\n> +test_description=\"Test that merge state is as expected after failed merge\"\n> +\n> +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n> +. ./test-lib.sh\n> +\n> +test_expect_success 'set up custom strategy' '\n> +\ttest_commit --no-tag \"Initial\" base base &&\n> +git show-ref &&\n\nIs this part of the test, or a leftover debugging aid?\n\n> +\n> +\tfor b in branch1 branch2 branch3\n> +\tdo\n> +\t\tgit checkout -b $b main &&\n> +\t\ttest_commit --no-tag \"Change on $b\" base $b\n> +\tdone &&\n> +\n> +\tgit checkout branch1 &&\n> +\ttest_must_fail git merge branch2 branch3 &&\n> +\tgit diff --exit-code --name-status &&\n> +\ttest_path_is_missing .git/MERGE_HEAD\n> +'\n\nHmph, I am not sure if the new behaviour is not too pessimistic.\nWhen octopus fails after successfully merging branch2 and then\nfailing the merge of branch3 (i.e. the last one) due to conflict,\nI think octpus users are used to be able to resolve it manually\nand make a commit. Are we making it impossible by doing the\nreset-restore dance here?\n\nI do not use, and more importantly, I do not recommend others to\nuse, Octopus anymore, and from that point of view, it is a good move\nto make Octopus harder to use on any non-trivial merge, but those\nwho still like Octopus may disagree.\n\nThanks.\n\n> +test_done\n"},{"id":"459483","messageId":"xmqqsfmw66gq.fsf@gitster.g","threadId":"57898","inReplyTo":"4a8b7c9e06df36444b94b929b2558f40e3f72e81.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 4/6] merge: make restore_state() restore staged state too","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T23:14:13Z","receivedAt":"2022-07-19T23:14:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> From: Elijah Newren \n>\n> merge can be invoked with uncommitted changes, including staged changes.\n> merge is responsible for restoring this state if some of the merge\n> strategies make changes. However, it was not restoring staged changes\n> due to the lack of the \"--index\" option to \"git stash apply\". Add the\n> option to fix this shortcoming.\n\nShouldn't this be testable?\n\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 8ce4336dd3f..2dc56fab70b 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -383,14 +383,14 @@ static void reset_hard(const struct object_id *oid, int verbose)\n> static void restore_state(const struct object_id *head,\n> \t\t\t const struct object_id *stash)\n> {\n> -\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n> +\tconst char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n> \n> \tif (is_null_oid(stash))\n> \t\treturn;\n> \n> \treset_hard(head, 1);\n> \n> -\targs[2] = oid_to_hex(stash);\n> +\targs[3] = oid_to_hex(stash);\n> \n> \t/*\n> \t * It is OK to ignore error here, for example when there was\n\nOK.\n\n\n"},{"id":"459484","messageId":"xmqqleso66gj.fsf@gitster.g","threadId":"57898","inReplyTo":"230d84f09c8b75d15168d5ff8205b543fa8ca91d.1655621424.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 2/6] merge: remove unused variable","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T23:14:20Z","receivedAt":"2022-07-19T23:14:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> From: Elijah Newren \n>\n> restore_state() had a local variable sb that is not used, and in fact,\n> was never used even in the original commit that introduced it,\n> 1c7b76be7d (\"Build in merge\", 2008-07-07). Remove it.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 2 --\n> 1 file changed, 2 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index f178f5a3ee1..00de224a2da 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -375,7 +375,6 @@ static void reset_hard(const struct object_id *oid, int verbose)\n> static void restore_state(const struct object_id *head,\n> \t\t\t const struct object_id *stash)\n> {\n> -\tstruct strbuf sb = STRBUF_INIT;\n> \tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n> \n> \tif (is_null_oid(stash))\n> @@ -391,7 +390,6 @@ static void restore_state(const struct object_id *head,\n> \t */\n> \trun_command_v_opt(args, RUN_GIT_CMD);\n> \n> -\tstrbuf_release(&sb);\n> \trefresh_cache(REFRESH_QUIET);\n> }\n\nObviously correct ;-)\n"},{"id":"459486","messageId":"xmqq7d4865t6.fsf@gitster.g","threadId":"57898","inReplyTo":"xmqqsfmw66gq.fsf@gitster.g","subject":"Re: [PATCH v2 4/6] merge: make restore_state() restore staged state too","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T23:28:21Z","receivedAt":"2022-07-19T23:28:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> \"Elijah Newren via GitGitGadget\" writes:\n>\n>> From: Elijah Newren \n>>\n>> merge can be invoked with uncommitted changes, including staged changes.\n>> merge is responsible for restoring this state if some of the merge\n>> strategies make changes. However, it was not restoring staged changes\n>> due to the lack of the \"--index\" option to \"git stash apply\". Add the\n>> option to fix this shortcoming.\n>\n> Shouldn't this be testable?\n\nI actually take this part (which implied that the change is a good\nidea) back. I think we have clearly documented for the past 17\nyears that you can have local changes but your index must match the\nHEAD before you start your merge.\n\nIf \"stash apply\" vs \"stash apply --index\" makes any difference,\nthere is something wrong. We should be aborting the \"git merge\"\neven before we even start mucking with the working tree and the\nindex with strategies, no? I think it is the bug, if this change\nmakes any difference, to be fixed---we shouldn't be proceeding to\neven create a stash with index changes to begin with.\n\n>\n>> Signed-off-by: Elijah Newren \n>> ---\n>> builtin/merge.c | 4 ++--\n>> 1 file changed, 2 insertions(+), 2 deletions(-)\n>>\n>> diff --git a/builtin/merge.c b/builtin/merge.c\n>> index 8ce4336dd3f..2dc56fab70b 100644\n>> --- a/builtin/merge.c\n>> +++ b/builtin/merge.c\n>> @@ -383,14 +383,14 @@ static void reset_hard(const struct object_id *oid, int verbose)\n>> static void restore_state(const struct object_id *head,\n>> \t\t\t const struct object_id *stash)\n>> {\n>> -\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n>> +\tconst char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n>> \n>> \tif (is_null_oid(stash))\n>> \t\treturn;\n>> \n>> \treset_hard(head, 1);\n>> \n>> -\targs[2] = oid_to_hex(stash);\n>> +\targs[3] = oid_to_hex(stash);\n>> \n>> \t/*\n>> \t * It is OK to ignore error here, for example when there was\n>\n> OK.\n"},{"id":"459488","messageId":"CAPig+cTqN6nVRGhm-bG6F2RGACAKi7RYa4yp_qqsp7F5pWVaBA@mail.gmail.com","threadId":"57898","inReplyTo":"xmqqzgh466ia.fsf@gitster.g","subject":"Re: [PATCH v2 6/6] merge: do not exit restore_state() prematurely","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-07-20T00:09:03Z","receivedAt":"2022-07-20T00:09:18Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"[replying to Junio's email since I don't have the original available...]\n\nOn Tue, Jul 19, 2022 at 7:22 PM Junio C Hamano wrote:\n> \"Elijah Newren via GitGitGadget\" writes:\n> > + for b in branch1 branch2 branch3\n> > + do\n> > + git checkout -b $b main &&\n> > + test_commit --no-tag \"Change on $b\" base $b\n> > + done &&\n\nLet's break out of the loop with `|| return 1` if something in the\nloop body fails.\n\n for b in branch1 branch2 branch3\n do\n git checkout -b $b main &&\n test_commit --no-tag \"Change on $b\" base $b || return 1\n done &&\n"},{"id":"459559","messageId":"CABPp-BGMzyJR3C2EREXV5y50N30c9EuVTt9mExcxnfQT2ot3Vg@mail.gmail.com","threadId":"57898","inReplyTo":"xmqqo7xk7m73.fsf@gitster.g","subject":"Re: [PATCH v2 3/6] merge: fix save_state() to work when there are racy-dirty files","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-21T01:09:55Z","receivedAt":"2022-07-21T01:10:12Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 19, 2022 at 3:49 PM Junio C Hamano wrote:\n>\n> ZheNing Hu writes:\n>\n> > Elijah Newren via GitGitGadget 于2022年6月19日周日 14:50写道：\n> >>\n> >> From: Elijah Newren \n> >>\n> >> When there are racy-dirty files, but no files are modified,\n> >> `git stash create` exits with unsuccessful status. This causes merge\n> >> to fail. Refresh the index first to avoid this problem.\n>\n> Racily dirty? Or just being stat-dirty is sufficient to cause the\n> \"stash create\" to fail?\n>\n> > I just want to show what sence will meet this errors:\n> >\n> > 1. touch file\n> > 2. git add file\n> > 3. git stash push (user may do it before git merge)\n> > 4. touch file (update file but not update its content)\n> > 5. git merge (call git stash create and return 1)\n>\n> I think, from the above reproduction recipe, that the breakage does\n> not depend on racily-clean index entries (i.e. file touched within\n> the same timestamp as the last write of the index without changing\n> their size). So s/racy-dirty/stat-dirty/ (both on the title and the\n> body) would be a sufficient fix.\n\nYep, stat-dirty. I'll fix up the title and body; thanks.\n"},{"id":"459560","messageId":"CABPp-BEfpwmEcR3TR8xu_+1TQfOvOTQceNE7-H76rsno6TmFPg@mail.gmail.com","threadId":"57898","inReplyTo":"xmqq7d4865t6.fsf@gitster.g","subject":"Re: [PATCH v2 4/6] merge: make restore_state() restore staged state too","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-21T01:37:05Z","receivedAt":"2022-07-21T01:37:21Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 19, 2022 at 4:28 PM Junio C Hamano wrote:\n>\n> Junio C Hamano writes:\n>\n> > \"Elijah Newren via GitGitGadget\" writes:\n> >\n> >> From: Elijah Newren \n> >>\n> >> merge can be invoked with uncommitted changes, including staged changes.\n> >> merge is responsible for restoring this state if some of the merge\n> >> strategies make changes. However, it was not restoring staged changes\n> >> due to the lack of the \"--index\" option to \"git stash apply\". Add the\n> >> option to fix this shortcoming.\n> >\n> > Shouldn't this be testable?\n\nYes, I will add a test.\n\n> I actually take this part (which implied that the change is a good\n> idea) back. I think we have clearly documented for the past 17\n> years that you can have local changes but your index must match the\n> HEAD before you start your merge.\n\nActually, we don't enforce that the index must match HEAD in all\ncases, as noted in commit 55f39cf755 (\"merge: fix misleading pre-merge\ncheck documentation\", 2018-06-30). That commit also pointed out how\nthe documentation was a bit unclear in this area.\n\nWe also apparently fail to enforce the condition in at least two cases\nthat weren't a valid exception, which I just found while working on a\ntestcase for this patch. (Thus, we have one more sordid tale to add\nto the saga in commit 9822175d2b (\"Ensure index matches head before\ninvoking merge machinery, round N\", 2019-08-17))\n\nHowever, the failed enforcement and the \"valid\" special exceptions\naren't too relevant here, so...\n\n> If \"stash apply\" vs \"stash apply --index\" makes any difference,\n> there is something wrong. We should be aborting the \"git merge\"\n> even before we even start mucking with the working tree and the\n> index with strategies, no? I think it is the bug, if this change\n> makes any difference, to be fixed---we shouldn't be proceeding to\n> even create a stash with index changes to begin with.\n\nI agree with you that generally if the index does not match HEAD, then\n(A) we should abort the merge, and (B) the working tree and index need\nto be left intact when the merge aborts.\n\nBut I don't think your conclusion follows from those two items,\nbecause of the last sentence of this comment:\n\n /*\n * At this point, we need a real merge. No matter what strategy\n * we use, it would operate on the index, possibly affecting the\n * working tree, and when resolved cleanly, have the desired\n * tree in the index -- this means that the index must be in\n * sync with the head commit. The strategies are responsible\n * to ensure this.\n */\n\nDue to this requirement, if a user has staged changes before starting\nthe merge, builtin/merge.c will:\n\n * stash the changes\n * try all the merge strategies in turn, each of which report they\ncannot function due to index not matching HEAD\n * restore the changes via \"git stash apply\"\n\nThis sequence has the net effect of not quite cleanly aborting the\nmerge -- it also unstashes the user's changes.\n\nOne way to fix this problem is the simple patch I proposed. An\nalternative fix would be to rip out the extra code from all the merge\nstrategies that enforces the index matches HEAD requirement, and then\nadding enforcement of that condition early in builtin/merge.c. That\nalternative fix probably would have saved us from a lot of the\nheadache detailed in commit 9822175d2b above, but it may also make\nrecursive and ort a bit slower (which had relied on unpack-trees to do\nsome of this checking, and thus they'd have some redundant checks).\n"},{"id":"459561","messageId":"CABPp-BH0kWHNtwjX7HsBztP_MRB3xEQsGNmZzNj-WGxFTYuhfQ@mail.gmail.com","threadId":"57898","inReplyTo":"xmqq7d487lsq.fsf@gitster.g","subject":"Re: [PATCH v2 5/6] merge: ensure we can actually restore pre-merge state","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-21T02:03:13Z","receivedAt":"2022-07-21T02:03:27Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 19, 2022 at 3:57 PM Junio C Hamano wrote:\n>\n> \"Elijah Newren via GitGitGadget\" writes:\n>\n> > From: Elijah Newren \n> >\n> > Merge strategies can fail -- not just have conflicts, but give up and\n> > say that they are unable to handle the current type of merge. However,\n> > they can also make changes to the index and working tree before giving\n> > up; merge-octopus does this, for example. Currently, we do not expect\n> > the individual strategies to clean up after themselves, but instead\n> > expect builtin/merge.c to do so. For it to be able to, it needs to save\n> > the state before trying the merge strategy so it can have something to\n> > restore to. Therefore, remove the shortcut bypassing the save_state()\n> > call.\n> >\n> > Signed-off-by: Elijah Newren \n> > ---\n> > builtin/merge.c | 8 +++-----\n> > 1 file changed, 3 insertions(+), 5 deletions(-)\n> >\n> > diff --git a/builtin/merge.c b/builtin/merge.c\n> > index 2dc56fab70b..aaee8f6a553 100644\n> > --- a/builtin/merge.c\n> > +++ b/builtin/merge.c\n> > @@ -1663,12 +1663,10 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> > * tree in the index -- this means that the index must be in\n> > * sync with the head commit. The strategies are responsible\n> > * to ensure this.\n> > + *\n> > + * Stash away the local changes so that we can try more than one.\n> > */\n>\n> The comment explains why we limited the save_state() to avoid wasted\n> cycles and SSD wear and tear by looking at the number of strategies.\n> But because we are removing the restriction (which I am not 100%\n> sure is a good idea), \"so that we can try more than one\" no longer\n> applies as the reason why we run save_state() here.\n\nI should probably change it to \"Stash away the local changes so that\nwe can try more than one and/or recover from merge strategies\nbailing\".\n\nIn regards to the \"good idea\" side, I don't really like it either, but:\n\n 1. Merge strategies are allowed to make whatever changes they want\nto the index and working tree. They have no requirement to clean up\nafter themselves.\n 2. Merge strategies are allowed to bail and say, \"Nevermind, not\nonly can I not successfully merge this, I can't even leave conflicts\nfor the users to resolve; I just can't handle it at all. Try some\nother strategy.\" (See the \"exit 2\" codepath of commit 98efc8f3d8\n(\"octopus: allow manual resolve on the last round.\", 2006-01-13), for\nexample)\n 3. Merge strategies can bail with the \"try some other strategy\"\nresponse _after_ mucking with the index and working tree. octopus\ndoes (again, see the commit mentioned above)\n 4. builtin/merge.c previously would _only_ cleanup for merge\nstrategies if there was more than 1.\n\nThis combination is clearly broken. We need to fix either item 1 or\nitem 4 (or maybe item 3?). Since 1 might run into issues with\nuser-custom merge strategies which may have been written to mimic\noctopus and thus rely on the assurances in items 1-3 above, I figured\nfixing item 4 was the easiest.\n"},{"id":"459562","messageId":"CABPp-BHTJR5dUGKqKoH8TUGPZYvQn27E978HkMqPSzcctKmA7w@mail.gmail.com","threadId":"57898","inReplyTo":"CAPig+cTqN6nVRGhm-bG6F2RGACAKi7RYa4yp_qqsp7F5pWVaBA@mail.gmail.com","subject":"Re: [PATCH v2 6/6] merge: do not exit restore_state() prematurely","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-21T02:03:24Z","receivedAt":"2022-07-21T02:03:38Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 19, 2022 at 5:09 PM Eric Sunshine wrote:\n>\n> [replying to Junio's email since I don't have the original available...]\n>\n> On Tue, Jul 19, 2022 at 7:22 PM Junio C Hamano wrote:\n> > \"Elijah Newren via GitGitGadget\" writes:\n> > > + for b in branch1 branch2 branch3\n> > > + do\n> > > + git checkout -b $b main &&\n> > > + test_commit --no-tag \"Change on $b\" base $b\n> > > + done &&\n>\n> Let's break out of the loop with `|| return 1` if something in the\n> loop body fails.\n>\n> for b in branch1 branch2 branch3\n> do\n> git checkout -b $b main &&\n> test_commit --no-tag \"Change on $b\" base $b || return 1\n> done &&\n\nOkay, will do.\n"},{"id":"459564","messageId":"CABPp-BEywagXQtVQVxZgTCaH+TxrA9VBrMr2Lyv_3uADoqYY+Q@mail.gmail.com","threadId":"57898","inReplyTo":"xmqqzgh466ia.fsf@gitster.g","subject":"Re: [PATCH v2 6/6] merge: do not exit restore_state() prematurely","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-21T03:27:06Z","receivedAt":"2022-07-21T03:27:22Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 19, 2022 at 4:13 PM Junio C Hamano wrote:\n>\n> \"Elijah Newren via GitGitGadget\" writes:\n>\n> > Fix the main problem by making sure that restore_state() only skips the\n> > stash application if the stash is null rather than skipping the whole\n> > function.\n>\n> OK.\n>\n>\n> > However, there is a secondary problem -- since merge.c forks\n> > subprocesses to do the cleanup, the in-memory index is left out-of-sync.\n> > While there was a refresh_cache(REFRESH_QUIET) call that attempted to\n> > correct that, that function would not handle cases where the previous\n> > merge strategy added conflicted entries. We need to drop the index and\n> > re-read it to handle such cases.\n>\n> Absolutely right.\n>\n> > diff --git a/builtin/merge.c b/builtin/merge.c\n> > index aaee8f6a553..a21dece1b55 100644\n> > --- a/builtin/merge.c\n> > +++ b/builtin/merge.c\n> > @@ -385,11 +385,11 @@ static void restore_state(const struct object_id *head,\n> > {\n> > const char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n> >\n> > - if (is_null_oid(stash))\n> > - return;\n> > -\n> > reset_hard(head, 1);\n> >\n> > + if (is_null_oid(stash))\n> > + goto refresh_cache;\n> > +\n> > args[3] = oid_to_hex(stash);\n> >\n> > /*\n> > @@ -398,7 +398,9 @@ static void restore_state(const struct object_id *head,\n> > */\n> > run_command_v_opt(args, RUN_GIT_CMD);\n> >\n> > - refresh_cache(REFRESH_QUIET);\n> > +refresh_cache:\n> > + if (discard_cache() < 0 || read_cache() < 0)\n> > + die(_(\"could not read index\"));\n>\n> Don't we need refresh_cache() after re-reading the on-disk index, or\n> do we have nothing to do further after restore_state() returns and\n> the stat-info being stale does not matter? Given that [3/6] exists,\n> I suspect that we do want to make sure the in-core index is refreshed\n> before we go ahead and run the next merge, no?\n\nI don't think so; the situation for [3/6] is different. The basic\ntimeline is as follows:\n 1. \n 2. User decides to merge one or more branches\n 3. merge does save_state() [i.e. \"git stash create\"]\n 4. The first strategy fails\n 5. We restore the state before trying the next strategy\n\nThe current code is dealing with step 5. Patch [3/6] was to prevent\nfailures in step 3 from users creating stat-dirty files in step 1.\nOnce step 3 runs, the only way to become stat-dirty again is if the\nuser simultaneously messes with their checkout while the \"git merge\"\ncommand is running. Attempting to preventatively handle users\nmodifying the working tree simultaneously with concurrent git commands\nlike `git merge` seems like a losing proposition to me; it'd be a huge\ncan of worms and have a million holes. I don't think that's worth it.\n\n> > }\n> >\n> > /* This is called when no merge was necessary. */\n>\n> > diff --git a/t/t7607-merge-state.sh b/t/t7607-merge-state.sh\n> > new file mode 100755\n>\n> As long we are adding a brand-new script for new tests, probably we\n> should add tests for other steps (like [4/6]) here, perhaps?\n\nYes.\n\n>\n> > index 00000000000..655478cd0b3\n> > --- /dev/null\n> > +++ b/t/t7607-merge-state.sh\n> > @@ -0,0 +1,25 @@\n> > +#!/bin/sh\n> > +\n> > +test_description=\"Test that merge state is as expected after failed merge\"\n> > +\n> > +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> > +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n> > +. ./test-lib.sh\n> > +\n> > +test_expect_success 'set up custom strategy' '\n> > + test_commit --no-tag \"Initial\" base base &&\n> > +git show-ref &&\n>\n> Is this part of the test, or a leftover debugging aid?\n\nLooks like part of a leftover debugging aid; sorry about that. Will clean up.\n\n> > +\n> > + for b in branch1 branch2 branch3\n> > + do\n> > + git checkout -b $b main &&\n> > + test_commit --no-tag \"Change on $b\" base $b\n> > + done &&\n> > +\n> > + git checkout branch1 &&\n> > + test_must_fail git merge branch2 branch3 &&\n> > + git diff --exit-code --name-status &&\n> > + test_path_is_missing .git/MERGE_HEAD\n> > +'\n>\n> Hmph, I am not sure if the new behaviour is not too pessimistic.\n> When octopus fails after successfully merging branch2 and then\n> failing the merge of branch3 (i.e. the last one) due to conflict,\n\nThat's not what's happening here. It is not failing due to conflict,\noctopus is reporting that it cannot even leave things in a conflicted\nstate for the user, and is actually incapable of handling this\nparticular type of merge. Part of the output seen when attempting\nthis merge includes:\n fatal: merge program failed\n Should not be doing an octopus.\n\nSee previous discussion at\nhttps://lore.kernel.org/git/xmqq35hdd205.fsf@gitster.g/. To make this\nclearer, perhaps I should use \"test_expect_code 2\" instead of\ntest_must_fail, and also grep the output/error for the above messages.\n\n> I think octpus users are used to be able to resolve it manually\n> and make a commit. Are we making it impossible by doing the\n> reset-restore dance here?\n\nNo, we are not changing what octopus can handle here. The code above\nis not triggered when octopus returns that it ran into conflicts for\nthe user to resolve. It is only triggered when octopus says it cannot\nhandle the merge in question. See your commit 98efc8f3d8 (\"octopus:\nallow manual resolve on the last round.\", 2006-01-13), and note that\nthe the \"exit 2\" code path is the one that we are hitting. I'll add\nsome comments to the testcase and rewrite the commit message to try to\nmake this clearer.\n"},{"id":"459588","messageId":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v2.git.1655621424.gitgitgadget@gmail.com","subject":"[PATCH v3 0/7] Fix merge restore state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T08:16:24Z","receivedAt":"2022-07-21T08:16:47Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"NOTE: Rebased on master, yet again, because (1) Junio merged his commit to\nmaster separately, and (2) Ævar's intentional duplication of my second\npatch[1] later conflicted with other changes I had to make in the same area.\n(Which isn't a big deal, but for future reference, it would be nicer to\navoid conflicts by omitting the fixup I had already submitted[2] instead of\nintentionally duplicating it).\n\nThis started as a simple series to fix restore_state() in builtin/merge.c,\nfixing an issue reported by ZheNing Hu[3]. It's grown so much it's hard to\ncall it simple. Anyway...\n\nChanges since v2:\n\n * Removed the first two patches, as noted above in the comment about\n rebasing.\n * Inserted new patches 3, 4, and 5 to fix some related bugs. Folks are more\n likely to object to patch 5 than the others; people should probably take\n a look at that one if they have limited time.\n * Dramatically reworded commit messages given the misunderstandings of what\n was being addressed and done. Hopefully it is much clearer what the last\n three patches are doing and what they are not doing, and why.\n * Added several new testcases\n\n[1]\nhttps://lore.kernel.org/git/patch-1.1-7d90f26b73f-20220520T115426Z-avarab@gmail.com/\n[2] https://lore.kernel.org/git/xmqqedyyghsc.fsf@gitster.g/ [3]\nhttps://lore.kernel.org/git/CAOLTT8R7QmpvaFPTRs3xTpxr7eiuxF-ZWtvUUSC0-JOo9Y+SqA@mail.gmail.com/\n\nElijah Newren (7):\n merge-ort-wrappers: make printed message match the one from recursive\n merge-resolve: abort if index does not match HEAD\n merge: do not abort early if one strategy fails to handle the merge\n merge: fix save_state() to work when there are stat-dirty files\n merge: make restore_state() restore staged state too\n merge: ensure we can actually restore pre-merge state\n merge: do not exit restore_state() prematurely\n\n builtin/merge.c | 59 ++++++++++++++++++------\n git-merge-resolve.sh | 10 ++++\n merge-ort-wrappers.c | 7 ++-\n t/t6402-merge-rename.sh | 2 +-\n t/t6424-merge-unrelated-index-changes.sh | 58 +++++++++++++++++++++++\n t/t6439-merge-co-error-msgs.sh | 1 +\n t/t7607-merge-state.sh | 32 +++++++++++++\n 7 files changed, 154 insertions(+), 15 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\n\nbase-commit: e72d93e88cb20b06e88e6e7d81bd1dc4effe453f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1231%2Fnewren%2Ffix-merge-restore-state-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1231/newren/fix-merge-restore-state-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/1231\n\nRange-diff vs v2:\n\n 1: 6147e72c309 < -: ----------- t6424: make sure a failed merge preserves local changes\n 2: 230d84f09c8 < -: ----------- merge: remove unused variable\n -: ----------- > 1: e39b2e15ece merge-ort-wrappers: make printed message match the one from recursive\n -: ----------- > 2: 2810dec7608 merge-resolve: abort if index does not match HEAD\n -: ----------- > 3: b41853e3f99 merge: do not abort early if one strategy fails to handle the merge\n 3: 89e5e633241 ! 4: 64700338a28 merge: fix save_state() to work when there are racy-dirty files\n @@ Metadata\n Author: Elijah Newren \n \n ## Commit message ##\n - merge: fix save_state() to work when there are racy-dirty files\n + merge: fix save_state() to work when there are stat-dirty files\n \n - When there are racy-dirty files, but no files are modified,\n + When there are stat-dirty files, but no files are modified,\n `git stash create` exits with unsuccessful status. This causes merge\n - to fail. Refresh the index first to avoid this problem.\n + to fail. Copy some code from sequencer.c's create_autostash to refresh\n + the index first to avoid this problem.\n \n Signed-off-by: Elijah Newren \n \n @@ builtin/merge.c: static int save_state(struct object_id *stash)\n \tstrvec_pushl(&cp.args, \"stash\", \"create\", NULL);\n \tcp.out = -1;\n \tcp.git_cmd = 1;\n +\n + ## t/t6424-merge-unrelated-index-changes.sh ##\n +@@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'subtree' '\n + \ttest_path_is_missing .git/MERGE_HEAD\n + '\n + \n ++test_expect_success 'avoid failure due to stat-dirty files' '\n ++\tgit reset --hard &&\n ++\tgit checkout B^0 &&\n ++\n ++\t# Make \"a\" be stat-dirty\n ++\ttest-tool chmtime =+1 a &&\n ++\n ++\t# stat-dirty file should not prevent stash creation in builtin/merge.c\n ++\tgit merge -s resolve -s recursive D^0\n ++'\n ++\n + test_expect_success 'resolve && recursive && ort' '\n + \tgit reset --hard &&\n + \tgit checkout B^0 &&\n 4: 4a8b7c9e06d ! 5: 91c495c770e merge: make restore_state() restore staged state too\n @@ Metadata\n ## Commit message ##\n merge: make restore_state() restore staged state too\n \n - merge can be invoked with uncommitted changes, including staged changes.\n - merge is responsible for restoring this state if some of the merge\n - strategies make changes. However, it was not restoring staged changes\n - due to the lack of the \"--index\" option to \"git stash apply\". Add the\n - option to fix this shortcoming.\n + There are multiple issues at play here:\n +\n + 1) If `git merge` is invoked with staged changes, it should abort\n + without doing any merging, and the user's working tree and index\n + should be the same as before merge was invoked.\n + 2) Merge strategies are responsible for enforcing the index == HEAD\n + requirement. (See 9822175d2b (\"Ensure index matches head before\n + invoking merge machinery, round N\", 2019-08-17) for some history\n + around this.)\n + 3) Merge strategies can bail saying they are not an appropriate\n + handler for the merge in question (possibly allowing other\n + strategies to be used instead).\n + 4) Merge strategies can make changes to the index and working tree,\n + and have no expectation to clean up after themselves, *even* if\n + they bail out and say they are not an appropriate handler for\n + the merge in question. (The `octopus` merge strategy does this,\n + for example.)\n + 5) Because of (3) and (4), builtin/merge.c stashes state before\n + trying merge strategies and restores it afterward.\n +\n + Unfortunately, if users had staged changes before calling `git merge`,\n + builtin/merge.c could do the following:\n +\n +    * stash the changes, in order to clean up after the strategies\n +    * try all the merge strategies in turn, each of which report they\n + cannot function due to the index not matching HEAD\n +    * restore the changes via \"git stash apply\"\n +\n + But that last step would have the net effect of unstaging the user's\n + changes. Fix this by adding the \"--index\" option to \"git stash apply\".\n + While at it, also squelch the stash apply output; we already report\n + \"Rewinding the tree to pristine...\" and don't need a detailed `git\n + status` report afterwards.\n \n Signed-off-by: Elijah Newren \n \n @@ builtin/merge.c: static void reset_hard(const struct object_id *oid, int verbose\n \t\t\t const struct object_id *stash)\n {\n -\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n -+\tconst char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n ++\tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n ++\t\t\t NULL, NULL };\n \n \tif (is_null_oid(stash))\n \t\treturn;\n @@ builtin/merge.c: static void reset_hard(const struct object_id *oid, int verbose\n \treset_hard(head, 1);\n \n -\targs[2] = oid_to_hex(stash);\n -+\targs[3] = oid_to_hex(stash);\n ++\targs[4] = oid_to_hex(stash);\n \n \t/*\n \t * It is OK to ignore error here, for example when there was\n +\n + ## t/t6424-merge-unrelated-index-changes.sh ##\n +@@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'resolve && recursive && ort' '\n + \n + \ttest_seq 0 10 >a &&\n + \tgit add a &&\n ++\tgit rev-parse :a >expect &&\n + \n + \tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n + \ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n +@@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'resolve && recursive && ort' '\n + \tgrep \"Trying merge strategy resolve...\" output &&\n + \tgrep \"Trying merge strategy recursive...\" output &&\n + \tgrep \"Trying merge strategy ort...\" output &&\n +-\tgrep \"No merge strategy handled the merge.\" output\n ++\tgrep \"No merge strategy handled the merge.\" output &&\n ++\n ++\t# Changes to \"a\" should remain staged\n ++\tgit rev-parse :a >actual &&\n ++\ttest_cmp expect actual\n + '\n + \n + test_done\n 5: a03075167c1 ! 6: 887967c1f3f merge: ensure we can actually restore pre-merge state\n @@ Metadata\n ## Commit message ##\n merge: ensure we can actually restore pre-merge state\n \n - Merge strategies can fail -- not just have conflicts, but give up and\n - say that they are unable to handle the current type of merge. However,\n - they can also make changes to the index and working tree before giving\n - up; merge-octopus does this, for example. Currently, we do not expect\n - the individual strategies to clean up after themselves, but instead\n - expect builtin/merge.c to do so. For it to be able to, it needs to save\n - the state before trying the merge strategy so it can have something to\n - restore to. Therefore, remove the shortcut bypassing the save_state()\n - call.\n + Merge strategies can:\n + * succeed with a clean merge\n + * succeed with a conflicted merge\n + * fail to handle the given type of merge\n +\n + If one is thinking in terms of automatic mergeability, they would use\n + the word \"fail\" instead of \"succeed\" for the second bullet, but I am\n + focusing here on ability of the merge strategy to handle the given\n + inputs, not on whether the given inputs are mergeable. The third\n + category is about the merge strategy failing to know how to handle the\n + given data; examples include:\n +\n + * Passing more than 2 branches to 'recursive' or 'ort'\n + * Passing 2 or fewer branches to 'octopus'\n + * Trying to do more complicated merges with 'resolve' (I believe\n + directory/file conflicts will cause it to bail.)\n + * Octopus running into a merge conflict for any branch OTHER than\n + the final one (see the \"exit 2\" codepath of commit 98efc8f3d8\n + (\"octopus: allow manual resolve on the last round.\", 2006-01-13))\n +\n + That final one is particularly interesting, because it shows that the\n + merge strategy can muck with the index and working tree, and THEN bail\n + and say \"sorry, this strategy cannot handle this type of merge; use\n + something else\".\n +\n + Further, we do not currently expect the individual strategies to clean\n + up after themselves, but instead expect builtin/merge.c to do so. For\n + it to be able to, it needs to save the state before trying the merge\n + strategy so it can have something to restore to. Therefore, remove the\n + shortcut bypassing the save_state() call.\n +\n + There is another bug on the restore_state() side of things, so no\n + testcase will be added until the next commit when we have addressed that\n + issue as well.\n \n Signed-off-by: Elijah Newren \n \n @@ builtin/merge.c: int cmd_merge(int argc, const char **argv, const char *prefix)\n \t * sync with the head commit. The strategies are responsible\n \t * to ensure this.\n +\t *\n -+\t * Stash away the local changes so that we can try more than one.\n ++\t * Stash away the local changes so that we can try more than one\n ++\t * and/or recover from merge strategies bailing while leaving the\n ++\t * index and working tree polluted.\n \t */\n -\tif (use_strategies_nr == 1 ||\n -\t /*\n 6: 0783b48c121 ! 7: 81c40492a62 merge: do not exit restore_state() prematurely\n @@ Commit message\n appropriate function to do the work which would update the in-memory\n index automatically. For now, just do the simple fix.)\n \n + Also, add a testcase checking this, one for which the octopus strategy\n + fails on the first commit it attempts to merge, and thus which it\n + cannot handle at all and must completely bail on (as per the \"exit 2\"\n + code path of commit 98efc8f3d8 (\"octopus: allow manual resolve on the\n + last round.\", 2006-01-13)).\n +\n Reported-by: ZheNing Hu \n Signed-off-by: Elijah Newren \n \n ## builtin/merge.c ##\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n - {\n - \tconst char *args[] = { \"stash\", \"apply\", \"--index\", NULL, NULL };\n + \tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n + \t\t\t NULL, NULL };\n \n -\tif (is_null_oid(stash))\n -\t\treturn;\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n +\tif (is_null_oid(stash))\n +\t\tgoto refresh_cache;\n +\n - \targs[3] = oid_to_hex(stash);\n + \targs[4] = oid_to_hex(stash);\n \n \t/*\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n @@ t/t7607-merge-state.sh (new)\n +\n +test_expect_success 'set up custom strategy' '\n +\ttest_commit --no-tag \"Initial\" base base &&\n -+git show-ref &&\n +\n +\tfor b in branch1 branch2 branch3\n +\tdo\n +\t\tgit checkout -b $b main &&\n -+\t\ttest_commit --no-tag \"Change on $b\" base $b\n ++\t\ttest_commit --no-tag \"Change on $b\" base $b || return 1\n +\tdone &&\n +\n +\tgit checkout branch1 &&\n -+\ttest_must_fail git merge branch2 branch3 &&\n ++\t# This is a merge that octopus cannot handle. Note, that it does not\n ++\t# just hit conflicts, it completely fails and says that it cannot\n ++\t# handle this type of merge.\n ++\ttest_expect_code 2 git merge branch2 branch3 >output 2>&1 &&\n ++\tgrep \"fatal: merge program failed\" output &&\n ++\tgrep \"Should not be doing an octopus\" output &&\n ++\n ++\t# Make sure we did not leave stray changes around when no appropriate\n ++\t# merge strategy was found\n +\tgit diff --exit-code --name-status &&\n +\ttest_path_is_missing .git/MERGE_HEAD\n +'\n\n-- \ngitgitgadget\n"},{"id":"459587","messageId":"e39b2e15ece14ba2b1118ae95e0d90ed60589b41.1658391391.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","subject":"[PATCH v3 1/7] merge-ort-wrappers: make printed message match the one from recursive","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T08:16:25Z","receivedAt":"2022-07-21T08:16:48Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nWhen the index does not match HEAD, the merge strategies are responsible\nto detect that condition and abort. The merge-ort-wrappers had code to\nimplement this and meant to copy the error message from merge-recursive\nbut deviated in two ways, both due to the message in merge-recursive\nbeing processed by another function that made additional changes:\n * It added an implicit \"error: \" prefix\n * It added an implicit trailing newline\n\nAdd these things, but do so in a couple extra steps to avoid having\ntranslators need to translate another not-quite-identical string.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort-wrappers.c | 7 ++++++-\n 1 file changed, 6 insertions(+), 1 deletion(-)\n\ndiff --git a/merge-ort-wrappers.c b/merge-ort-wrappers.c\nindex ad041061695..d2c416bb5c0 100644\n--- a/merge-ort-wrappers.c\n+++ b/merge-ort-wrappers.c\n@@ -10,8 +10,13 @@ static int unclean(struct merge_options *opt, struct tree *head)\n \tstruct strbuf sb = STRBUF_INIT;\n \n \tif (head && repo_index_has_changes(opt->repo, head, &sb)) {\n-\t\tfprintf(stderr, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n+\t\tstruct strbuf err = STRBUF_INIT;\n+\t\tstrbuf_addstr(&err, \"error: \");\n+\t\tstrbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n \t\t sb.buf);\n+\t\tstrbuf_addch(&err, '\\n');\n+\t\tfputs(err.buf, stderr);\n+\t\tstrbuf_release(&err);\n \t\tstrbuf_release(&sb);\n \t\treturn -1;\n \t}\n-- \ngitgitgadget\n\n"},{"id":"459589","messageId":"2810dec7608d68a5a7906e70f88278c5ab9aa0e2.1658391391.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","subject":"[PATCH v3 2/7] merge-resolve: abort if index does not match HEAD","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T08:16:26Z","receivedAt":"2022-07-21T08:16:51Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nAs noted in commit 9822175d2b (\"Ensure index matches head before\ninvoking merge machinery, round N\", 2019-08-17), we have had a very\nlong history of problems with failing to enforce the requirement that\nindex matches HEAD when starting a merge. One of the commits\nreferenced in the long tale of issues arising from lax enforcement of\nthis requirement was commit 55f39cf755 (\"merge: fix misleading\npre-merge check documentation\", 2018-06-30), which tried to document\nthe requirement and noted there were some exceptions. As mentioned in\nthat commit message, the `resolve` strategy was the one strategy that\ndid not have an explicit index matching HEAD check, and the reason it\ndidn't was that I wasn't able to discover any cases where the\nimplementation would fail to catch the problem and abort, and didn't\nwant to introduce unnecessary performance overhead of adding another\ncheck.\n\nWell, today I discovered a testcase where the implementation does not\ncatch the problem and so an explicit check is needed. Add a testcase\nthat previously would have failed, and update git-merge-resolve.sh to\nhave an explicit check. Note that the code is copied from 3ec62ad9ff\n(\"merge-octopus: abort if index does not match HEAD\", 2016-04-09), so\nthat we reuse the same message and avoid making translators need to\ntranslate some new message.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 20 ++++++++++++++++++\n git-merge-resolve.sh | 10 +++++++++\n t/t6424-merge-unrelated-index-changes.sh | 26 ++++++++++++++++++++++++\n 3 files changed, 56 insertions(+)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 23170f2d2a6..13884b8e836 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -1599,6 +1599,26 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t\t */\n \t\trefresh_cache(REFRESH_QUIET);\n \t\tif (allow_trivial && fast_forward != FF_ONLY) {\n+\t\t\t/*\n+\t\t\t * Must first ensure that index matches HEAD before\n+\t\t\t * attempting a trivial merge.\n+\t\t\t */\n+\t\t\tstruct tree *head_tree = get_commit_tree(head_commit);\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tif (repo_index_has_changes(the_repository, head_tree,\n+\t\t\t\t\t\t &sb)) {\n+\t\t\t\tstruct strbuf err = STRBUF_INIT;\n+\t\t\t\tstrbuf_addstr(&err, \"error: \");\n+\t\t\t\tstrbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n+\t\t\t\t\t sb.buf);\n+\t\t\t\tstrbuf_addch(&err, '\\n');\n+\t\t\t\tfputs(err.buf, stderr);\n+\t\t\t\tstrbuf_release(&err);\n+\t\t\t\tstrbuf_release(&sb);\n+\t\t\t\treturn -1;\n+\t\t\t}\n+\n \t\t\t/* See if it is really trivial. */\n \t\t\tgit_committer_info(IDENT_STRICT);\n \t\t\tprintf(_(\"Trying really trivial in-index merge...\\n\"));\ndiff --git a/git-merge-resolve.sh b/git-merge-resolve.sh\nindex 343fe7bccd0..77e93121bf8 100755\n--- a/git-merge-resolve.sh\n+++ b/git-merge-resolve.sh\n@@ -5,6 +5,16 @@\n #\n # Resolve two trees, using enhanced multi-base read-tree.\n \n+. git-sh-setup\n+\n+# Abort if index does not match HEAD\n+if ! git diff-index --quiet --cached HEAD --\n+then\n+ gettextln \"Error: Your local changes to the following files would be overwritten by merge\"\n+ git diff-index --cached --name-only HEAD -- | sed -e 's/^/ /'\n+ exit 2\n+fi\n+\n # The first parameters up to -- are merge bases; the rest are heads.\n bases= head= remotes= sep_seen=\n for arg\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex b6e424a427b..f35d3182b86 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -114,6 +114,32 @@ test_expect_success 'resolve, non-trivial' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'resolve, trivial, related file removed' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\tgit rm a &&\n+\ttest_path_is_missing a &&\n+\n+\ttest_must_fail git merge -s resolve C^0 &&\n+\n+\ttest_path_is_missing a &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n+test_expect_success 'resolve, non-trivial, related file removed' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\tgit rm a &&\n+\ttest_path_is_missing a &&\n+\n+\ttest_must_fail git merge -s resolve D^0 &&\n+\n+\ttest_path_is_missing a &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n test_expect_success 'recursive' '\n \tgit reset --hard &&\n \tgit checkout B^0 &&\n-- \ngitgitgadget\n\n"},{"id":"459590","messageId":"b41853e3f9908ab458bcb28684d817677e32367b.1658391391.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","subject":"[PATCH v3 3/7] merge: do not abort early if one strategy fails to handle the merge","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T08:16:27Z","receivedAt":"2022-07-21T08:16:51Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nbuiltin/merge is setup to allow multiple strategies to be specified,\nand it will find the \"best\" result and use it. This is defeated if\nsome of the merge strategies abort early when they cannot handle the\nmerge. Fix the logic that calls recursive and ort to not do such an\nearly abort, but instead return \"2\" or \"unhandled\" so that the next\nstrategy can try to handle the merge.\n\nComing up with a testcase for this is somewhat difficult, since\nrecursive and ort both handle nearly any two-headed merge (there is\na separate code path that checks for non-two-headed merges and\nalready returns \"2\" for them). So use a somewhat synthetic testcase\nof having the index not match HEAD before the merge starts, since all\nmerge strategies will abort for that.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 6 ++++--\n t/t6402-merge-rename.sh | 2 +-\n t/t6424-merge-unrelated-index-changes.sh | 16 ++++++++++++++++\n t/t6439-merge-co-error-msgs.sh | 1 +\n 4 files changed, 22 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 13884b8e836..dec7375bf2a 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -754,8 +754,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n \t\telse\n \t\t\tclean = merge_recursive(&o, head, remoteheads->item,\n \t\t\t\t\t\treversed, &result);\n-\t\tif (clean < 0)\n-\t\t\texit(128);\n+\t\tif (clean < 0) {\n+\t\t\trollback_lock_file(&lock);\n+\t\t\treturn 2;\n+\t\t}\n \t\tif (write_locked_index(&the_index, &lock,\n \t\t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n \t\t\tdie(_(\"unable to write %s\"), get_index_file());\ndiff --git a/t/t6402-merge-rename.sh b/t/t6402-merge-rename.sh\nindex 3a32b1a45cf..772238e582c 100755\n--- a/t/t6402-merge-rename.sh\n+++ b/t/t6402-merge-rename.sh\n@@ -210,7 +210,7 @@ test_expect_success 'updated working tree file should prevent the merge' '\n \techo >>M one line addition &&\n \tcat M >M.saved &&\n \tgit update-index M &&\n-\ttest_expect_code 128 git pull --no-rebase . yellow &&\n+\ttest_expect_code 2 git pull --no-rebase . yellow &&\n \ttest_cmp M M.saved &&\n \trm -f M.saved\n '\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex f35d3182b86..8b749e19083 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -268,4 +268,20 @@ test_expect_success 'subtree' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'resolve && recursive && ort' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\ttest_seq 0 10 >a &&\n+\tgit add a &&\n+\n+\tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n+\ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n+\n+\tgrep \"Trying merge strategy resolve...\" output &&\n+\tgrep \"Trying merge strategy recursive...\" output &&\n+\tgrep \"Trying merge strategy ort...\" output &&\n+\tgrep \"No merge strategy handled the merge.\" output\n+'\n+\n test_done\ndiff --git a/t/t6439-merge-co-error-msgs.sh b/t/t6439-merge-co-error-msgs.sh\nindex 5bfb027099a..52cf0c87690 100755\n--- a/t/t6439-merge-co-error-msgs.sh\n+++ b/t/t6439-merge-co-error-msgs.sh\n@@ -47,6 +47,7 @@ test_expect_success 'untracked files overwritten by merge (fast and non-fast for\n \t\texport GIT_MERGE_VERBOSITY &&\n \t\ttest_must_fail git merge branch 2>out2\n \t) &&\n+\techo \"Merge with strategy ${GIT_TEST_MERGE_ALGORITHM:-ort} failed.\" >>expect &&\n \ttest_cmp out2 expect &&\n \tgit reset --hard HEAD^\n '\n-- \ngitgitgadget\n\n"},{"id":"459591","messageId":"64700338a28b04be93d8d4bc8b2c75cab75ab5cd.1658391391.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","subject":"[PATCH v3 4/7] merge: fix save_state() to work when there are stat-dirty files","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T08:16:28Z","receivedAt":"2022-07-21T08:16:54Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nWhen there are stat-dirty files, but no files are modified,\n`git stash create` exits with unsuccessful status. This causes merge\nto fail. Copy some code from sequencer.c's create_autostash to refresh\nthe index first to avoid this problem.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 8 ++++++++\n t/t6424-merge-unrelated-index-changes.sh | 11 +++++++++++\n 2 files changed, 19 insertions(+)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex dec7375bf2a..4170c30317e 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -313,8 +313,16 @@ static int save_state(struct object_id *stash)\n \tint len;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tstruct strbuf buffer = STRBUF_INIT;\n+\tstruct lock_file lock_file = LOCK_INIT;\n+\tint fd;\n \tint rc = -1;\n \n+\tfd = repo_hold_locked_index(the_repository, &lock_file, 0);\n+\trefresh_cache(REFRESH_QUIET);\n+\tif (0 <= fd)\n+\t\trepo_update_index_if_able(the_repository, &lock_file);\n+\trollback_lock_file(&lock_file);\n+\n \tstrvec_pushl(&cp.args, \"stash\", \"create\", NULL);\n \tcp.out = -1;\n \tcp.git_cmd = 1;\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex 8b749e19083..3019d030e07 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -268,6 +268,17 @@ test_expect_success 'subtree' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'avoid failure due to stat-dirty files' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\t# Make \"a\" be stat-dirty\n+\ttest-tool chmtime =+1 a &&\n+\n+\t# stat-dirty file should not prevent stash creation in builtin/merge.c\n+\tgit merge -s resolve -s recursive D^0\n+'\n+\n test_expect_success 'resolve && recursive && ort' '\n \tgit reset --hard &&\n \tgit checkout B^0 &&\n-- \ngitgitgadget\n\n"},{"id":"459592","messageId":"887967c1f3fd6f03cf1d0bb3c19ed16819541092.1658391391.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","subject":"[PATCH v3 6/7] merge: ensure we can actually restore pre-merge state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T08:16:30Z","receivedAt":"2022-07-21T08:17:03Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nMerge strategies can:\n * succeed with a clean merge\n * succeed with a conflicted merge\n * fail to handle the given type of merge\n\nIf one is thinking in terms of automatic mergeability, they would use\nthe word \"fail\" instead of \"succeed\" for the second bullet, but I am\nfocusing here on ability of the merge strategy to handle the given\ninputs, not on whether the given inputs are mergeable. The third\ncategory is about the merge strategy failing to know how to handle the\ngiven data; examples include:\n\n * Passing more than 2 branches to 'recursive' or 'ort'\n * Passing 2 or fewer branches to 'octopus'\n * Trying to do more complicated merges with 'resolve' (I believe\n directory/file conflicts will cause it to bail.)\n * Octopus running into a merge conflict for any branch OTHER than\n the final one (see the \"exit 2\" codepath of commit 98efc8f3d8\n (\"octopus: allow manual resolve on the last round.\", 2006-01-13))\n\nThat final one is particularly interesting, because it shows that the\nmerge strategy can muck with the index and working tree, and THEN bail\nand say \"sorry, this strategy cannot handle this type of merge; use\nsomething else\".\n\nFurther, we do not currently expect the individual strategies to clean\nup after themselves, but instead expect builtin/merge.c to do so. For\nit to be able to, it needs to save the state before trying the merge\nstrategy so it can have something to restore to. Therefore, remove the\nshortcut bypassing the save_state() call.\n\nThere is another bug on the restore_state() side of things, so no\ntestcase will be added until the next commit when we have addressed that\nissue as well.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex f807bf335bd..11bb4bab0a1 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -1686,12 +1686,12 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t * tree in the index -- this means that the index must be in\n \t * sync with the head commit. The strategies are responsible\n \t * to ensure this.\n+\t *\n+\t * Stash away the local changes so that we can try more than one\n+\t * and/or recover from merge strategies bailing while leaving the\n+\t * index and working tree polluted.\n \t */\n-\tif (use_strategies_nr == 1 ||\n-\t /*\n-\t * Stash away the local changes so that we can try more than one.\n-\t */\n-\t save_state(&stash))\n+\tif (save_state(&stash))\n \t\toidclr(&stash);\n \n \tfor (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n-- \ngitgitgadget\n\n"},{"id":"459593","messageId":"91c495c770e3f7f91ae655a503bdd1cd99935e40.1658391391.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","subject":"[PATCH v3 5/7] merge: make restore_state() restore staged state too","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T08:16:29Z","receivedAt":"2022-07-21T08:17:04Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nThere are multiple issues at play here:\n\n 1) If `git merge` is invoked with staged changes, it should abort\n without doing any merging, and the user's working tree and index\n should be the same as before merge was invoked.\n 2) Merge strategies are responsible for enforcing the index == HEAD\n requirement. (See 9822175d2b (\"Ensure index matches head before\n invoking merge machinery, round N\", 2019-08-17) for some history\n around this.)\n 3) Merge strategies can bail saying they are not an appropriate\n handler for the merge in question (possibly allowing other\n strategies to be used instead).\n 4) Merge strategies can make changes to the index and working tree,\n and have no expectation to clean up after themselves, *even* if\n they bail out and say they are not an appropriate handler for\n the merge in question. (The `octopus` merge strategy does this,\n for example.)\n 5) Because of (3) and (4), builtin/merge.c stashes state before\n trying merge strategies and restores it afterward.\n\nUnfortunately, if users had staged changes before calling `git merge`,\nbuiltin/merge.c could do the following:\n\n   * stash the changes, in order to clean up after the strategies\n   * try all the merge strategies in turn, each of which report they\n cannot function due to the index not matching HEAD\n   * restore the changes via \"git stash apply\"\n\nBut that last step would have the net effect of unstaging the user's\nchanges. Fix this by adding the \"--index\" option to \"git stash apply\".\nWhile at it, also squelch the stash apply output; we already report\n\"Rewinding the tree to pristine...\" and don't need a detailed `git\nstatus` report afterwards.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 5 +++--\n t/t6424-merge-unrelated-index-changes.sh | 7 ++++++-\n 2 files changed, 9 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 4170c30317e..f807bf335bd 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -383,14 +383,15 @@ static void reset_hard(const struct object_id *oid, int verbose)\n static void restore_state(const struct object_id *head,\n \t\t\t const struct object_id *stash)\n {\n-\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n+\tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n+\t\t\t NULL, NULL };\n \n \tif (is_null_oid(stash))\n \t\treturn;\n \n \treset_hard(head, 1);\n \n-\targs[2] = oid_to_hex(stash);\n+\targs[4] = oid_to_hex(stash);\n \n \t/*\n \t * It is OK to ignore error here, for example when there was\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex 3019d030e07..c96649448fa 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -285,6 +285,7 @@ test_expect_success 'resolve && recursive && ort' '\n \n \ttest_seq 0 10 >a &&\n \tgit add a &&\n+\tgit rev-parse :a >expect &&\n \n \tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n \ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n@@ -292,7 +293,11 @@ test_expect_success 'resolve && recursive && ort' '\n \tgrep \"Trying merge strategy resolve...\" output &&\n \tgrep \"Trying merge strategy recursive...\" output &&\n \tgrep \"Trying merge strategy ort...\" output &&\n-\tgrep \"No merge strategy handled the merge.\" output\n+\tgrep \"No merge strategy handled the merge.\" output &&\n+\n+\t# Changes to \"a\" should remain staged\n+\tgit rev-parse :a >actual &&\n+\ttest_cmp expect actual\n '\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"459594","messageId":"81c40492a62e81100c66a8ccc1ec200fb2e6fca5.1658391391.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","subject":"[PATCH v3 7/7] merge: do not exit restore_state() prematurely","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T08:16:31Z","receivedAt":"2022-07-21T08:17:07Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nPreviously, if the user:\n\n* Had no local changes before starting the merge\n* A merge strategy makes changes to the working tree/index but returns\n with exit status 2\n\nThen we'd call restore_state() to clean up the changes and either let\nthe next merge strategy run (if there is one), or exit telling the user\nthat no merge strategy could handle the merge. Unfortunately,\nrestore_state() did not clean up the changes as expected; that function\nwas a no-op if the stash was a null, and the stash would be null if\nthere were no local changes before starting the merge. So, instead of\n\"Rewinding the tree to pristine...\" as the code claimed, restore_state()\nwould leave garbage around in the index and working tree (possibly\nincluding conflicts) for either the next merge strategy or for the user\nafter aborting the merge. And in the case of aborting the merge, the\nuser would be unable to run \"git merge --abort\" to get rid of the\nunintended leftover conflicts, because the merge control files were not\nwritten as it was presumed that we had restored to a clean state\nalready.\n\nFix the main problem by making sure that restore_state() only skips the\nstash application if the stash is null rather than skipping the whole\nfunction.\n\nHowever, there is a secondary problem -- since merge.c forks\nsubprocesses to do the cleanup, the in-memory index is left out-of-sync.\nWhile there was a refresh_cache(REFRESH_QUIET) call that attempted to\ncorrect that, that function would not handle cases where the previous\nmerge strategy added conflicted entries. We need to drop the index and\nre-read it to handle such cases.\n\n(Alternatively, we could stop forking subprocesses and instead call some\nappropriate function to do the work which would update the in-memory\nindex automatically. For now, just do the simple fix.)\n\nAlso, add a testcase checking this, one for which the octopus strategy\nfails on the first commit it attempts to merge, and thus which it\ncannot handle at all and must completely bail on (as per the \"exit 2\"\ncode path of commit 98efc8f3d8 (\"octopus: allow manual resolve on the\nlast round.\", 2006-01-13)).\n\nReported-by: ZheNing Hu \nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 10 ++++++----\n t/t7607-merge-state.sh | 32 ++++++++++++++++++++++++++++++++\n 2 files changed, 38 insertions(+), 4 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 11bb4bab0a1..7fb4414ebb7 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -386,11 +386,11 @@ static void restore_state(const struct object_id *head,\n \tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n \t\t\t NULL, NULL };\n \n-\tif (is_null_oid(stash))\n-\t\treturn;\n-\n \treset_hard(head, 1);\n \n+\tif (is_null_oid(stash))\n+\t\tgoto refresh_cache;\n+\n \targs[4] = oid_to_hex(stash);\n \n \t/*\n@@ -399,7 +399,9 @@ static void restore_state(const struct object_id *head,\n \t */\n \trun_command_v_opt(args, RUN_GIT_CMD);\n \n-\trefresh_cache(REFRESH_QUIET);\n+refresh_cache:\n+\tif (discard_cache() < 0 || read_cache() < 0)\n+\t\tdie(_(\"could not read index\"));\n }\n \n /* This is called when no merge was necessary. */\ndiff --git a/t/t7607-merge-state.sh b/t/t7607-merge-state.sh\nnew file mode 100755\nindex 00000000000..fc33d57357b\n--- /dev/null\n+++ b/t/t7607-merge-state.sh\n@@ -0,0 +1,32 @@\n+#!/bin/sh\n+\n+test_description=\"Test that merge state is as expected after failed merge\"\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+. ./test-lib.sh\n+\n+test_expect_success 'set up custom strategy' '\n+\ttest_commit --no-tag \"Initial\" base base &&\n+\n+\tfor b in branch1 branch2 branch3\n+\tdo\n+\t\tgit checkout -b $b main &&\n+\t\ttest_commit --no-tag \"Change on $b\" base $b || return 1\n+\tdone &&\n+\n+\tgit checkout branch1 &&\n+\t# This is a merge that octopus cannot handle. Note, that it does not\n+\t# just hit conflicts, it completely fails and says that it cannot\n+\t# handle this type of merge.\n+\ttest_expect_code 2 git merge branch2 branch3 >output 2>&1 &&\n+\tgrep \"fatal: merge program failed\" output &&\n+\tgrep \"Should not be doing an octopus\" output &&\n+\n+\t# Make sure we did not leave stray changes around when no appropriate\n+\t# merge strategy was found\n+\tgit diff --exit-code --name-status &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n+test_done\n-- \ngitgitgadget\n"},{"id":"459637","messageId":"xmqqsfmulb6w.fsf@gitster.g","threadId":"57898","inReplyTo":"e39b2e15ece14ba2b1118ae95e0d90ed60589b41.1658391391.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 1/7] merge-ort-wrappers: make printed message match the one from recursive","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-21T15:47:35Z","receivedAt":"2022-07-21T15:48:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> \tif (head && repo_index_has_changes(opt->repo, head, &sb)) {\n> -\t\tfprintf(stderr, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> +\t\tstruct strbuf err = STRBUF_INIT;\n> +\t\tstrbuf_addstr(&err, \"error: \");\n> +\t\tstrbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> \t\t sb.buf);\n> +\t\tstrbuf_addch(&err, '\\n');\n> +\t\tfputs(err.buf, stderr);\n> +\t\tstrbuf_release(&err);\n\nMakes me wonder why this is not a mere\n\n\terror(_(\"Your local chagnes ... by merge:\\n %s\"), sb.buf);\n\nthat reuses the exact string. The err() function in merge-recursive.c \nis strangely complex (and probably buggy---if it is not buffering\noutput, it adds \"error: \" prefix to opt->obuf before calling vaddf\nto add the message, and then sends that to error() to give it\nanother \"error: \" prefix), but all the above does is to send a\nmessage to standard error stream.\n\n> \t\tstrbuf_release(&sb);\n> \t\treturn -1;\n> \t}\n"},{"id":"459642","messageId":"xmqqlesmla6b.fsf@gitster.g","threadId":"57898","inReplyTo":"b41853e3f9908ab458bcb28684d817677e32367b.1658391391.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 3/7] merge: do not abort early if one strategy fails to handle the merge","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-21T16:09:32Z","receivedAt":"2022-07-21T16:09:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> @@ -754,8 +754,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n> \t\telse\n> \t\t\tclean = merge_recursive(&o, head, remoteheads->item,\n> \t\t\t\t\t\treversed, &result);\n> -\t\tif (clean < 0)\n> -\t\t\texit(128);\n> +\t\tif (clean < 0) {\n> +\t\t\trollback_lock_file(&lock);\n> +\t\t\treturn 2;\n> +\t\t}\n\nVery good find. I however wonder if negative returns are signaling\na situation where they cannot cleanly recover from (but even if it\nis the case, if we are willing to do the save-restore dance, then it\nis probably OK).\n\n> diff --git a/t/t6402-merge-rename.sh b/t/t6402-merge-rename.sh\n> index 3a32b1a45cf..772238e582c 100755\n> --- a/t/t6402-merge-rename.sh\n> +++ b/t/t6402-merge-rename.sh\n> @@ -210,7 +210,7 @@ test_expect_success 'updated working tree file should prevent the merge' '\n> \techo >>M one line addition &&\n> \tcat M >M.saved &&\n> \tgit update-index M &&\n> -\ttest_expect_code 128 git pull --no-rebase . yellow &&\n> +\ttest_expect_code 2 git pull --no-rebase . yellow &&\n> \ttest_cmp M M.saved &&\n> \trm -f M.saved\n> '\n\nUnderstandable.\n\n> diff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\n> index f35d3182b86..8b749e19083 100755\n> --- a/t/t6424-merge-unrelated-index-changes.sh\n> +++ b/t/t6424-merge-unrelated-index-changes.sh\n> @@ -268,4 +268,20 @@ test_expect_success 'subtree' '\n> \ttest_path_is_missing .git/MERGE_HEAD\n> '\n> \n> +test_expect_success 'resolve && recursive && ort' '\n> +\tgit reset --hard &&\n> +\tgit checkout B^0 &&\n> +\n> +\ttest_seq 0 10 >a &&\n> +\tgit add a &&\n> +\n> +\tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n> +\ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n> +\n> +\tgrep \"Trying merge strategy resolve...\" output &&\n> +\tgrep \"Trying merge strategy recursive...\" output &&\n> +\tgrep \"Trying merge strategy ort...\" output &&\n> +\tgrep \"No merge strategy handled the merge.\" output\n> +'\n\nMakes sense.\n\n> test_done\n> diff --git a/t/t6439-merge-co-error-msgs.sh b/t/t6439-merge-co-error-msgs.sh\n> index 5bfb027099a..52cf0c87690 100755\n> --- a/t/t6439-merge-co-error-msgs.sh\n> +++ b/t/t6439-merge-co-error-msgs.sh\n> @@ -47,6 +47,7 @@ test_expect_success 'untracked files overwritten by merge (fast and non-fast for\n> \t\texport GIT_MERGE_VERBOSITY &&\n> \t\ttest_must_fail git merge branch 2>out2\n> \t) &&\n> +\techo \"Merge with strategy ${GIT_TEST_MERGE_ALGORITHM:-ort} failed.\" >>expect &&\n> \ttest_cmp out2 expect &&\n> \tgit reset --hard HEAD^\n> '\n"},{"id":"459653","messageId":"xmqqfsiul9vj.fsf@gitster.g","threadId":"57898","inReplyTo":"91c495c770e3f7f91ae655a503bdd1cd99935e40.1658391391.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 5/7] merge: make restore_state() restore staged state too","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-21T16:16:00Z","receivedAt":"2022-07-21T16:17:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> 4) Merge strategies can make changes to the index and working tree,\n> and have no expectation to clean up after themselves, *even* if\n> they bail out and say they are not an appropriate handler for\n> the merge in question. (The `octopus` merge strategy does this,\n> for example.)\n\nI personally consider this is a bug in `octopus` (and I can do so\nwithout offending anybody, as `octopus` was what I did), but because\nthe point of having pluggable merge strategies is to allow end users\nand third parties write their own. So save-and-restore dance would\nbe a prudent approach to the issue than forbidding this \"buggy\"\nbehaviour.\n\n> Unfortunately, if users had staged changes before calling `git merge`,\n> builtin/merge.c could do the following:\n>\n>    * stash the changes, in order to clean up after the strategies\n>    * try all the merge strategies in turn, each of which report they\n> cannot function due to the index not matching HEAD\n>    * restore the changes via \"git stash apply\"\n\n> \ttest_seq 0 10 >a &&\n> \tgit add a &&\n> +\tgit rev-parse :a >expect &&\n> ...\n> +\t# Changes to \"a\" should remain staged\n> +\tgit rev-parse :a >actual &&\n> +\ttest_cmp expect actual\n\nMakes sense.\n\nThanks.\n"},{"id":"459654","messageId":"xmqq8roml9h6.fsf@gitster.g","threadId":"57898","inReplyTo":"91c495c770e3f7f91ae655a503bdd1cd99935e40.1658391391.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 5/7] merge: make restore_state() restore staged state too","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-21T16:24:37Z","receivedAt":"2022-07-21T16:24:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n>    * stash the changes, in order to clean up after the strategies\n>    * try all the merge strategies in turn, each of which report they\n> cannot function due to the index not matching HEAD\n>    * restore the changes via \"git stash apply\"\n\nA tangent that does not make much difference in the end, but I am\nfinding these lines curious and somewhat annoying. Why do we have\n  sandwitching the plain whitespace only on lines that begin\nwith an asterisk?\n"},{"id":"459657","messageId":"xmqq35eul95b.fsf@gitster.g","threadId":"57898","inReplyTo":"887967c1f3fd6f03cf1d0bb3c19ed16819541092.1658391391.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 6/7] merge: ensure we can actually restore pre-merge state","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-21T16:31:44Z","receivedAt":"2022-07-21T16:31:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index f807bf335bd..11bb4bab0a1 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -1686,12 +1686,12 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> \t * tree in the index -- this means that the index must be in\n> \t * sync with the head commit. The strategies are responsible\n> \t * to ensure this.\n> +\t *\n> +\t * Stash away the local changes so that we can try more than one\n> +\t * and/or recover from merge strategies bailing while leaving the\n> +\t * index and working tree polluted.\n> \t */\n\nMakes sense. We may want to special-case strategies that are known\nnot to have the buggy \"leave contaminated tree when bailing out\"\nbehaviour to avoid waste. I expect that more than 99.99% of the\ntime people are feeding a single other commit to ort or recursive,\nand if these are known to be safe, a lot will be saved by not saving\n\"just in case\". But that can be left for later, after the series\nsolidifies.\n\nThanks.\n\n> -\tif (use_strategies_nr == 1 ||\n> -\t /*\n> -\t * Stash away the local changes so that we can try more than one.\n> -\t */\n> -\t save_state(&stash))\n> +\tif (save_state(&stash))\n> \t\toidclr(&stash);\n> \n> \tfor (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n"},{"id":"459658","messageId":"xmqqy1wmjugj.fsf@gitster.g","threadId":"57898","inReplyTo":"81c40492a62e81100c66a8ccc1ec200fb2e6fca5.1658391391.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 7/7] merge: do not exit restore_state() prematurely","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-21T16:34:20Z","receivedAt":"2022-07-21T16:34:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> @@ -386,11 +386,11 @@ static void restore_state(const struct object_id *head,\n> \tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n> \t\t\t NULL, NULL };\n> \n> -\tif (is_null_oid(stash))\n> -\t\treturn;\n> -\n> \treset_hard(head, 1);\n> \n> +\tif (is_null_oid(stash))\n> +\t\tgoto refresh_cache;\n> +\n\nOK, so the idea is that we can call restore_state() without having\nanything worth \"restoring\" in the stash, and what it means is that\nwe are restoring to HEAD. As the current state does not necessarily\nmatch HEAD, we should do the \"reset --hard\" part even if there was\nnothing to stash.\n\nMakes sense.\n\n"},{"id":"459716","messageId":"CABPp-BEddgN5QmFkfejC6jZXMAGTivQBtV8YQ8Jq0EZzPhAM8Q@mail.gmail.com","threadId":"57898","inReplyTo":"xmqqsfmulb6w.fsf@gitster.g","subject":"Re: [PATCH v3 1/7] merge-ort-wrappers: make printed message match the one from recursive","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-21T19:51:50Z","receivedAt":"2022-07-21T19:52:04Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Jul 21, 2022 at 8:47 AM Junio C Hamano wrote:\n>\n> \"Elijah Newren via GitGitGadget\" writes:\n>\n> > if (head && repo_index_has_changes(opt->repo, head, &sb)) {\n> > - fprintf(stderr, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> > + struct strbuf err = STRBUF_INIT;\n> > + strbuf_addstr(&err, \"error: \");\n> > + strbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> > sb.buf);\n> > + strbuf_addch(&err, '\\n');\n> > + fputs(err.buf, stderr);\n> > + strbuf_release(&err);\n>\n> Makes me wonder why this is not a mere\n>\n> error(_(\"Your local chagnes ... by merge:\\n %s\"), sb.buf);\n>\n> that reuses the exact string. The err() function in merge-recursive.c\n> is strangely complex (and probably buggy---if it is not buffering\n> output, it adds \"error: \" prefix to opt->obuf before calling vaddf\n> to add the message, and then sends that to error() to give it\n> another \"error: \" prefix), but all the above does is to send a\n> message to standard error stream.\n\nAh, that would be nicer; thanks for the pointer. I would still need\nto prefix it with an\n strbuf_addch(&sb, '\\n');\nbut two lines certainly beats six.\n\n>\n> > strbuf_release(&sb);\n> > return -1;\n> > }\n"},{"id":"459717","messageId":"CABPp-BG5Dvoft_RhSz5PicGJyL3-j2PDqoxF4+cfH3k3NhVz4g@mail.gmail.com","threadId":"57898","inReplyTo":"xmqq8roml9h6.fsf@gitster.g","subject":"Re: [PATCH v3 5/7] merge: make restore_state() restore staged state too","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-21T19:52:01Z","receivedAt":"2022-07-21T19:52:16Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Jul 21, 2022 at 9:24 AM Junio C Hamano wrote:\n>\n> \"Elijah Newren via GitGitGadget\" writes:\n>\n> > * stash the changes, in order to clean up after the strategies\n> > * try all the merge strategies in turn, each of which report they\n> > cannot function due to the index not matching HEAD\n> > * restore the changes via \"git stash apply\"\n>\n> A tangent that does not make much difference in the end, but I am\n> finding these lines curious and somewhat annoying. Why do we have\n>   sandwitching the plain whitespace only on lines that begin\n> with an asterisk?\n\nEek! That's nasty. I was switching back and forth between responding\nto emails and coding up fixes, and copy-pasted part of my email into\nthe commit message. So, apparently gmail hates me, and I just didn't\nnotice.\n\nI'll clean these up; I'm rerolling for the \"error()\" cleanup you\nmentioned anyway.\n"},{"id":"459721","messageId":"xmqq1quegrj5.fsf@gitster.g","threadId":"57898","inReplyTo":"CABPp-BEddgN5QmFkfejC6jZXMAGTivQBtV8YQ8Jq0EZzPhAM8Q@mail.gmail.com","subject":"Re: [PATCH v3 1/7] merge-ort-wrappers: make printed message match the one from recursive","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-21T20:05:50Z","receivedAt":"2022-07-21T20:06:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n> On Thu, Jul 21, 2022 at 8:47 AM Junio C Hamano wrote:\n>>\n>> \"Elijah Newren via GitGitGadget\" writes:\n>>\n>> > if (head && repo_index_has_changes(opt->repo, head, &sb)) {\n>> > - fprintf(stderr, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n>> > + struct strbuf err = STRBUF_INIT;\n>> > + strbuf_addstr(&err, \"error: \");\n>> > + strbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n>> > sb.buf);\n>> > + strbuf_addch(&err, '\\n');\n>> > + fputs(err.buf, stderr);\n>> > + strbuf_release(&err);\n>>\n>> Makes me wonder why this is not a mere\n>>\n>> error(_(\"Your local chagnes ... by merge:\\n %s\"), sb.buf);\n>>\n>> that reuses the exact string. The err() function in merge-recursive.c\n>> is strangely complex (and probably buggy---if it is not buffering\n>> output, it adds \"error: \" prefix to opt->obuf before calling vaddf\n>> to add the message, and then sends that to error() to give it\n>> another \"error: \" prefix), but all the above does is to send a\n>> message to standard error stream.\n>\n> Ah, that would be nicer; thanks for the pointer. I would still need\n> to prefix it with an\n> strbuf_addch(&sb, '\\n');\n> but two lines certainly beats six.\n\nYour \"strbuf\" version uses the same format string as my error()\nthing and then manually add one LF at the end, before sending it to\nfputs(), which, unlike puts() does not add any extra LF at the end.\n\nerror() gives a terminating newline at the end.\n\nDo you still need to add one more?\n"},{"id":"459725","messageId":"CABPp-BGttaVetjRnxYM8ti0fHNcvsNYc9NcWXKQwH4Qqbii9wg@mail.gmail.com","threadId":"57898","inReplyTo":"xmqq1quegrj5.fsf@gitster.g","subject":"Re: [PATCH v3 1/7] merge-ort-wrappers: make printed message match the one from recursive","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-21T21:14:03Z","receivedAt":"2022-07-21T21:14:17Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Jul 21, 2022 at 1:05 PM Junio C Hamano wrote:\n>\n> Elijah Newren writes:\n>\n> > On Thu, Jul 21, 2022 at 8:47 AM Junio C Hamano wrote:\n> >>\n> >> \"Elijah Newren via GitGitGadget\" writes:\n> >>\n> >> > if (head && repo_index_has_changes(opt->repo, head, &sb)) {\n> >> > - fprintf(stderr, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> >> > + struct strbuf err = STRBUF_INIT;\n> >> > + strbuf_addstr(&err, \"error: \");\n> >> > + strbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> >> > sb.buf);\n> >> > + strbuf_addch(&err, '\\n');\n> >> > + fputs(err.buf, stderr);\n> >> > + strbuf_release(&err);\n> >>\n> >> Makes me wonder why this is not a mere\n> >>\n> >> error(_(\"Your local chagnes ... by merge:\\n %s\"), sb.buf);\n> >>\n> >> that reuses the exact string. The err() function in merge-recursive.c\n> >> is strangely complex (and probably buggy---if it is not buffering\n> >> output, it adds \"error: \" prefix to opt->obuf before calling vaddf\n> >> to add the message, and then sends that to error() to give it\n> >> another \"error: \" prefix), but all the above does is to send a\n> >> message to standard error stream.\n> >\n> > Ah, that would be nicer; thanks for the pointer. I would still need\n> > to prefix it with an\n> > strbuf_addch(&sb, '\\n');\n> > but two lines certainly beats six.\n>\n> Your \"strbuf\" version uses the same format string as my error()\n> thing and then manually add one LF at the end, before sending it to\n> fputs(), which, unlike puts() does not add any extra LF at the end.\n>\n> error() gives a terminating newline at the end.\n>\n> Do you still need to add one more?\n\nAh, sorry, my mistake. I somehow thought error() just added the\n\"error: \" prefix. So, indeed, this is just a change from\nfprintf(stderr, ...) to error(...). No second newline needed.\n\nThanks!\n"},{"id":"459745","messageId":"bd36d16c8d93176bac12acaf90f654a0acb16cd6.1658466942.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","subject":"[PATCH v4 1/7] merge-ort-wrappers: make printed message match the one from recursive","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-22T05:15:36Z","receivedAt":"2022-07-22T05:15:52Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nWhen the index does not match HEAD, the merge strategies are responsible\nto detect that condition and abort. The merge-ort-wrappers had code to\nimplement this and meant to copy the error message from merge-recursive\nbut deviated in two ways, both due to the message in merge-recursive\nbeing processed by another function that made additional changes:\n * It added an implicit \"error: \" prefix\n * It added an implicit trailing newline\nWe can get these things by making use of the error() function.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort-wrappers.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort-wrappers.c b/merge-ort-wrappers.c\nindex ad041061695..748924a69ba 100644\n--- a/merge-ort-wrappers.c\n+++ b/merge-ort-wrappers.c\n@@ -10,8 +10,8 @@ static int unclean(struct merge_options *opt, struct tree *head)\n \tstruct strbuf sb = STRBUF_INIT;\n \n \tif (head && repo_index_has_changes(opt->repo, head, &sb)) {\n-\t\tfprintf(stderr, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n-\t\t sb.buf);\n+\t\terror(_(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n+\t\t sb.buf);\n \t\tstrbuf_release(&sb);\n \t\treturn -1;\n \t}\n-- \ngitgitgadget\n\n"},{"id":"459746","messageId":"b79f44e54b9611fa2b10a4e1cb666992d006951c.1658466942.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","subject":"[PATCH v4 2/7] merge-resolve: abort if index does not match HEAD","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-22T05:15:37Z","receivedAt":"2022-07-22T05:15:56Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nAs noted in commit 9822175d2b (\"Ensure index matches head before\ninvoking merge machinery, round N\", 2019-08-17), we have had a very\nlong history of problems with failing to enforce the requirement that\nindex matches HEAD when starting a merge. One of the commits\nreferenced in the long tale of issues arising from lax enforcement of\nthis requirement was commit 55f39cf755 (\"merge: fix misleading\npre-merge check documentation\", 2018-06-30), which tried to document\nthe requirement and noted there were some exceptions. As mentioned in\nthat commit message, the `resolve` strategy was the one strategy that\ndid not have an explicit index matching HEAD check, and the reason it\ndidn't was that I wasn't able to discover any cases where the\nimplementation would fail to catch the problem and abort, and didn't\nwant to introduce unnecessary performance overhead of adding another\ncheck.\n\nWell, today I discovered a testcase where the implementation does not\ncatch the problem and so an explicit check is needed. Add a testcase\nthat previously would have failed, and update git-merge-resolve.sh to\nhave an explicit check. Note that the code is copied from 3ec62ad9ff\n(\"merge-octopus: abort if index does not match HEAD\", 2016-04-09), so\nthat we reuse the same message and avoid making translators need to\ntranslate some new message.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 20 ++++++++++++++++++\n git-merge-resolve.sh | 10 +++++++++\n t/t6424-merge-unrelated-index-changes.sh | 26 ++++++++++++++++++++++++\n 3 files changed, 56 insertions(+)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 23170f2d2a6..13884b8e836 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -1599,6 +1599,26 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t\t */\n \t\trefresh_cache(REFRESH_QUIET);\n \t\tif (allow_trivial && fast_forward != FF_ONLY) {\n+\t\t\t/*\n+\t\t\t * Must first ensure that index matches HEAD before\n+\t\t\t * attempting a trivial merge.\n+\t\t\t */\n+\t\t\tstruct tree *head_tree = get_commit_tree(head_commit);\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tif (repo_index_has_changes(the_repository, head_tree,\n+\t\t\t\t\t\t &sb)) {\n+\t\t\t\tstruct strbuf err = STRBUF_INIT;\n+\t\t\t\tstrbuf_addstr(&err, \"error: \");\n+\t\t\t\tstrbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n+\t\t\t\t\t sb.buf);\n+\t\t\t\tstrbuf_addch(&err, '\\n');\n+\t\t\t\tfputs(err.buf, stderr);\n+\t\t\t\tstrbuf_release(&err);\n+\t\t\t\tstrbuf_release(&sb);\n+\t\t\t\treturn -1;\n+\t\t\t}\n+\n \t\t\t/* See if it is really trivial. */\n \t\t\tgit_committer_info(IDENT_STRICT);\n \t\t\tprintf(_(\"Trying really trivial in-index merge...\\n\"));\ndiff --git a/git-merge-resolve.sh b/git-merge-resolve.sh\nindex 343fe7bccd0..77e93121bf8 100755\n--- a/git-merge-resolve.sh\n+++ b/git-merge-resolve.sh\n@@ -5,6 +5,16 @@\n #\n # Resolve two trees, using enhanced multi-base read-tree.\n \n+. git-sh-setup\n+\n+# Abort if index does not match HEAD\n+if ! git diff-index --quiet --cached HEAD --\n+then\n+ gettextln \"Error: Your local changes to the following files would be overwritten by merge\"\n+ git diff-index --cached --name-only HEAD -- | sed -e 's/^/ /'\n+ exit 2\n+fi\n+\n # The first parameters up to -- are merge bases; the rest are heads.\n bases= head= remotes= sep_seen=\n for arg\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex b6e424a427b..f35d3182b86 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -114,6 +114,32 @@ test_expect_success 'resolve, non-trivial' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'resolve, trivial, related file removed' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\tgit rm a &&\n+\ttest_path_is_missing a &&\n+\n+\ttest_must_fail git merge -s resolve C^0 &&\n+\n+\ttest_path_is_missing a &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n+test_expect_success 'resolve, non-trivial, related file removed' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\tgit rm a &&\n+\ttest_path_is_missing a &&\n+\n+\ttest_must_fail git merge -s resolve D^0 &&\n+\n+\ttest_path_is_missing a &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n test_expect_success 'recursive' '\n \tgit reset --hard &&\n \tgit checkout B^0 &&\n-- \ngitgitgadget\n\n"},{"id":"459747","messageId":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v3.git.1658391391.gitgitgadget@gmail.com","subject":"[PATCH v4 0/7] Fix merge restore state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-22T05:15:35Z","receivedAt":"2022-07-22T05:15:56Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This started as a simple series to fix restore_state() in builtin/merge.c,\nfixing an issue reported by ZheNing Hu[3]. It's grown so much it's hard to\ncall it simple. Anyway...\n\nChanges since v3:\n\n * Removed some accidental   characters from a commit message\n * Made use of the error() function to simplify the first patch\n\n[1]\nhttps://lore.kernel.org/git/CAOLTT8R7QmpvaFPTRs3xTpxr7eiuxF-ZWtvUUSC0-JOo9Y+SqA@mail.gmail.com/\n\nElijah Newren (7):\n merge-ort-wrappers: make printed message match the one from recursive\n merge-resolve: abort if index does not match HEAD\n merge: do not abort early if one strategy fails to handle the merge\n merge: fix save_state() to work when there are stat-dirty files\n merge: make restore_state() restore staged state too\n merge: ensure we can actually restore pre-merge state\n merge: do not exit restore_state() prematurely\n\n builtin/merge.c | 59 ++++++++++++++++++------\n git-merge-resolve.sh | 10 ++++\n merge-ort-wrappers.c | 4 +-\n t/t6402-merge-rename.sh | 2 +-\n t/t6424-merge-unrelated-index-changes.sh | 58 +++++++++++++++++++++++\n t/t6439-merge-co-error-msgs.sh | 1 +\n t/t7607-merge-state.sh | 32 +++++++++++++\n 7 files changed, 150 insertions(+), 16 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\n\nbase-commit: e72d93e88cb20b06e88e6e7d81bd1dc4effe453f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1231%2Fnewren%2Ffix-merge-restore-state-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1231/newren/fix-merge-restore-state-v4\nPull-Request: https://github.com/gitgitgadget/git/pull/1231\n\nRange-diff vs v3:\n\n 1: e39b2e15ece ! 1: bd36d16c8d9 merge-ort-wrappers: make printed message match the one from recursive\n @@ Commit message\n being processed by another function that made additional changes:\n * It added an implicit \"error: \" prefix\n * It added an implicit trailing newline\n -\n - Add these things, but do so in a couple extra steps to avoid having\n - translators need to translate another not-quite-identical string.\n + We can get these things by making use of the error() function.\n \n Signed-off-by: Elijah Newren \n \n @@ merge-ort-wrappers.c: static int unclean(struct merge_options *opt, struct tree\n \n \tif (head && repo_index_has_changes(opt->repo, head, &sb)) {\n -\t\tfprintf(stderr, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n -+\t\tstruct strbuf err = STRBUF_INIT;\n -+\t\tstrbuf_addstr(&err, \"error: \");\n -+\t\tstrbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n - \t\t sb.buf);\n -+\t\tstrbuf_addch(&err, '\\n');\n -+\t\tfputs(err.buf, stderr);\n -+\t\tstrbuf_release(&err);\n +-\t\t sb.buf);\n ++\t\terror(_(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n ++\t\t sb.buf);\n \t\tstrbuf_release(&sb);\n \t\treturn -1;\n \t}\n 2: 2810dec7608 = 2: b79f44e54b9 merge-resolve: abort if index does not match HEAD\n 3: b41853e3f99 = 3: 02930448ea1 merge: do not abort early if one strategy fails to handle the merge\n 4: 64700338a28 = 4: daf8d224160 merge: fix save_state() to work when there are stat-dirty files\n 5: 91c495c770e ! 5: f401bd5ad0d merge: make restore_state() restore staged state too\n @@ Commit message\n Unfortunately, if users had staged changes before calling `git merge`,\n builtin/merge.c could do the following:\n \n -    * stash the changes, in order to clean up after the strategies\n -    * try all the merge strategies in turn, each of which report they\n + * stash the changes, in order to clean up after the strategies\n + * try all the merge strategies in turn, each of which report they\n cannot function due to the index not matching HEAD\n -    * restore the changes via \"git stash apply\"\n + * restore the changes via \"git stash apply\"\n \n But that last step would have the net effect of unstaging the user's\n changes. Fix this by adding the \"--index\" option to \"git stash apply\".\n 6: 887967c1f3f = 6: ad5354c219c merge: ensure we can actually restore pre-merge state\n 7: 81c40492a62 = 7: 6212d572604 merge: do not exit restore_state() prematurely\n\n-- \ngitgitgadget\n"},{"id":"459748","messageId":"02930448ea1fbf7084b9d78813908b6355304457.1658466942.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","subject":"[PATCH v4 3/7] merge: do not abort early if one strategy fails to handle the merge","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-22T05:15:38Z","receivedAt":"2022-07-22T05:15:57Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nbuiltin/merge is setup to allow multiple strategies to be specified,\nand it will find the \"best\" result and use it. This is defeated if\nsome of the merge strategies abort early when they cannot handle the\nmerge. Fix the logic that calls recursive and ort to not do such an\nearly abort, but instead return \"2\" or \"unhandled\" so that the next\nstrategy can try to handle the merge.\n\nComing up with a testcase for this is somewhat difficult, since\nrecursive and ort both handle nearly any two-headed merge (there is\na separate code path that checks for non-two-headed merges and\nalready returns \"2\" for them). So use a somewhat synthetic testcase\nof having the index not match HEAD before the merge starts, since all\nmerge strategies will abort for that.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 6 ++++--\n t/t6402-merge-rename.sh | 2 +-\n t/t6424-merge-unrelated-index-changes.sh | 16 ++++++++++++++++\n t/t6439-merge-co-error-msgs.sh | 1 +\n 4 files changed, 22 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 13884b8e836..dec7375bf2a 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -754,8 +754,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n \t\telse\n \t\t\tclean = merge_recursive(&o, head, remoteheads->item,\n \t\t\t\t\t\treversed, &result);\n-\t\tif (clean < 0)\n-\t\t\texit(128);\n+\t\tif (clean < 0) {\n+\t\t\trollback_lock_file(&lock);\n+\t\t\treturn 2;\n+\t\t}\n \t\tif (write_locked_index(&the_index, &lock,\n \t\t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n \t\t\tdie(_(\"unable to write %s\"), get_index_file());\ndiff --git a/t/t6402-merge-rename.sh b/t/t6402-merge-rename.sh\nindex 3a32b1a45cf..772238e582c 100755\n--- a/t/t6402-merge-rename.sh\n+++ b/t/t6402-merge-rename.sh\n@@ -210,7 +210,7 @@ test_expect_success 'updated working tree file should prevent the merge' '\n \techo >>M one line addition &&\n \tcat M >M.saved &&\n \tgit update-index M &&\n-\ttest_expect_code 128 git pull --no-rebase . yellow &&\n+\ttest_expect_code 2 git pull --no-rebase . yellow &&\n \ttest_cmp M M.saved &&\n \trm -f M.saved\n '\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex f35d3182b86..8b749e19083 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -268,4 +268,20 @@ test_expect_success 'subtree' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'resolve && recursive && ort' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\ttest_seq 0 10 >a &&\n+\tgit add a &&\n+\n+\tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n+\ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n+\n+\tgrep \"Trying merge strategy resolve...\" output &&\n+\tgrep \"Trying merge strategy recursive...\" output &&\n+\tgrep \"Trying merge strategy ort...\" output &&\n+\tgrep \"No merge strategy handled the merge.\" output\n+'\n+\n test_done\ndiff --git a/t/t6439-merge-co-error-msgs.sh b/t/t6439-merge-co-error-msgs.sh\nindex 5bfb027099a..52cf0c87690 100755\n--- a/t/t6439-merge-co-error-msgs.sh\n+++ b/t/t6439-merge-co-error-msgs.sh\n@@ -47,6 +47,7 @@ test_expect_success 'untracked files overwritten by merge (fast and non-fast for\n \t\texport GIT_MERGE_VERBOSITY &&\n \t\ttest_must_fail git merge branch 2>out2\n \t) &&\n+\techo \"Merge with strategy ${GIT_TEST_MERGE_ALGORITHM:-ort} failed.\" >>expect &&\n \ttest_cmp out2 expect &&\n \tgit reset --hard HEAD^\n '\n-- \ngitgitgadget\n\n"},{"id":"459749","messageId":"daf8d22416003b050cb6b7f4defbb6276ea4a278.1658466942.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","subject":"[PATCH v4 4/7] merge: fix save_state() to work when there are stat-dirty files","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-22T05:15:39Z","receivedAt":"2022-07-22T05:16:00Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nWhen there are stat-dirty files, but no files are modified,\n`git stash create` exits with unsuccessful status. This causes merge\nto fail. Copy some code from sequencer.c's create_autostash to refresh\nthe index first to avoid this problem.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 8 ++++++++\n t/t6424-merge-unrelated-index-changes.sh | 11 +++++++++++\n 2 files changed, 19 insertions(+)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex dec7375bf2a..4170c30317e 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -313,8 +313,16 @@ static int save_state(struct object_id *stash)\n \tint len;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tstruct strbuf buffer = STRBUF_INIT;\n+\tstruct lock_file lock_file = LOCK_INIT;\n+\tint fd;\n \tint rc = -1;\n \n+\tfd = repo_hold_locked_index(the_repository, &lock_file, 0);\n+\trefresh_cache(REFRESH_QUIET);\n+\tif (0 <= fd)\n+\t\trepo_update_index_if_able(the_repository, &lock_file);\n+\trollback_lock_file(&lock_file);\n+\n \tstrvec_pushl(&cp.args, \"stash\", \"create\", NULL);\n \tcp.out = -1;\n \tcp.git_cmd = 1;\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex 8b749e19083..3019d030e07 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -268,6 +268,17 @@ test_expect_success 'subtree' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'avoid failure due to stat-dirty files' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\t# Make \"a\" be stat-dirty\n+\ttest-tool chmtime =+1 a &&\n+\n+\t# stat-dirty file should not prevent stash creation in builtin/merge.c\n+\tgit merge -s resolve -s recursive D^0\n+'\n+\n test_expect_success 'resolve && recursive && ort' '\n \tgit reset --hard &&\n \tgit checkout B^0 &&\n-- \ngitgitgadget\n\n"},{"id":"459750","messageId":"f401bd5ad0dd7564412e72d19a4193ad3f64e638.1658466942.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","subject":"[PATCH v4 5/7] merge: make restore_state() restore staged state too","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-22T05:15:40Z","receivedAt":"2022-07-22T05:16:01Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nThere are multiple issues at play here:\n\n 1) If `git merge` is invoked with staged changes, it should abort\n without doing any merging, and the user's working tree and index\n should be the same as before merge was invoked.\n 2) Merge strategies are responsible for enforcing the index == HEAD\n requirement. (See 9822175d2b (\"Ensure index matches head before\n invoking merge machinery, round N\", 2019-08-17) for some history\n around this.)\n 3) Merge strategies can bail saying they are not an appropriate\n handler for the merge in question (possibly allowing other\n strategies to be used instead).\n 4) Merge strategies can make changes to the index and working tree,\n and have no expectation to clean up after themselves, *even* if\n they bail out and say they are not an appropriate handler for\n the merge in question. (The `octopus` merge strategy does this,\n for example.)\n 5) Because of (3) and (4), builtin/merge.c stashes state before\n trying merge strategies and restores it afterward.\n\nUnfortunately, if users had staged changes before calling `git merge`,\nbuiltin/merge.c could do the following:\n\n * stash the changes, in order to clean up after the strategies\n * try all the merge strategies in turn, each of which report they\n cannot function due to the index not matching HEAD\n * restore the changes via \"git stash apply\"\n\nBut that last step would have the net effect of unstaging the user's\nchanges. Fix this by adding the \"--index\" option to \"git stash apply\".\nWhile at it, also squelch the stash apply output; we already report\n\"Rewinding the tree to pristine...\" and don't need a detailed `git\nstatus` report afterwards.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 5 +++--\n t/t6424-merge-unrelated-index-changes.sh | 7 ++++++-\n 2 files changed, 9 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 4170c30317e..f807bf335bd 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -383,14 +383,15 @@ static void reset_hard(const struct object_id *oid, int verbose)\n static void restore_state(const struct object_id *head,\n \t\t\t const struct object_id *stash)\n {\n-\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n+\tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n+\t\t\t NULL, NULL };\n \n \tif (is_null_oid(stash))\n \t\treturn;\n \n \treset_hard(head, 1);\n \n-\targs[2] = oid_to_hex(stash);\n+\targs[4] = oid_to_hex(stash);\n \n \t/*\n \t * It is OK to ignore error here, for example when there was\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex 3019d030e07..c96649448fa 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -285,6 +285,7 @@ test_expect_success 'resolve && recursive && ort' '\n \n \ttest_seq 0 10 >a &&\n \tgit add a &&\n+\tgit rev-parse :a >expect &&\n \n \tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n \ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n@@ -292,7 +293,11 @@ test_expect_success 'resolve && recursive && ort' '\n \tgrep \"Trying merge strategy resolve...\" output &&\n \tgrep \"Trying merge strategy recursive...\" output &&\n \tgrep \"Trying merge strategy ort...\" output &&\n-\tgrep \"No merge strategy handled the merge.\" output\n+\tgrep \"No merge strategy handled the merge.\" output &&\n+\n+\t# Changes to \"a\" should remain staged\n+\tgit rev-parse :a >actual &&\n+\ttest_cmp expect actual\n '\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"459751","messageId":"ad5354c219cc266972d4ea37a7e0acf9a981ffec.1658466942.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","subject":"[PATCH v4 6/7] merge: ensure we can actually restore pre-merge state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-22T05:15:41Z","receivedAt":"2022-07-22T05:16:02Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nMerge strategies can:\n * succeed with a clean merge\n * succeed with a conflicted merge\n * fail to handle the given type of merge\n\nIf one is thinking in terms of automatic mergeability, they would use\nthe word \"fail\" instead of \"succeed\" for the second bullet, but I am\nfocusing here on ability of the merge strategy to handle the given\ninputs, not on whether the given inputs are mergeable. The third\ncategory is about the merge strategy failing to know how to handle the\ngiven data; examples include:\n\n * Passing more than 2 branches to 'recursive' or 'ort'\n * Passing 2 or fewer branches to 'octopus'\n * Trying to do more complicated merges with 'resolve' (I believe\n directory/file conflicts will cause it to bail.)\n * Octopus running into a merge conflict for any branch OTHER than\n the final one (see the \"exit 2\" codepath of commit 98efc8f3d8\n (\"octopus: allow manual resolve on the last round.\", 2006-01-13))\n\nThat final one is particularly interesting, because it shows that the\nmerge strategy can muck with the index and working tree, and THEN bail\nand say \"sorry, this strategy cannot handle this type of merge; use\nsomething else\".\n\nFurther, we do not currently expect the individual strategies to clean\nup after themselves, but instead expect builtin/merge.c to do so. For\nit to be able to, it needs to save the state before trying the merge\nstrategy so it can have something to restore to. Therefore, remove the\nshortcut bypassing the save_state() call.\n\nThere is another bug on the restore_state() side of things, so no\ntestcase will be added until the next commit when we have addressed that\nissue as well.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex f807bf335bd..11bb4bab0a1 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -1686,12 +1686,12 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t * tree in the index -- this means that the index must be in\n \t * sync with the head commit. The strategies are responsible\n \t * to ensure this.\n+\t *\n+\t * Stash away the local changes so that we can try more than one\n+\t * and/or recover from merge strategies bailing while leaving the\n+\t * index and working tree polluted.\n \t */\n-\tif (use_strategies_nr == 1 ||\n-\t /*\n-\t * Stash away the local changes so that we can try more than one.\n-\t */\n-\t save_state(&stash))\n+\tif (save_state(&stash))\n \t\toidclr(&stash);\n \n \tfor (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n-- \ngitgitgadget\n\n"},{"id":"459752","messageId":"6212d572604cd8ec5c57ab0c66da53bd95ed6005.1658466942.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","subject":"[PATCH v4 7/7] merge: do not exit restore_state() prematurely","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-22T05:15:42Z","receivedAt":"2022-07-22T05:16:04Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nPreviously, if the user:\n\n* Had no local changes before starting the merge\n* A merge strategy makes changes to the working tree/index but returns\n with exit status 2\n\nThen we'd call restore_state() to clean up the changes and either let\nthe next merge strategy run (if there is one), or exit telling the user\nthat no merge strategy could handle the merge. Unfortunately,\nrestore_state() did not clean up the changes as expected; that function\nwas a no-op if the stash was a null, and the stash would be null if\nthere were no local changes before starting the merge. So, instead of\n\"Rewinding the tree to pristine...\" as the code claimed, restore_state()\nwould leave garbage around in the index and working tree (possibly\nincluding conflicts) for either the next merge strategy or for the user\nafter aborting the merge. And in the case of aborting the merge, the\nuser would be unable to run \"git merge --abort\" to get rid of the\nunintended leftover conflicts, because the merge control files were not\nwritten as it was presumed that we had restored to a clean state\nalready.\n\nFix the main problem by making sure that restore_state() only skips the\nstash application if the stash is null rather than skipping the whole\nfunction.\n\nHowever, there is a secondary problem -- since merge.c forks\nsubprocesses to do the cleanup, the in-memory index is left out-of-sync.\nWhile there was a refresh_cache(REFRESH_QUIET) call that attempted to\ncorrect that, that function would not handle cases where the previous\nmerge strategy added conflicted entries. We need to drop the index and\nre-read it to handle such cases.\n\n(Alternatively, we could stop forking subprocesses and instead call some\nappropriate function to do the work which would update the in-memory\nindex automatically. For now, just do the simple fix.)\n\nAlso, add a testcase checking this, one for which the octopus strategy\nfails on the first commit it attempts to merge, and thus which it\ncannot handle at all and must completely bail on (as per the \"exit 2\"\ncode path of commit 98efc8f3d8 (\"octopus: allow manual resolve on the\nlast round.\", 2006-01-13)).\n\nReported-by: ZheNing Hu \nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 10 ++++++----\n t/t7607-merge-state.sh | 32 ++++++++++++++++++++++++++++++++\n 2 files changed, 38 insertions(+), 4 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 11bb4bab0a1..7fb4414ebb7 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -386,11 +386,11 @@ static void restore_state(const struct object_id *head,\n \tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n \t\t\t NULL, NULL };\n \n-\tif (is_null_oid(stash))\n-\t\treturn;\n-\n \treset_hard(head, 1);\n \n+\tif (is_null_oid(stash))\n+\t\tgoto refresh_cache;\n+\n \targs[4] = oid_to_hex(stash);\n \n \t/*\n@@ -399,7 +399,9 @@ static void restore_state(const struct object_id *head,\n \t */\n \trun_command_v_opt(args, RUN_GIT_CMD);\n \n-\trefresh_cache(REFRESH_QUIET);\n+refresh_cache:\n+\tif (discard_cache() < 0 || read_cache() < 0)\n+\t\tdie(_(\"could not read index\"));\n }\n \n /* This is called when no merge was necessary. */\ndiff --git a/t/t7607-merge-state.sh b/t/t7607-merge-state.sh\nnew file mode 100755\nindex 00000000000..fc33d57357b\n--- /dev/null\n+++ b/t/t7607-merge-state.sh\n@@ -0,0 +1,32 @@\n+#!/bin/sh\n+\n+test_description=\"Test that merge state is as expected after failed merge\"\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+. ./test-lib.sh\n+\n+test_expect_success 'set up custom strategy' '\n+\ttest_commit --no-tag \"Initial\" base base &&\n+\n+\tfor b in branch1 branch2 branch3\n+\tdo\n+\t\tgit checkout -b $b main &&\n+\t\ttest_commit --no-tag \"Change on $b\" base $b || return 1\n+\tdone &&\n+\n+\tgit checkout branch1 &&\n+\t# This is a merge that octopus cannot handle. Note, that it does not\n+\t# just hit conflicts, it completely fails and says that it cannot\n+\t# handle this type of merge.\n+\ttest_expect_code 2 git merge branch2 branch3 >output 2>&1 &&\n+\tgrep \"fatal: merge program failed\" output &&\n+\tgrep \"Should not be doing an octopus\" output &&\n+\n+\t# Make sure we did not leave stray changes around when no appropriate\n+\t# merge strategy was found\n+\tgit diff --exit-code --name-status &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n+test_done\n-- \ngitgitgadget\n"},{"id":"459762","messageId":"220722.86sfmts9vr.gmgdl@evledraar.gmail.com","threadId":"57898","inReplyTo":"b79f44e54b9611fa2b10a4e1cb666992d006951c.1658466942.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 2/7] merge-resolve: abort if index does not match HEAD","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-22T10:27:55Z","receivedAt":"2022-07-22T10:46:23Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jul 22 2022, Elijah Newren via GitGitGadget wrote:\n\n> From: Elijah Newren \n>\n> As noted in commit 9822175d2b (\"Ensure index matches head before\n> invoking merge machinery, round N\", 2019-08-17), we have had a very\n> long history of problems with failing to enforce the requirement that\n> index matches HEAD when starting a merge. One of the commits\n> referenced in the long tale of issues arising from lax enforcement of\n> this requirement was commit 55f39cf755 (\"merge: fix misleading\n> pre-merge check documentation\", 2018-06-30), which tried to document\n> the requirement and noted there were some exceptions. As mentioned in\n> that commit message, the `resolve` strategy was the one strategy that\n> did not have an explicit index matching HEAD check, and the reason it\n> didn't was that I wasn't able to discover any cases where the\n> implementation would fail to catch the problem and abort, and didn't\n> want to introduce unnecessary performance overhead of adding another\n> check.\n>\n> Well, today I discovered a testcase where the implementation does not\n> catch the problem and so an explicit check is needed. Add a testcase\n> that previously would have failed, and update git-merge-resolve.sh to\n> have an explicit check. Note that the code is copied from 3ec62ad9ff\n> (\"merge-octopus: abort if index does not match HEAD\", 2016-04-09), so\n> that we reuse the same message and avoid making translators need to\n> translate some new message.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 20 ++++++++++++++++++\n> git-merge-resolve.sh | 10 +++++++++\n> t/t6424-merge-unrelated-index-changes.sh | 26 ++++++++++++++++++++++++\n> 3 files changed, 56 insertions(+)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 23170f2d2a6..13884b8e836 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -1599,6 +1599,26 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> \t\t */\n> \t\trefresh_cache(REFRESH_QUIET);\n> \t\tif (allow_trivial && fast_forward != FF_ONLY) {\n> +\t\t\t/*\n> +\t\t\t * Must first ensure that index matches HEAD before\n> +\t\t\t * attempting a trivial merge.\n> +\t\t\t */\n> +\t\t\tstruct tree *head_tree = get_commit_tree(head_commit);\n> +\t\t\tstruct strbuf sb = STRBUF_INIT;\n> +\n> +\t\t\tif (repo_index_has_changes(the_repository, head_tree,\n> +\t\t\t\t\t\t &sb)) {\n> +\t\t\t\tstruct strbuf err = STRBUF_INIT;\n> +\t\t\t\tstrbuf_addstr(&err, \"error: \");\n> +\t\t\t\tstrbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> +\t\t\t\t\t sb.buf);\n> +\t\t\t\tstrbuf_addch(&err, '\\n');\n\nAt first glance I was expecting this to construct an error message to\nemit it somewhere else that stderr, so I wondered if you couldn't use\nthe \"error_routine\" facility to avoid re-inventing \"error: \" etc.,\nbut...\n\n> +\t\t\t\tfputs(err.buf, stderr);\n\n...we emit it to stderr anyway...?\n\n> +\t\t\t\tstrbuf_release(&err);\n> +\t\t\t\tstrbuf_release(&sb);\n> +\t\t\t\treturn -1;\n> +\t\t\t}\n> +\n> \t\t\t/* See if it is really trivial. */\n> \t\t\tgit_committer_info(IDENT_STRICT);\n> \t\t\tprintf(_(\"Trying really trivial in-index merge...\\n\"));\n> diff --git a/git-merge-resolve.sh b/git-merge-resolve.sh\n> index 343fe7bccd0..77e93121bf8 100755\n> --- a/git-merge-resolve.sh\n> +++ b/git-merge-resolve.sh\n> @@ -5,6 +5,16 @@\n> #\n> # Resolve two trees, using enhanced multi-base read-tree.\n> \n> +. git-sh-setup\n> +\n> +# Abort if index does not match HEAD\n> +if ! git diff-index --quiet --cached HEAD --\n> +then\n> + gettextln \"Error: Your local changes to the following files would be overwritten by merge\"\n> + git diff-index --cached --name-only HEAD -- | sed -e 's/^/ /'\n> + exit 2\n> +fi\n\n(The \"...\" continued below)\n\nJust in trying to poke holes in this I made this an \"exit 0\", and\nneither of the tests you added failed, but the last one (\"resolve &&\nrecursive && ort\") in the t6424*.sh will fail, is that intentional?\n\nI don't know enough about the context here, but given our *.sh->C\nmigration elsewhere it's a bit unfortunate to see more *.sh code added\nback. We have \"git merge\" driving this, isn't it OK to have it make this\ncheck before invoking \"resolve\" (may be a stupid question).\n\nFor this code in particular it:\n\n * Uses spaces, not tabs\n * We lose the diff-index .. --name-only exit code (segfault), but so\n did the older version\n * I wonder if bending over backwards to emit the exact message we\n emitted before is worth it\n\nIf you just make this something like (untested):\n\n\t{\n\t\tgettext \"error: \" &&\n\t\tgettextln \"Your local...\"\n\t}\n\nYou could re-use the translation from the *.c one (and the \"error: \" one\nwe'll get from usage.c).\n\nThat leaves \"\\n %s\" as the difference, but we could just remove that\nfrom the _() and emit it unconditionally, no?\n\n\n> # The first parameters up to -- are merge bases; the rest are heads.\n> bases= head= remotes= sep_seen=\n> for arg\n> diff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\n> index b6e424a427b..f35d3182b86 100755\n> --- a/t/t6424-merge-unrelated-index-changes.sh\n> +++ b/t/t6424-merge-unrelated-index-changes.sh\n> @@ -114,6 +114,32 @@ test_expect_success 'resolve, non-trivial' '\n> \ttest_path_is_missing .git/MERGE_HEAD\n> '\n> \n> +test_expect_success 'resolve, trivial, related file removed' '\n> +\tgit reset --hard &&\n> +\tgit checkout B^0 &&\n> +\n> +\tgit rm a &&\n> +\ttest_path_is_missing a &&\n> +\n> +\ttest_must_fail git merge -s resolve C^0 &&\n> +\n> +\ttest_path_is_missing a &&\n> +\ttest_path_is_missing .git/MERGE_HEAD\n> +'\n> +\n> +test_expect_success 'resolve, non-trivial, related file removed' '\n> +\tgit reset --hard &&\n> +\tgit checkout B^0 &&\n> +\n> +\tgit rm a &&\n> +\ttest_path_is_missing a &&\n> +\n> +\ttest_must_fail git merge -s resolve D^0 &&\n> +\n> +\ttest_path_is_missing a &&\n> +\ttest_path_is_missing .git/MERGE_HEAD\n> +'\n> +\n> test_expect_success 'recursive' '\n> \tgit reset --hard &&\n> \tgit checkout B^0 &&\n\n...I tried with this change on top, it seems to me like you'd want this\nin any case, it passes the tests both with & without the C code change,\nso can't we just use error() here?\n\t\n\tdiff --git a/builtin/merge.c b/builtin/merge.c\n\tindex 7fb4414ebb7..64def49734a 100644\n\t--- a/builtin/merge.c\n\t+++ b/builtin/merge.c\n\t@@ -1621,13 +1621,8 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n\t \n\t \t\t\tif (repo_index_has_changes(the_repository, head_tree,\n\t \t\t\t\t\t\t &sb)) {\n\t-\t\t\t\tstruct strbuf err = STRBUF_INIT;\n\t-\t\t\t\tstrbuf_addstr(&err, \"error: \");\n\t-\t\t\t\tstrbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n\t-\t\t\t\t\t sb.buf);\n\t-\t\t\t\tstrbuf_addch(&err, '\\n');\n\t-\t\t\t\tfputs(err.buf, stderr);\n\t-\t\t\t\tstrbuf_release(&err);\n\t+\t\t\t\terror(_(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n\t+\t\t\t\t sb.buf);\n\t \t\t\t\tstrbuf_release(&sb);\n\t \t\t\t\treturn -1;\n\t \t\t\t}\n\tdiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\n\tindex c96649448fa..1df130b9ee6 100755\n\t--- a/t/t6424-merge-unrelated-index-changes.sh\n\t+++ b/t/t6424-merge-unrelated-index-changes.sh\n\t@@ -96,7 +96,12 @@ test_expect_success 'resolve, trivial' '\n\t \n\t \ttouch random_file && git add random_file &&\n\t \n\t-\ttest_must_fail git merge -s resolve C^0 &&\n\t+\tsed -e \"s/^> //g\" >expect <<-\\EOF &&\n\t+\t> error: Your local changes to the following files would be overwritten by merge:\n\t+\t> random_file\n\t+\tEOF\n\t+\ttest_must_fail git merge -s resolve C^0 2>actual &&\n\t+\ttest_cmp expect actual &&\n\t \ttest_path_is_file random_file &&\n\t \tgit rev-parse --verify :random_file &&\n\t \ttest_path_is_missing .git/MERGE_HEAD\n\t\n"},{"id":"459764","messageId":"220722.86o7xhs9qg.gmgdl@evledraar.gmail.com","threadId":"57898","inReplyTo":"02930448ea1fbf7084b9d78813908b6355304457.1658466942.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 3/7] merge: do not abort early if one strategy fails to handle the merge","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-22T10:47:54Z","receivedAt":"2022-07-22T10:49:37Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jul 22 2022, Elijah Newren via GitGitGadget wrote:\n\n> From: Elijah Newren \n>\n> builtin/merge is setup to allow multiple strategies to be specified,\n> and it will find the \"best\" result and use it. This is defeated if\n> some of the merge strategies abort early when they cannot handle the\n> merge. Fix the logic that calls recursive and ort to not do such an\n> early abort, but instead return \"2\" or \"unhandled\" so that the next\n> strategy can try to handle the merge.\n>\n> Coming up with a testcase for this is somewhat difficult, since\n> recursive and ort both handle nearly any two-headed merge (there is\n> a separate code path that checks for non-two-headed merges and\n> already returns \"2\" for them). So use a somewhat synthetic testcase\n> of having the index not match HEAD before the merge starts, since all\n> merge strategies will abort for that.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 6 ++++--\n> t/t6402-merge-rename.sh | 2 +-\n> t/t6424-merge-unrelated-index-changes.sh | 16 ++++++++++++++++\n> t/t6439-merge-co-error-msgs.sh | 1 +\n> 4 files changed, 22 insertions(+), 3 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 13884b8e836..dec7375bf2a 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -754,8 +754,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n> \t\telse\n> \t\t\tclean = merge_recursive(&o, head, remoteheads->item,\n> \t\t\t\t\t\treversed, &result);\n> -\t\tif (clean < 0)\n> -\t\t\texit(128);\n> +\t\tif (clean < 0) {\n> +\t\t\trollback_lock_file(&lock);\n> +\t\t\treturn 2;\n> +\t\t}\n> \t\tif (write_locked_index(&the_index, &lock,\n> \t\t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n> \t\t\tdie(_(\"unable to write %s\"), get_index_file());\n> diff --git a/t/t6402-merge-rename.sh b/t/t6402-merge-rename.sh\n> index 3a32b1a45cf..772238e582c 100755\n> --- a/t/t6402-merge-rename.sh\n> +++ b/t/t6402-merge-rename.sh\n> @@ -210,7 +210,7 @@ test_expect_success 'updated working tree file should prevent the merge' '\n> \techo >>M one line addition &&\n> \tcat M >M.saved &&\n> \tgit update-index M &&\n> -\ttest_expect_code 128 git pull --no-rebase . yellow &&\n> +\ttest_expect_code 2 git pull --no-rebase . yellow &&\n> \ttest_cmp M M.saved &&\n> \trm -f M.saved\n> '\n> diff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\n> index f35d3182b86..8b749e19083 100755\n> --- a/t/t6424-merge-unrelated-index-changes.sh\n> +++ b/t/t6424-merge-unrelated-index-changes.sh\n> @@ -268,4 +268,20 @@ test_expect_success 'subtree' '\n> \ttest_path_is_missing .git/MERGE_HEAD\n> '\n> \n> +test_expect_success 'resolve && recursive && ort' '\n> +\tgit reset --hard &&\n> +\tgit checkout B^0 &&\n> +\n> +\ttest_seq 0 10 >a &&\n> +\tgit add a &&\n> +\n> +\tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n> +\ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n> +\n> +\tgrep \"Trying merge strategy resolve...\" output &&\n> +\tgrep \"Trying merge strategy recursive...\" output &&\n> +\tgrep \"Trying merge strategy ort...\" output &&\n> +\tgrep \"No merge strategy handled the merge.\" output\n> +'\n\nAh, re my feedback on 2/7 I hadn't read ahead. This is the test I\nmentioned as failing with the code added in 2/7 if it's tweaked to be\ns/exit 2/exit 0/.\n\nSo it's a bit odd to have code added in 2/7 that's tested in 3/7. I\nthink this would be much easier to understand if these tests came before\nall these code changes, so then as the changes are made we can see how\nthe behavior changes.\n\nBut short of that at least having the relevant part of this for 2/7 in\nthat commit would be better, i.e. the thing that tests that new\n\"diff-index\" check in some way...\n\n"},{"id":"459765","messageId":"220722.86k085s9g3.gmgdl@evledraar.gmail.com","threadId":"57898","inReplyTo":"f401bd5ad0dd7564412e72d19a4193ad3f64e638.1658466942.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 5/7] merge: make restore_state() restore staged state too","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-22T10:53:10Z","receivedAt":"2022-07-22T10:55:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jul 22 2022, Elijah Newren via GitGitGadget wrote:\n\n> From: Elijah Newren \n>\n> There are multiple issues at play here:\n>\n> 1) If `git merge` is invoked with staged changes, it should abort\n> without doing any merging, and the user's working tree and index\n> should be the same as before merge was invoked.\n> 2) Merge strategies are responsible for enforcing the index == HEAD\n> requirement. (See 9822175d2b (\"Ensure index matches head before\n> invoking merge machinery, round N\", 2019-08-17) for some history\n> around this.)\n> 3) Merge strategies can bail saying they are not an appropriate\n> handler for the merge in question (possibly allowing other\n> strategies to be used instead).\n> 4) Merge strategies can make changes to the index and working tree,\n> and have no expectation to clean up after themselves, *even* if\n> they bail out and say they are not an appropriate handler for\n> the merge in question. (The `octopus` merge strategy does this,\n> for example.)\n> 5) Because of (3) and (4), builtin/merge.c stashes state before\n> trying merge strategies and restores it afterward.\n>\n> Unfortunately, if users had staged changes before calling `git merge`,\n> builtin/merge.c could do the following:\n>\n> * stash the changes, in order to clean up after the strategies\n> * try all the merge strategies in turn, each of which report they\n> cannot function due to the index not matching HEAD\n> * restore the changes via \"git stash apply\"\n>\n> But that last step would have the net effect of unstaging the user's\n> changes. Fix this by adding the \"--index\" option to \"git stash apply\".\n> While at it, also squelch the stash apply output; we already report\n> \"Rewinding the tree to pristine...\" and don't need a detailed `git\n> status` report afterwards.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 5 +++--\n> t/t6424-merge-unrelated-index-changes.sh | 7 ++++++-\n> 2 files changed, 9 insertions(+), 3 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 4170c30317e..f807bf335bd 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -383,14 +383,15 @@ static void reset_hard(const struct object_id *oid, int verbose)\n> static void restore_state(const struct object_id *head,\n> \t\t\t const struct object_id *stash)\n> {\n> -\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n> +\tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n> +\t\t\t NULL, NULL };\n> \n> \tif (is_null_oid(stash))\n> \t\treturn;\n> \n> \treset_hard(head, 1);\n> \n> -\targs[2] = oid_to_hex(stash);\n> +\targs[4] = oid_to_hex(stash);\n> \n> \t/*\n> \t * It is OK to ignore error here, for example when there was\n\nJust a nit/side comment: This is one of these older-style arg\nconstructions that we've replaced with strvec in most other places.\n\nLet's leave this alone for now (especially in a v4), but FWIW I wouldn't\nmind if these sort of changes were strvec converted while at it:\n\t\n\tdiff --git a/builtin/merge.c b/builtin/merge.c\n\tindex 64def49734a..c3a3a1fde50 100644\n\t--- a/builtin/merge.c\n\t+++ b/builtin/merge.c\n\t@@ -383,21 +383,23 @@ static void reset_hard(const struct object_id *oid, int verbose)\n\t static void restore_state(const struct object_id *head,\n\t \t\t\t const struct object_id *stash)\n\t {\n\t-\tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n\t-\t\t\t NULL, NULL };\n\t+\tstruct strvec args = STRVEC_INIT;\n\t+\n\t+\tstrvec_pushl(&args, \"stash\", \"apply\", \"--index\", \"--quiet\", NULL);\n\t \n\t \treset_hard(head, 1);\n\t \n\t \tif (is_null_oid(stash))\n\t \t\tgoto refresh_cache;\n\t \n\t-\targs[4] = oid_to_hex(stash);\n\t+\tstrvec_push(&args, oid_to_hex(stash));\n\t \n\t \t/*\n\t \t * It is OK to ignore error here, for example when there was\n\t \t * nothing to restore.\n\t \t */\n\t-\trun_command_v_opt(args, RUN_GIT_CMD);\n\t+\trun_command_v_opt(args.v, RUN_GIT_CMD);\n\t+\tstrvec_clear(&args);\n\t \n\t refresh_cache:\n\t \tif (discard_cache() < 0 || read_cache() < 0)\n\nI.e. it takes about as much mental energy to review that as counting the\nargs elements and seeing that 2 to 4 is correct :)\n"},{"id":"459807","messageId":"CABPp-BHToEDv=8W9nf4XDC8b+5=EVuc=OnoT6xCRGWxB9CKWfw@mail.gmail.com","threadId":"57898","inReplyTo":"220722.86sfmts9vr.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v4 2/7] merge-resolve: abort if index does not match HEAD","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-23T00:28:17Z","receivedAt":"2022-07-23T00:28:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Jul 22, 2022 at 3:46 AM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> On Fri, Jul 22 2022, Elijah Newren via GitGitGadget wrote:\n>\n> > From: Elijah Newren \n> >\n> > As noted in commit 9822175d2b (\"Ensure index matches head before\n> > invoking merge machinery, round N\", 2019-08-17), we have had a very\n> > long history of problems with failing to enforce the requirement that\n> > index matches HEAD when starting a merge. One of the commits\n> > referenced in the long tale of issues arising from lax enforcement of\n> > this requirement was commit 55f39cf755 (\"merge: fix misleading\n> > pre-merge check documentation\", 2018-06-30), which tried to document\n> > the requirement and noted there were some exceptions. As mentioned in\n> > that commit message, the `resolve` strategy was the one strategy that\n> > did not have an explicit index matching HEAD check, and the reason it\n> > didn't was that I wasn't able to discover any cases where the\n> > implementation would fail to catch the problem and abort, and didn't\n> > want to introduce unnecessary performance overhead of adding another\n> > check.\n> >\n> > Well, today I discovered a testcase where the implementation does not\n> > catch the problem and so an explicit check is needed. Add a testcase\n> > that previously would have failed, and update git-merge-resolve.sh to\n> > have an explicit check. Note that the code is copied from 3ec62ad9ff\n> > (\"merge-octopus: abort if index does not match HEAD\", 2016-04-09), so\n> > that we reuse the same message and avoid making translators need to\n> > translate some new message.\n> >\n> > Signed-off-by: Elijah Newren \n> > ---\n> > builtin/merge.c | 20 ++++++++++++++++++\n> > git-merge-resolve.sh | 10 +++++++++\n> > t/t6424-merge-unrelated-index-changes.sh | 26 ++++++++++++++++++++++++\n> > 3 files changed, 56 insertions(+)\n> >\n> > diff --git a/builtin/merge.c b/builtin/merge.c\n> > index 23170f2d2a6..13884b8e836 100644\n> > --- a/builtin/merge.c\n> > +++ b/builtin/merge.c\n> > @@ -1599,6 +1599,26 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> > */\n> > refresh_cache(REFRESH_QUIET);\n> > if (allow_trivial && fast_forward != FF_ONLY) {\n> > + /*\n> > + * Must first ensure that index matches HEAD before\n> > + * attempting a trivial merge.\n> > + */\n> > + struct tree *head_tree = get_commit_tree(head_commit);\n> > + struct strbuf sb = STRBUF_INIT;\n> > +\n> > + if (repo_index_has_changes(the_repository, head_tree,\n> > + &sb)) {\n> > + struct strbuf err = STRBUF_INIT;\n> > + strbuf_addstr(&err, \"error: \");\n> > + strbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> > + sb.buf);\n> > + strbuf_addch(&err, '\\n');\n>\n> At first glance I was expecting this to construct an error message to\n> emit it somewhere else that stderr, so I wondered if you couldn't use\n> the \"error_routine\" facility to avoid re-inventing \"error: \" etc.,\n> but...\n>\n> > + fputs(err.buf, stderr);\n>\n> ...we emit it to stderr anyway...?\n>\n> > + strbuf_release(&err);\n> > + strbuf_release(&sb);\n> > + return -1;\n> > + }\n> > +\n\nUGH! I fixed the other one of these in my reroll yesterday[1]. I\n_knew_ I had copied that code somewhere else, but for some reason I\nthought it was in a different series and went searching for it. Don't\nknow why I couldn't remember that it was in the same series, and I'm\nnot sure how I missed it when I went looking. I mean, I know I was\ntired yesterday, but that's still kinda bad.\n\nAnyway, thanks for catching; I'll fix this one too.\n\n[1] https://lore.kernel.org/git/xmqqsfmulb6w.fsf@gitster.g/\n\n> > /* See if it is really trivial. */\n> > git_committer_info(IDENT_STRICT);\n> > printf(_(\"Trying really trivial in-index merge...\\n\"));\n> > diff --git a/git-merge-resolve.sh b/git-merge-resolve.sh\n> > index 343fe7bccd0..77e93121bf8 100755\n> > --- a/git-merge-resolve.sh\n> > +++ b/git-merge-resolve.sh\n> > @@ -5,6 +5,16 @@\n> > #\n> > # Resolve two trees, using enhanced multi-base read-tree.\n> >\n> > +. git-sh-setup\n> > +\n> > +# Abort if index does not match HEAD\n> > +if ! git diff-index --quiet --cached HEAD --\n> > +then\n> > + gettextln \"Error: Your local changes to the following files would be overwritten by merge\"\n> > + git diff-index --cached --name-only HEAD -- | sed -e 's/^/ /'\n> > + exit 2\n> > +fi\n>\n> (The \"...\" continued below)\n>\n> Just in trying to poke holes in this I made this an \"exit 0\", and\n> neither of the tests you added failed, but the last one (\"resolve &&\n> recursive && ort\") in the t6424*.sh will fail, is that intentional?\n\nNope it's not intentional. I had tested one fix (for these\ngit-merge-resolve.sh changes) and verified they were good (and\nnecessary), then found another bug (the one fixed by the\nbuiltin/merge.c changes) and added a test for it, and just decided to\namend it into the same commit. Turns out the builtin/merge.c changes\nmask the fix for git-merge-resolve.sh here and makes that code go\nunexercised. I'll split these two bugfixes into separate patches, and\ntweak one of the two testcases to make sure it continues to exercise\nthe new codepath added to git-merge-resolve.sh.\n\n> I don't know enough about the context here, but given our *.sh->C\n> migration elsewhere it's a bit unfortunate to see more *.sh code added\n> back.\n\nThis seems like a curious objection. \"We are trying to get rid of\nshell scripts, so don't even fix bugs in any of the existing ones.\" ?\n\n> We have \"git merge\" driving this, isn't it OK to have it make this\n> check before invoking \"resolve\" (may be a stupid question).\n\nAh, I can kind of see where you're coming from now, but that seems to\nme to be bending over backwards in attempting to fix a component\nwritten in shell without actually modifying the shell.\nbuiltin/merge.c is some glue code that can call multiple different\nstrategies, but isn't the place for the implementation of the\nstrategies themselves, and I'd hate to see us put half the\nimplementation in one place and half in another. In addition, besides\nthe separation of concerns issue::\n\n * We document that users can add their own merge strategies (a\nshell or executable named git-merge-$USERNAME and \"git merge -s\n$USERNAME\" will call them)\n * git-merge-resolve and git-merge-octopus serve as examples\n * Our examples should demonstrate correct behavior and perform\ndocumented, required steps. This particular check is important:\n\n /*\n * At this point, we need a real merge. No matter what strategy\n * we use, it would operate on the index, possibly affecting the\n * working tree, and when resolved cleanly, have the desired\n * tree in the index -- this means that the index must be in\n * sync with the head commit. The strategies are responsible\n * to ensure this.\n */\n\nSo, even if someone were to reimplement git-merge-resolve.sh in C, and\nstart the deprecation process with some merge.useBuiltinResolve config\nsetting (similar to rebase.useBuiltin), I'd still want this shell fix\nadded to git-merge-resolve.sh in the meantime, both as an important\nbugfix, and so that people looking for merge strategy examples who\nfind this script hopefully find a new enough version with this\nimportant check included.\n\nIn general, if merge strategies do not perform this check, we have\nobserved that they often will either (a) discard users' staged changes\n(best case) or (b) smash staged changes into the created commit and\nthus create some kind of evil merge (making it look like they created\na merge normally, and then amended the merge with additional changes).\n\nWe're lucky that the way resolve was implemented, other git calls\nwould usually incidentally catch such issues for us without an\nexplicit check. We were also lucky that the observed behavior was\n'(a)' rather than '(b)' for resolve. But the issue should still be\nfixed.\n\n> For this code in particular it:\n>\n> * Uses spaces, not tabs\n\nYes, that's fair, but as I mentioned in the commit message, it was\ncopied from git-merge-octopus.sh. So, as you say below, \"so did the\nolder version\".\n\n> * We lose the diff-index .. --name-only exit code (segfault), but so\n> did the older version\n\nUm, I don't understand this objection. I think you are referring to\nthe pipe to sed, but if so...who cares? The exit code would be lost\nanyway because we aren't running under errexit, and the next line of\ncode ignores any and all previous exit codes when it runs \"exit 2\".\nAnd if you're not referring to the pipe to sed but the fact that it\nunconditionally returns an exit code of 2 on the next line, then yes\nthat is the expected return code. Whatever the diff-index segfault\nreturns would be the wrong exit status and could fool the\nbuiltin/merge.c into doing the wrong thing. It expects merge\nstrategies to return one of three exit codes: 0, 1, or 2:\n\n /*\n * The backend exits with 1 when conflicts are\n * left to be resolved, with 2 when it does not\n * handle the given merge at all.\n */\n\nSo, ignoring the return code from diff-index is correct behavior here.\n\nWere you thinking this was a test script or something?\n\n> * I wonder if bending over backwards to emit the exact message we\n> emitted before is worth it\n>\n> If you just make this something like (untested):\n>\n> {\n> gettext \"error: \" &&\n> gettextln \"Your local...\"\n> }\n>\n> You could re-use the translation from the *.c one (and the \"error: \" one\n> we'll get from usage.c).\n>\n> That leaves \"\\n %s\" as the difference, but we could just remove that\n> from the _() and emit it unconditionally, no?\n\n??\n\nCopying a few lines from git-merge-octopus.sh to get the same fix it\nhas is \"bending over backwards\"? That's what I call \"doing the\neasiest thing possible\" (and which _also_ has the benefit of being\nbattle tested code), and then you describe a bunch of gymnastics as an\nalternative? I see your suggestion as running afoul of the objection\nyou are raising, and the code I'm adding as being a solution to that\nparticular objection. So this particular flag you are raising is\nconfusing to me.\n\n> > # The first parameters up to -- are merge bases; the rest are heads.\n> > bases= head= remotes= sep_seen=\n> > for arg\n> > diff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\n> > index b6e424a427b..f35d3182b86 100755\n> > --- a/t/t6424-merge-unrelated-index-changes.sh\n> > +++ b/t/t6424-merge-unrelated-index-changes.sh\n> > @@ -114,6 +114,32 @@ test_expect_success 'resolve, non-trivial' '\n> > test_path_is_missing .git/MERGE_HEAD\n> > '\n> >\n> > +test_expect_success 'resolve, trivial, related file removed' '\n> > + git reset --hard &&\n> > + git checkout B^0 &&\n> > +\n> > + git rm a &&\n> > + test_path_is_missing a &&\n> > +\n> > + test_must_fail git merge -s resolve C^0 &&\n> > +\n> > + test_path_is_missing a &&\n> > + test_path_is_missing .git/MERGE_HEAD\n> > +'\n> > +\n> > +test_expect_success 'resolve, non-trivial, related file removed' '\n> > + git reset --hard &&\n> > + git checkout B^0 &&\n> > +\n> > + git rm a &&\n> > + test_path_is_missing a &&\n> > +\n> > + test_must_fail git merge -s resolve D^0 &&\n> > +\n> > + test_path_is_missing a &&\n> > + test_path_is_missing .git/MERGE_HEAD\n> > +'\n> > +\n> > test_expect_success 'recursive' '\n> > git reset --hard &&\n> > git checkout B^0 &&\n>\n> ...I tried with this change on top, it seems to me like you'd want this\n> in any case, it passes the tests both with & without the C code change,\n> so can't we just use error() here?\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 7fb4414ebb7..64def49734a 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -1621,13 +1621,8 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n>\n> if (repo_index_has_changes(the_repository, head_tree,\n> &sb)) {\n> - struct strbuf err = STRBUF_INIT;\n> - strbuf_addstr(&err, \"error: \");\n> - strbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> - sb.buf);\n> - strbuf_addch(&err, '\\n');\n> - fputs(err.buf, stderr);\n> - strbuf_release(&err);\n> + error(_(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n> + sb.buf);\n> strbuf_release(&sb);\n> return -1;\n\nYes, this is the same change suggested by Junio for patch 1 which I\nshould have also applied here. Thanks for catching it!\n"},{"id":"459808","messageId":"CABPp-BFeo2nxH38D1Cvd6RLRRt8PLOH+BLsoikchWv81rbvbFg@mail.gmail.com","threadId":"57898","inReplyTo":"220722.86o7xhs9qg.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v4 3/7] merge: do not abort early if one strategy fails to handle the merge","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-23T00:36:30Z","receivedAt":"2022-07-23T00:36:45Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Jul 22, 2022 at 3:49 AM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> On Fri, Jul 22 2022, Elijah Newren via GitGitGadget wrote:\n>\n> > From: Elijah Newren \n> >\n> > builtin/merge is setup to allow multiple strategies to be specified,\n> > and it will find the \"best\" result and use it. This is defeated if\n> > some of the merge strategies abort early when they cannot handle the\n> > merge. Fix the logic that calls recursive and ort to not do such an\n> > early abort, but instead return \"2\" or \"unhandled\" so that the next\n> > strategy can try to handle the merge.\n> >\n> > Coming up with a testcase for this is somewhat difficult, since\n> > recursive and ort both handle nearly any two-headed merge (there is\n> > a separate code path that checks for non-two-headed merges and\n> > already returns \"2\" for them). So use a somewhat synthetic testcase\n> > of having the index not match HEAD before the merge starts, since all\n> > merge strategies will abort for that.\n> >\n> > Signed-off-by: Elijah Newren \n> > ---\n> > builtin/merge.c | 6 ++++--\n> > t/t6402-merge-rename.sh | 2 +-\n> > t/t6424-merge-unrelated-index-changes.sh | 16 ++++++++++++++++\n> > t/t6439-merge-co-error-msgs.sh | 1 +\n> > 4 files changed, 22 insertions(+), 3 deletions(-)\n> >\n> > diff --git a/builtin/merge.c b/builtin/merge.c\n> > index 13884b8e836..dec7375bf2a 100644\n> > --- a/builtin/merge.c\n> > +++ b/builtin/merge.c\n> > @@ -754,8 +754,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n> > else\n> > clean = merge_recursive(&o, head, remoteheads->item,\n> > reversed, &result);\n> > - if (clean < 0)\n> > - exit(128);\n> > + if (clean < 0) {\n> > + rollback_lock_file(&lock);\n> > + return 2;\n> > + }\n> > if (write_locked_index(&the_index, &lock,\n> > COMMIT_LOCK | SKIP_IF_UNCHANGED))\n> > die(_(\"unable to write %s\"), get_index_file());\n> > diff --git a/t/t6402-merge-rename.sh b/t/t6402-merge-rename.sh\n> > index 3a32b1a45cf..772238e582c 100755\n> > --- a/t/t6402-merge-rename.sh\n> > +++ b/t/t6402-merge-rename.sh\n> > @@ -210,7 +210,7 @@ test_expect_success 'updated working tree file should prevent the merge' '\n> > echo >>M one line addition &&\n> > cat M >M.saved &&\n> > git update-index M &&\n> > - test_expect_code 128 git pull --no-rebase . yellow &&\n> > + test_expect_code 2 git pull --no-rebase . yellow &&\n> > test_cmp M M.saved &&\n> > rm -f M.saved\n> > '\n> > diff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\n> > index f35d3182b86..8b749e19083 100755\n> > --- a/t/t6424-merge-unrelated-index-changes.sh\n> > +++ b/t/t6424-merge-unrelated-index-changes.sh\n> > @@ -268,4 +268,20 @@ test_expect_success 'subtree' '\n> > test_path_is_missing .git/MERGE_HEAD\n> > '\n> >\n> > +test_expect_success 'resolve && recursive && ort' '\n> > + git reset --hard &&\n> > + git checkout B^0 &&\n> > +\n> > + test_seq 0 10 >a &&\n> > + git add a &&\n> > +\n> > + sane_unset GIT_TEST_MERGE_ALGORITHM &&\n> > + test_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n> > +\n> > + grep \"Trying merge strategy resolve...\" output &&\n> > + grep \"Trying merge strategy recursive...\" output &&\n> > + grep \"Trying merge strategy ort...\" output &&\n> > + grep \"No merge strategy handled the merge.\" output\n> > +'\n\nOops, 'resolve' should really be at the end of the list rather than at\nthe beginning. And the test description should be better.\n\n> Ah, re my feedback on 2/7 I hadn't read ahead. This is the test I\n> mentioned as failing with the code added in 2/7 if it's tweaked to be\n> s/exit 2/exit 0/.\n>\n> So it's a bit odd to have code added in 2/7 that's tested in 3/7. I\n> think this would be much easier to understand if these tests came before\n> all these code changes, so then as the changes are made we can see how\n> the behavior changes.\n\nThis testcase belongs in this patch. The use of \"resolve\" here was\ntotally incidental to the testcase in question; I could have used\n\"octopus\" or \"ours\" or created a new strategy and used it.\n\n(Actually, using 'ours' here runs into the problem we fix in the final\npatch. So maybe just like 'resolve', using 'ours' might be confusing\nto readers of the series as they think issues from other patches are\ninvolved.)\n\n> But short of that at least having the relevant part of this for 2/7 in\n> that commit would be better, i.e. the thing that tests that new\n> \"diff-index\" check in some way...\n\nI'll switch this test to using 'octopus' instead of 'resolve' just so\nit doesn't get confused in this way.\n"},{"id":"459810","messageId":"b656756fd37405114eec67d642ab8a02a9b90bbb.1658541198.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"[PATCH v5 2/8] merge-resolve: abort if index does not match HEAD","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:12Z","receivedAt":"2022-07-23T01:53:30Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nAs noted in commit 9822175d2b (\"Ensure index matches head before\ninvoking merge machinery, round N\", 2019-08-17), we have had a very\nlong history of problems with failing to enforce the requirement that\nindex matches HEAD when starting a merge. One of the commits\nreferenced in the long tale of issues arising from lax enforcement of\nthis requirement was commit 55f39cf755 (\"merge: fix misleading\npre-merge check documentation\", 2018-06-30), which tried to document\nthe requirement and noted there were some exceptions. As mentioned in\nthat commit message, the `resolve` strategy was the one strategy that\ndid not have an explicit index matching HEAD check, and the reason it\ndidn't was that I wasn't able to discover any cases where the\nimplementation would fail to catch the problem and abort, and didn't\nwant to introduce unnecessary performance overhead of adding another\ncheck.\n\nWell, today I discovered a testcase where the implementation does not\ncatch the problem and so an explicit check is needed. Add a testcase\nthat previously would have failed, and update git-merge-resolve.sh to\nhave an explicit check. Note that the code is copied from 3ec62ad9ff\n(\"merge-octopus: abort if index does not match HEAD\", 2016-04-09), so\nthat we reuse the same message and avoid making translators need to\ntranslate some new message.\n\nSigned-off-by: Elijah Newren \n---\n git-merge-resolve.sh | 10 ++++++++++\n t/t6424-merge-unrelated-index-changes.sh | 13 +++++++++++++\n 2 files changed, 23 insertions(+)\n\ndiff --git a/git-merge-resolve.sh b/git-merge-resolve.sh\nindex 343fe7bccd0..77e93121bf8 100755\n--- a/git-merge-resolve.sh\n+++ b/git-merge-resolve.sh\n@@ -5,6 +5,16 @@\n #\n # Resolve two trees, using enhanced multi-base read-tree.\n \n+. git-sh-setup\n+\n+# Abort if index does not match HEAD\n+if ! git diff-index --quiet --cached HEAD --\n+then\n+ gettextln \"Error: Your local changes to the following files would be overwritten by merge\"\n+ git diff-index --cached --name-only HEAD -- | sed -e 's/^/ /'\n+ exit 2\n+fi\n+\n # The first parameters up to -- are merge bases; the rest are heads.\n bases= head= remotes= sep_seen=\n for arg\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex b6e424a427b..eabe6bda832 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -114,6 +114,19 @@ test_expect_success 'resolve, non-trivial' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'resolve, non-trivial, related file removed' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\tgit rm a &&\n+\ttest_path_is_missing a &&\n+\n+\ttest_must_fail git merge -s resolve D^0 &&\n+\n+\ttest_path_is_missing a &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n test_expect_success 'recursive' '\n \tgit reset --hard &&\n \tgit checkout B^0 &&\n-- \ngitgitgadget\n\n"},{"id":"459811","messageId":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v4.git.1658466942.gitgitgadget@gmail.com","subject":"[PATCH v5 0/8] Fix merge restore state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:10Z","receivedAt":"2022-07-23T01:53:30Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This started as a simple series to fix restore_state() in builtin/merge.c,\nfixing an issue reported by ZheNing Hu[3]. It now fixes several bugs and has\ngrown so much it's hard to call it simple. Anyway...\n\nChanges since v4:\n\n * Made use of the error() function in another place to simplify code\n (should have caught this in v3)\n * Split the fixes for 'resolve' and the trivial merge into separate\n patches, and make sure one doesn't mask the other but both codepaths are\n exercises in the testsuite\n * better test descriptions\n * use strvec to simplify some code\n\n[1]\nhttps://lore.kernel.org/git/CAOLTT8R7QmpvaFPTRs3xTpxr7eiuxF-ZWtvUUSC0-JOo9Y+SqA@mail.gmail.com/\n\nElijah Newren (8):\n merge-ort-wrappers: make printed message match the one from recursive\n merge-resolve: abort if index does not match HEAD\n merge: abort if index does not match HEAD for trivial merges\n merge: do not abort early if one strategy fails to handle the merge\n merge: fix save_state() to work when there are stat-dirty files\n merge: make restore_state() restore staged state too\n merge: ensure we can actually restore pre-merge state\n merge: do not exit restore_state() prematurely\n\n builtin/merge.c | 57 ++++++++++++++++-----\n git-merge-resolve.sh | 10 ++++\n merge-ort-wrappers.c | 4 +-\n t/t6402-merge-rename.sh | 2 +-\n t/t6424-merge-unrelated-index-changes.sh | 65 ++++++++++++++++++++++++\n t/t6439-merge-co-error-msgs.sh | 1 +\n t/t7607-merge-state.sh | 32 ++++++++++++\n 7 files changed, 154 insertions(+), 17 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\n\nbase-commit: e72d93e88cb20b06e88e6e7d81bd1dc4effe453f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1231%2Fnewren%2Ffix-merge-restore-state-v5\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1231/newren/fix-merge-restore-state-v5\nPull-Request: https://github.com/gitgitgadget/git/pull/1231\n\nRange-diff vs v4:\n\n 1: bd36d16c8d9 = 1: bd36d16c8d9 merge-ort-wrappers: make printed message match the one from recursive\n 2: b79f44e54b9 ! 2: b656756fd37 merge-resolve: abort if index does not match HEAD\n @@ Commit message\n \n Signed-off-by: Elijah Newren \n \n - ## builtin/merge.c ##\n -@@ builtin/merge.c: int cmd_merge(int argc, const char **argv, const char *prefix)\n - \t\t */\n - \t\trefresh_cache(REFRESH_QUIET);\n - \t\tif (allow_trivial && fast_forward != FF_ONLY) {\n -+\t\t\t/*\n -+\t\t\t * Must first ensure that index matches HEAD before\n -+\t\t\t * attempting a trivial merge.\n -+\t\t\t */\n -+\t\t\tstruct tree *head_tree = get_commit_tree(head_commit);\n -+\t\t\tstruct strbuf sb = STRBUF_INIT;\n -+\n -+\t\t\tif (repo_index_has_changes(the_repository, head_tree,\n -+\t\t\t\t\t\t &sb)) {\n -+\t\t\t\tstruct strbuf err = STRBUF_INIT;\n -+\t\t\t\tstrbuf_addstr(&err, \"error: \");\n -+\t\t\t\tstrbuf_addf(&err, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n -+\t\t\t\t\t sb.buf);\n -+\t\t\t\tstrbuf_addch(&err, '\\n');\n -+\t\t\t\tfputs(err.buf, stderr);\n -+\t\t\t\tstrbuf_release(&err);\n -+\t\t\t\tstrbuf_release(&sb);\n -+\t\t\t\treturn -1;\n -+\t\t\t}\n -+\n - \t\t\t/* See if it is really trivial. */\n - \t\t\tgit_committer_info(IDENT_STRICT);\n - \t\t\tprintf(_(\"Trying really trivial in-index merge...\\n\"));\n -\n ## git-merge-resolve.sh ##\n @@\n #\n @@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'resolve, non-triv\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n -+test_expect_success 'resolve, trivial, related file removed' '\n -+\tgit reset --hard &&\n -+\tgit checkout B^0 &&\n -+\n -+\tgit rm a &&\n -+\ttest_path_is_missing a &&\n -+\n -+\ttest_must_fail git merge -s resolve C^0 &&\n -+\n -+\ttest_path_is_missing a &&\n -+\ttest_path_is_missing .git/MERGE_HEAD\n -+'\n -+\n +test_expect_success 'resolve, non-trivial, related file removed' '\n +\tgit reset --hard &&\n +\tgit checkout B^0 &&\n -: ----------- > 3: 3adfd921995 merge: abort if index does not match HEAD for trivial merges\n 3: 02930448ea1 ! 4: c5755271cf1 merge: do not abort early if one strategy fails to handle the merge\n @@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'subtree' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n -+test_expect_success 'resolve && recursive && ort' '\n ++test_expect_success 'with multiple strategies, recursive or ort failure do not early abort' '\n +\tgit reset --hard &&\n +\tgit checkout B^0 &&\n +\n @@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'subtree' '\n +\tgit add a &&\n +\n +\tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n -+\ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n ++\ttest_must_fail git merge -s recursive -s ort -s octopus C^0 >output 2>&1 &&\n +\n -+\tgrep \"Trying merge strategy resolve...\" output &&\n +\tgrep \"Trying merge strategy recursive...\" output &&\n +\tgrep \"Trying merge strategy ort...\" output &&\n ++\tgrep \"Trying merge strategy octopus...\" output &&\n +\tgrep \"No merge strategy handled the merge.\" output\n +'\n +\n 4: daf8d224160 ! 5: e7c6de9e0c1 merge: fix save_state() to work when there are stat-dirty files\n @@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'subtree' '\n +\tgit merge -s resolve -s recursive D^0\n +'\n +\n - test_expect_success 'resolve && recursive && ort' '\n + test_expect_success 'with multiple strategies, recursive or ort failure do not early abort' '\n \tgit reset --hard &&\n \tgit checkout B^0 &&\n 5: f401bd5ad0d ! 6: d39d6472455 merge: make restore_state() restore staged state too\n @@ Commit message\n changes. Fix this by adding the \"--index\" option to \"git stash apply\".\n While at it, also squelch the stash apply output; we already report\n \"Rewinding the tree to pristine...\" and don't need a detailed `git\n - status` report afterwards.\n + status` report afterwards. Also while at it, switch to using strvec\n + so folks don't have to count the arguments to ensure we avoided an\n + off-by-one error, and so it's easier to add additional arguments to\n + the command.\n \n Signed-off-by: Elijah Newren \n \n @@ builtin/merge.c: static void reset_hard(const struct object_id *oid, int verbose\n \t\t\t const struct object_id *stash)\n {\n -\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n -+\tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n -+\t\t\t NULL, NULL };\n ++\tstruct strvec args = STRVEC_INIT;\n \n \tif (is_null_oid(stash))\n \t\treturn;\n @@ builtin/merge.c: static void reset_hard(const struct object_id *oid, int verbose\n \treset_hard(head, 1);\n \n -\targs[2] = oid_to_hex(stash);\n -+\targs[4] = oid_to_hex(stash);\n ++\tstrvec_pushl(&args, \"stash\", \"apply\", \"--index\", \"--quiet\", NULL);\n ++\tstrvec_push(&args, oid_to_hex(stash));\n \n \t/*\n \t * It is OK to ignore error here, for example when there was\n + \t * nothing to restore.\n + \t */\n +-\trun_command_v_opt(args, RUN_GIT_CMD);\n ++\trun_command_v_opt(args.v, RUN_GIT_CMD);\n ++\tstrvec_clear(&args);\n + \n + \trefresh_cache(REFRESH_QUIET);\n + }\n \n ## t/t6424-merge-unrelated-index-changes.sh ##\n -@@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'resolve && recursive && ort' '\n +@@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'with multiple strategies, recursive or ort failure do not e\n \n \ttest_seq 0 10 >a &&\n \tgit add a &&\n +\tgit rev-parse :a >expect &&\n \n \tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n - \ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n -@@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'resolve && recursive && ort' '\n - \tgrep \"Trying merge strategy resolve...\" output &&\n + \ttest_must_fail git merge -s recursive -s ort -s octopus C^0 >output 2>&1 &&\n +@@ t/t6424-merge-unrelated-index-changes.sh: test_expect_success 'with multiple strategies, recursive or ort failure do not e\n \tgrep \"Trying merge strategy recursive...\" output &&\n \tgrep \"Trying merge strategy ort...\" output &&\n + \tgrep \"Trying merge strategy octopus...\" output &&\n -\tgrep \"No merge strategy handled the merge.\" output\n +\tgrep \"No merge strategy handled the merge.\" output &&\n +\n 6: ad5354c219c = 7: 7f5c6884d68 merge: ensure we can actually restore pre-merge state\n 7: 6212d572604 ! 8: 954dec526a2 merge: do not exit restore_state() prematurely\n @@ Commit message\n \n ## builtin/merge.c ##\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n - \tconst char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n - \t\t\t NULL, NULL };\n + {\n + \tstruct strvec args = STRVEC_INIT;\n \n -\tif (is_null_oid(stash))\n -\t\treturn;\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n +\tif (is_null_oid(stash))\n +\t\tgoto refresh_cache;\n +\n - \targs[4] = oid_to_hex(stash);\n + \tstrvec_pushl(&args, \"stash\", \"apply\", \"--index\", \"--quiet\", NULL);\n + \tstrvec_push(&args, oid_to_hex(stash));\n \n - \t/*\n @@ builtin/merge.c: static void restore_state(const struct object_id *head,\n - \t */\n - \trun_command_v_opt(args, RUN_GIT_CMD);\n + \trun_command_v_opt(args.v, RUN_GIT_CMD);\n + \tstrvec_clear(&args);\n \n -\trefresh_cache(REFRESH_QUIET);\n +refresh_cache:\n @@ t/t7607-merge-state.sh (new)\n +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n +. ./test-lib.sh\n +\n -+test_expect_success 'set up custom strategy' '\n ++test_expect_success 'Ensure we restore original state if no merge strategy handles it' '\n +\ttest_commit --no-tag \"Initial\" base base &&\n +\n +\tfor b in branch1 branch2 branch3\n\n-- \ngitgitgadget\n"},{"id":"459812","messageId":"bd36d16c8d93176bac12acaf90f654a0acb16cd6.1658541198.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"[PATCH v5 1/8] merge-ort-wrappers: make printed message match the one from recursive","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:11Z","receivedAt":"2022-07-23T01:53:30Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nWhen the index does not match HEAD, the merge strategies are responsible\nto detect that condition and abort. The merge-ort-wrappers had code to\nimplement this and meant to copy the error message from merge-recursive\nbut deviated in two ways, both due to the message in merge-recursive\nbeing processed by another function that made additional changes:\n * It added an implicit \"error: \" prefix\n * It added an implicit trailing newline\nWe can get these things by making use of the error() function.\n\nSigned-off-by: Elijah Newren \n---\n merge-ort-wrappers.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-ort-wrappers.c b/merge-ort-wrappers.c\nindex ad041061695..748924a69ba 100644\n--- a/merge-ort-wrappers.c\n+++ b/merge-ort-wrappers.c\n@@ -10,8 +10,8 @@ static int unclean(struct merge_options *opt, struct tree *head)\n \tstruct strbuf sb = STRBUF_INIT;\n \n \tif (head && repo_index_has_changes(opt->repo, head, &sb)) {\n-\t\tfprintf(stderr, _(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n-\t\t sb.buf);\n+\t\terror(_(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n+\t\t sb.buf);\n \t\tstrbuf_release(&sb);\n \t\treturn -1;\n \t}\n-- \ngitgitgadget\n\n"},{"id":"459813","messageId":"c5755271cf1e35c818647cfc75ce8f67c0b777ac.1658541198.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"[PATCH v5 4/8] merge: do not abort early if one strategy fails to handle the merge","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:14Z","receivedAt":"2022-07-23T01:53:36Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nbuiltin/merge is setup to allow multiple strategies to be specified,\nand it will find the \"best\" result and use it. This is defeated if\nsome of the merge strategies abort early when they cannot handle the\nmerge. Fix the logic that calls recursive and ort to not do such an\nearly abort, but instead return \"2\" or \"unhandled\" so that the next\nstrategy can try to handle the merge.\n\nComing up with a testcase for this is somewhat difficult, since\nrecursive and ort both handle nearly any two-headed merge (there is\na separate code path that checks for non-two-headed merges and\nalready returns \"2\" for them). So use a somewhat synthetic testcase\nof having the index not match HEAD before the merge starts, since all\nmerge strategies will abort for that.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 6 ++++--\n t/t6402-merge-rename.sh | 2 +-\n t/t6424-merge-unrelated-index-changes.sh | 16 ++++++++++++++++\n t/t6439-merge-co-error-msgs.sh | 1 +\n 4 files changed, 22 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex b43876f68e4..c120ad619c4 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -754,8 +754,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n \t\telse\n \t\t\tclean = merge_recursive(&o, head, remoteheads->item,\n \t\t\t\t\t\treversed, &result);\n-\t\tif (clean < 0)\n-\t\t\texit(128);\n+\t\tif (clean < 0) {\n+\t\t\trollback_lock_file(&lock);\n+\t\t\treturn 2;\n+\t\t}\n \t\tif (write_locked_index(&the_index, &lock,\n \t\t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n \t\t\tdie(_(\"unable to write %s\"), get_index_file());\ndiff --git a/t/t6402-merge-rename.sh b/t/t6402-merge-rename.sh\nindex 3a32b1a45cf..772238e582c 100755\n--- a/t/t6402-merge-rename.sh\n+++ b/t/t6402-merge-rename.sh\n@@ -210,7 +210,7 @@ test_expect_success 'updated working tree file should prevent the merge' '\n \techo >>M one line addition &&\n \tcat M >M.saved &&\n \tgit update-index M &&\n-\ttest_expect_code 128 git pull --no-rebase . yellow &&\n+\ttest_expect_code 2 git pull --no-rebase . yellow &&\n \ttest_cmp M M.saved &&\n \trm -f M.saved\n '\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex 187c761ad84..615061c7af4 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -275,4 +275,20 @@ test_expect_success 'subtree' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'with multiple strategies, recursive or ort failure do not early abort' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\ttest_seq 0 10 >a &&\n+\tgit add a &&\n+\n+\tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n+\ttest_must_fail git merge -s recursive -s ort -s octopus C^0 >output 2>&1 &&\n+\n+\tgrep \"Trying merge strategy recursive...\" output &&\n+\tgrep \"Trying merge strategy ort...\" output &&\n+\tgrep \"Trying merge strategy octopus...\" output &&\n+\tgrep \"No merge strategy handled the merge.\" output\n+'\n+\n test_done\ndiff --git a/t/t6439-merge-co-error-msgs.sh b/t/t6439-merge-co-error-msgs.sh\nindex 5bfb027099a..52cf0c87690 100755\n--- a/t/t6439-merge-co-error-msgs.sh\n+++ b/t/t6439-merge-co-error-msgs.sh\n@@ -47,6 +47,7 @@ test_expect_success 'untracked files overwritten by merge (fast and non-fast for\n \t\texport GIT_MERGE_VERBOSITY &&\n \t\ttest_must_fail git merge branch 2>out2\n \t) &&\n+\techo \"Merge with strategy ${GIT_TEST_MERGE_ALGORITHM:-ort} failed.\" >>expect &&\n \ttest_cmp out2 expect &&\n \tgit reset --hard HEAD^\n '\n-- \ngitgitgadget\n\n"},{"id":"459814","messageId":"3adfd9219954109d9d002459599aac4e0b96a04f.1658541198.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"[PATCH v5 3/8] merge: abort if index does not match HEAD for trivial merges","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:13Z","receivedAt":"2022-07-23T01:53:36Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nAs noted in the last commit and the links therein (especially commit\n9822175d2b (\"Ensure index matches head before invoking merge machinery,\nround N\", 2019-08-17), we have had a very long history of problems with\nfailing to enforce the requirement that index matches HEAD when starting\na merge.\n\nThe \"trivial merge\" logic in builtin/merge.c is yet another such case\nwe previously missed. Add a check for it to ensure it aborts if the\nindex does not match HEAD, and add a testcase where this fix is needed.\n\nNote that the fix here would also incidentally be an alternative fix\nfor the testcase added in the last patch, but the fix in the last patch\nis still needed when multiple merge strategies are in use, so tweak the\ntestcase from the previous commit so that it continues to exercise the\ncodepath added in the last commit.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 15 +++++++++++++++\n t/t6424-merge-unrelated-index-changes.sh | 22 +++++++++++++++++++++-\n 2 files changed, 36 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 23170f2d2a6..b43876f68e4 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -1599,6 +1599,21 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t\t */\n \t\trefresh_cache(REFRESH_QUIET);\n \t\tif (allow_trivial && fast_forward != FF_ONLY) {\n+\t\t\t/*\n+\t\t\t * Must first ensure that index matches HEAD before\n+\t\t\t * attempting a trivial merge.\n+\t\t\t */\n+\t\t\tstruct tree *head_tree = get_commit_tree(head_commit);\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tif (repo_index_has_changes(the_repository, head_tree,\n+\t\t\t\t\t\t &sb)) {\n+\t\t\t\terror(_(\"Your local changes to the following files would be overwritten by merge:\\n %s\"),\n+\t\t\t\t sb.buf);\n+\t\t\t\tstrbuf_release(&sb);\n+\t\t\t\treturn 2;\n+\t\t\t}\n+\n \t\t\t/* See if it is really trivial. */\n \t\t\tgit_committer_info(IDENT_STRICT);\n \t\t\tprintf(_(\"Trying really trivial in-index merge...\\n\"));\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex eabe6bda832..187c761ad84 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -114,6 +114,19 @@ test_expect_success 'resolve, non-trivial' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'resolve, trivial, related file removed' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\tgit rm a &&\n+\ttest_path_is_missing a &&\n+\n+\ttest_must_fail git merge -s resolve C^0 &&\n+\n+\ttest_path_is_missing a &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n test_expect_success 'resolve, non-trivial, related file removed' '\n \tgit reset --hard &&\n \tgit checkout B^0 &&\n@@ -121,7 +134,14 @@ test_expect_success 'resolve, non-trivial, related file removed' '\n \tgit rm a &&\n \ttest_path_is_missing a &&\n \n-\ttest_must_fail git merge -s resolve D^0 &&\n+\t# We also ask for recursive in order to turn off the \"allow_trivial\"\n+\t# setting in builtin/merge.c, and ensure that resolve really does\n+\t# correctly fail the merge (I guess this also tests that recursive\n+\t# correctly fails the merge, but the main thing we are attempting\n+\t# to test here is resolve and are just using the side effect of\n+\t# adding recursive to ensure that resolve is actually tested rather\n+\t# than the trivial merge codepath)\n+\ttest_must_fail git merge -s resolve -s recursive D^0 &&\n \n \ttest_path_is_missing a &&\n \ttest_path_is_missing .git/MERGE_HEAD\n-- \ngitgitgadget\n\n"},{"id":"459816","messageId":"e7c6de9e0c1c8ea7d759e0cd635a350c980d60a7.1658541198.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"[PATCH v5 5/8] merge: fix save_state() to work when there are stat-dirty files","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:15Z","receivedAt":"2022-07-23T01:53:36Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nWhen there are stat-dirty files, but no files are modified,\n`git stash create` exits with unsuccessful status. This causes merge\nto fail. Copy some code from sequencer.c's create_autostash to refresh\nthe index first to avoid this problem.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 8 ++++++++\n t/t6424-merge-unrelated-index-changes.sh | 11 +++++++++++\n 2 files changed, 19 insertions(+)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex c120ad619c4..780b4b9100a 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -313,8 +313,16 @@ static int save_state(struct object_id *stash)\n \tint len;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tstruct strbuf buffer = STRBUF_INIT;\n+\tstruct lock_file lock_file = LOCK_INIT;\n+\tint fd;\n \tint rc = -1;\n \n+\tfd = repo_hold_locked_index(the_repository, &lock_file, 0);\n+\trefresh_cache(REFRESH_QUIET);\n+\tif (0 <= fd)\n+\t\trepo_update_index_if_able(the_repository, &lock_file);\n+\trollback_lock_file(&lock_file);\n+\n \tstrvec_pushl(&cp.args, \"stash\", \"create\", NULL);\n \tcp.out = -1;\n \tcp.git_cmd = 1;\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex 615061c7af4..2c83210f9fd 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -275,6 +275,17 @@ test_expect_success 'subtree' '\n \ttest_path_is_missing .git/MERGE_HEAD\n '\n \n+test_expect_success 'avoid failure due to stat-dirty files' '\n+\tgit reset --hard &&\n+\tgit checkout B^0 &&\n+\n+\t# Make \"a\" be stat-dirty\n+\ttest-tool chmtime =+1 a &&\n+\n+\t# stat-dirty file should not prevent stash creation in builtin/merge.c\n+\tgit merge -s resolve -s recursive D^0\n+'\n+\n test_expect_success 'with multiple strategies, recursive or ort failure do not early abort' '\n \tgit reset --hard &&\n \tgit checkout B^0 &&\n-- \ngitgitgadget\n\n"},{"id":"459815","messageId":"d39d647245562f9949d2a52290ef03182b3dd78a.1658541198.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"[PATCH v5 6/8] merge: make restore_state() restore staged state too","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:16Z","receivedAt":"2022-07-23T01:53:37Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nThere are multiple issues at play here:\n\n 1) If `git merge` is invoked with staged changes, it should abort\n without doing any merging, and the user's working tree and index\n should be the same as before merge was invoked.\n 2) Merge strategies are responsible for enforcing the index == HEAD\n requirement. (See 9822175d2b (\"Ensure index matches head before\n invoking merge machinery, round N\", 2019-08-17) for some history\n around this.)\n 3) Merge strategies can bail saying they are not an appropriate\n handler for the merge in question (possibly allowing other\n strategies to be used instead).\n 4) Merge strategies can make changes to the index and working tree,\n and have no expectation to clean up after themselves, *even* if\n they bail out and say they are not an appropriate handler for\n the merge in question. (The `octopus` merge strategy does this,\n for example.)\n 5) Because of (3) and (4), builtin/merge.c stashes state before\n trying merge strategies and restores it afterward.\n\nUnfortunately, if users had staged changes before calling `git merge`,\nbuiltin/merge.c could do the following:\n\n * stash the changes, in order to clean up after the strategies\n * try all the merge strategies in turn, each of which report they\n cannot function due to the index not matching HEAD\n * restore the changes via \"git stash apply\"\n\nBut that last step would have the net effect of unstaging the user's\nchanges. Fix this by adding the \"--index\" option to \"git stash apply\".\nWhile at it, also squelch the stash apply output; we already report\n\"Rewinding the tree to pristine...\" and don't need a detailed `git\nstatus` report afterwards. Also while at it, switch to using strvec\nso folks don't have to count the arguments to ensure we avoided an\noff-by-one error, and so it's easier to add additional arguments to\nthe command.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 8 +++++---\n t/t6424-merge-unrelated-index-changes.sh | 7 ++++++-\n 2 files changed, 11 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 780b4b9100a..e0a3299e92e 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -383,20 +383,22 @@ static void reset_hard(const struct object_id *oid, int verbose)\n static void restore_state(const struct object_id *head,\n \t\t\t const struct object_id *stash)\n {\n-\tconst char *args[] = { \"stash\", \"apply\", NULL, NULL };\n+\tstruct strvec args = STRVEC_INIT;\n \n \tif (is_null_oid(stash))\n \t\treturn;\n \n \treset_hard(head, 1);\n \n-\targs[2] = oid_to_hex(stash);\n+\tstrvec_pushl(&args, \"stash\", \"apply\", \"--index\", \"--quiet\", NULL);\n+\tstrvec_push(&args, oid_to_hex(stash));\n \n \t/*\n \t * It is OK to ignore error here, for example when there was\n \t * nothing to restore.\n \t */\n-\trun_command_v_opt(args, RUN_GIT_CMD);\n+\trun_command_v_opt(args.v, RUN_GIT_CMD);\n+\tstrvec_clear(&args);\n \n \trefresh_cache(REFRESH_QUIET);\n }\ndiff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\nindex 2c83210f9fd..a61f20c22fe 100755\n--- a/t/t6424-merge-unrelated-index-changes.sh\n+++ b/t/t6424-merge-unrelated-index-changes.sh\n@@ -292,6 +292,7 @@ test_expect_success 'with multiple strategies, recursive or ort failure do not e\n \n \ttest_seq 0 10 >a &&\n \tgit add a &&\n+\tgit rev-parse :a >expect &&\n \n \tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n \ttest_must_fail git merge -s recursive -s ort -s octopus C^0 >output 2>&1 &&\n@@ -299,7 +300,11 @@ test_expect_success 'with multiple strategies, recursive or ort failure do not e\n \tgrep \"Trying merge strategy recursive...\" output &&\n \tgrep \"Trying merge strategy ort...\" output &&\n \tgrep \"Trying merge strategy octopus...\" output &&\n-\tgrep \"No merge strategy handled the merge.\" output\n+\tgrep \"No merge strategy handled the merge.\" output &&\n+\n+\t# Changes to \"a\" should remain staged\n+\tgit rev-parse :a >actual &&\n+\ttest_cmp expect actual\n '\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"459817","messageId":"7f5c6884d685f2b007db56333ad3441e8bc22641.1658541198.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"[PATCH v5 7/8] merge: ensure we can actually restore pre-merge state","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:17Z","receivedAt":"2022-07-23T01:53:46Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nMerge strategies can:\n * succeed with a clean merge\n * succeed with a conflicted merge\n * fail to handle the given type of merge\n\nIf one is thinking in terms of automatic mergeability, they would use\nthe word \"fail\" instead of \"succeed\" for the second bullet, but I am\nfocusing here on ability of the merge strategy to handle the given\ninputs, not on whether the given inputs are mergeable. The third\ncategory is about the merge strategy failing to know how to handle the\ngiven data; examples include:\n\n * Passing more than 2 branches to 'recursive' or 'ort'\n * Passing 2 or fewer branches to 'octopus'\n * Trying to do more complicated merges with 'resolve' (I believe\n directory/file conflicts will cause it to bail.)\n * Octopus running into a merge conflict for any branch OTHER than\n the final one (see the \"exit 2\" codepath of commit 98efc8f3d8\n (\"octopus: allow manual resolve on the last round.\", 2006-01-13))\n\nThat final one is particularly interesting, because it shows that the\nmerge strategy can muck with the index and working tree, and THEN bail\nand say \"sorry, this strategy cannot handle this type of merge; use\nsomething else\".\n\nFurther, we do not currently expect the individual strategies to clean\nup after themselves, but instead expect builtin/merge.c to do so. For\nit to be able to, it needs to save the state before trying the merge\nstrategy so it can have something to restore to. Therefore, remove the\nshortcut bypassing the save_state() call.\n\nThere is another bug on the restore_state() side of things, so no\ntestcase will be added until the next commit when we have addressed that\nissue as well.\n\nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex e0a3299e92e..3c4f415d87e 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -1682,12 +1682,12 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t * tree in the index -- this means that the index must be in\n \t * sync with the head commit. The strategies are responsible\n \t * to ensure this.\n+\t *\n+\t * Stash away the local changes so that we can try more than one\n+\t * and/or recover from merge strategies bailing while leaving the\n+\t * index and working tree polluted.\n \t */\n-\tif (use_strategies_nr == 1 ||\n-\t /*\n-\t * Stash away the local changes so that we can try more than one.\n-\t */\n-\t save_state(&stash))\n+\tif (save_state(&stash))\n \t\toidclr(&stash);\n \n \tfor (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n-- \ngitgitgadget\n\n"},{"id":"459818","messageId":"954dec526a2f6df591f84c4a866d61e4bccd8782.1658541198.git.gitgitgadget@gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"[PATCH v5 8/8] merge: do not exit restore_state() prematurely","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-23T01:53:18Z","receivedAt":"2022-07-23T01:53:47Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nPreviously, if the user:\n\n* Had no local changes before starting the merge\n* A merge strategy makes changes to the working tree/index but returns\n with exit status 2\n\nThen we'd call restore_state() to clean up the changes and either let\nthe next merge strategy run (if there is one), or exit telling the user\nthat no merge strategy could handle the merge. Unfortunately,\nrestore_state() did not clean up the changes as expected; that function\nwas a no-op if the stash was a null, and the stash would be null if\nthere were no local changes before starting the merge. So, instead of\n\"Rewinding the tree to pristine...\" as the code claimed, restore_state()\nwould leave garbage around in the index and working tree (possibly\nincluding conflicts) for either the next merge strategy or for the user\nafter aborting the merge. And in the case of aborting the merge, the\nuser would be unable to run \"git merge --abort\" to get rid of the\nunintended leftover conflicts, because the merge control files were not\nwritten as it was presumed that we had restored to a clean state\nalready.\n\nFix the main problem by making sure that restore_state() only skips the\nstash application if the stash is null rather than skipping the whole\nfunction.\n\nHowever, there is a secondary problem -- since merge.c forks\nsubprocesses to do the cleanup, the in-memory index is left out-of-sync.\nWhile there was a refresh_cache(REFRESH_QUIET) call that attempted to\ncorrect that, that function would not handle cases where the previous\nmerge strategy added conflicted entries. We need to drop the index and\nre-read it to handle such cases.\n\n(Alternatively, we could stop forking subprocesses and instead call some\nappropriate function to do the work which would update the in-memory\nindex automatically. For now, just do the simple fix.)\n\nAlso, add a testcase checking this, one for which the octopus strategy\nfails on the first commit it attempts to merge, and thus which it\ncannot handle at all and must completely bail on (as per the \"exit 2\"\ncode path of commit 98efc8f3d8 (\"octopus: allow manual resolve on the\nlast round.\", 2006-01-13)).\n\nReported-by: ZheNing Hu \nSigned-off-by: Elijah Newren \n---\n builtin/merge.c | 10 ++++++----\n t/t7607-merge-state.sh | 32 ++++++++++++++++++++++++++++++++\n 2 files changed, 38 insertions(+), 4 deletions(-)\n create mode 100755 t/t7607-merge-state.sh\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 3c4f415d87e..f7c92c0e64f 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -385,11 +385,11 @@ static void restore_state(const struct object_id *head,\n {\n \tstruct strvec args = STRVEC_INIT;\n \n-\tif (is_null_oid(stash))\n-\t\treturn;\n-\n \treset_hard(head, 1);\n \n+\tif (is_null_oid(stash))\n+\t\tgoto refresh_cache;\n+\n \tstrvec_pushl(&args, \"stash\", \"apply\", \"--index\", \"--quiet\", NULL);\n \tstrvec_push(&args, oid_to_hex(stash));\n \n@@ -400,7 +400,9 @@ static void restore_state(const struct object_id *head,\n \trun_command_v_opt(args.v, RUN_GIT_CMD);\n \tstrvec_clear(&args);\n \n-\trefresh_cache(REFRESH_QUIET);\n+refresh_cache:\n+\tif (discard_cache() < 0 || read_cache() < 0)\n+\t\tdie(_(\"could not read index\"));\n }\n \n /* This is called when no merge was necessary. */\ndiff --git a/t/t7607-merge-state.sh b/t/t7607-merge-state.sh\nnew file mode 100755\nindex 00000000000..89a62ac53b3\n--- /dev/null\n+++ b/t/t7607-merge-state.sh\n@@ -0,0 +1,32 @@\n+#!/bin/sh\n+\n+test_description=\"Test that merge state is as expected after failed merge\"\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+. ./test-lib.sh\n+\n+test_expect_success 'Ensure we restore original state if no merge strategy handles it' '\n+\ttest_commit --no-tag \"Initial\" base base &&\n+\n+\tfor b in branch1 branch2 branch3\n+\tdo\n+\t\tgit checkout -b $b main &&\n+\t\ttest_commit --no-tag \"Change on $b\" base $b || return 1\n+\tdone &&\n+\n+\tgit checkout branch1 &&\n+\t# This is a merge that octopus cannot handle. Note, that it does not\n+\t# just hit conflicts, it completely fails and says that it cannot\n+\t# handle this type of merge.\n+\ttest_expect_code 2 git merge branch2 branch3 >output 2>&1 &&\n+\tgrep \"fatal: merge program failed\" output &&\n+\tgrep \"Should not be doing an octopus\" output &&\n+\n+\t# Make sure we did not leave stray changes around when no appropriate\n+\t# merge strategy was found\n+\tgit diff --exit-code --name-status &&\n+\ttest_path_is_missing .git/MERGE_HEAD\n+'\n+\n+test_done\n-- \ngitgitgadget\n"},{"id":"459819","messageId":"CABPp-BGTqkqi5PKgbTSXOVC=TG25VhDRK_=b7SzJ2TZWzvVBWQ@mail.gmail.com","threadId":"57898","inReplyTo":"220722.86k085s9g3.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v4 5/7] merge: make restore_state() restore staged state too","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-23T01:56:15Z","receivedAt":"2022-07-23T01:56:30Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Jul 22, 2022 at 3:55 AM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> On Fri, Jul 22 2022, Elijah Newren via GitGitGadget wrote:\n>\n> > From: Elijah Newren \n> >\n> > There are multiple issues at play here:\n> >\n> > 1) If `git merge` is invoked with staged changes, it should abort\n> > without doing any merging, and the user's working tree and index\n> > should be the same as before merge was invoked.\n> > 2) Merge strategies are responsible for enforcing the index == HEAD\n> > requirement. (See 9822175d2b (\"Ensure index matches head before\n> > invoking merge machinery, round N\", 2019-08-17) for some history\n> > around this.)\n> > 3) Merge strategies can bail saying they are not an appropriate\n> > handler for the merge in question (possibly allowing other\n> > strategies to be used instead).\n> > 4) Merge strategies can make changes to the index and working tree,\n> > and have no expectation to clean up after themselves, *even* if\n> > they bail out and say they are not an appropriate handler for\n> > the merge in question. (The `octopus` merge strategy does this,\n> > for example.)\n> > 5) Because of (3) and (4), builtin/merge.c stashes state before\n> > trying merge strategies and restores it afterward.\n> >\n> > Unfortunately, if users had staged changes before calling `git merge`,\n> > builtin/merge.c could do the following:\n> >\n> > * stash the changes, in order to clean up after the strategies\n> > * try all the merge strategies in turn, each of which report they\n> > cannot function due to the index not matching HEAD\n> > * restore the changes via \"git stash apply\"\n> >\n> > But that last step would have the net effect of unstaging the user's\n> > changes. Fix this by adding the \"--index\" option to \"git stash apply\".\n> > While at it, also squelch the stash apply output; we already report\n> > \"Rewinding the tree to pristine...\" and don't need a detailed `git\n> > status` report afterwards.\n> >\n> > Signed-off-by: Elijah Newren \n> > ---\n> > builtin/merge.c | 5 +++--\n> > t/t6424-merge-unrelated-index-changes.sh | 7 ++++++-\n> > 2 files changed, 9 insertions(+), 3 deletions(-)\n> >\n> > diff --git a/builtin/merge.c b/builtin/merge.c\n> > index 4170c30317e..f807bf335bd 100644\n> > --- a/builtin/merge.c\n> > +++ b/builtin/merge.c\n> > @@ -383,14 +383,15 @@ static void reset_hard(const struct object_id *oid, int verbose)\n> > static void restore_state(const struct object_id *head,\n> > const struct object_id *stash)\n> > {\n> > - const char *args[] = { \"stash\", \"apply\", NULL, NULL };\n> > + const char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n> > + NULL, NULL };\n> >\n> > if (is_null_oid(stash))\n> > return;\n> >\n> > reset_hard(head, 1);\n> >\n> > - args[2] = oid_to_hex(stash);\n> > + args[4] = oid_to_hex(stash);\n> >\n> > /*\n> > * It is OK to ignore error here, for example when there was\n>\n> Just a nit/side comment: This is one of these older-style arg\n> constructions that we've replaced with strvec in most other places.\n>\n> Let's leave this alone for now (especially in a v4), but FWIW I wouldn't\n> mind if these sort of changes were strvec converted while at it:\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 64def49734a..c3a3a1fde50 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -383,21 +383,23 @@ static void reset_hard(const struct object_id *oid, int verbose)\n> static void restore_state(const struct object_id *head,\n> const struct object_id *stash)\n> {\n> - const char *args[] = { \"stash\", \"apply\", \"--index\", \"--quiet\",\n> - NULL, NULL };\n> + struct strvec args = STRVEC_INIT;\n> +\n> + strvec_pushl(&args, \"stash\", \"apply\", \"--index\", \"--quiet\", NULL);\n>\n> reset_hard(head, 1);\n>\n> if (is_null_oid(stash))\n> goto refresh_cache;\n>\n> - args[4] = oid_to_hex(stash);\n> + strvec_push(&args, oid_to_hex(stash));\n>\n> /*\n> * It is OK to ignore error here, for example when there was\n> * nothing to restore.\n> */\n> - run_command_v_opt(args, RUN_GIT_CMD);\n> + run_command_v_opt(args.v, RUN_GIT_CMD);\n> + strvec_clear(&args);\n>\n> refresh_cache:\n> if (discard_cache() < 0 || read_cache() < 0)\n>\n> I.e. it takes about as much mental energy to review that as counting the\n> args elements and seeing that 2 to 4 is correct :)\n\nI like this change and included it in my reroll. Thanks!\n"},{"id":"459828","messageId":"220723.86h738qsr9.gmgdl@evledraar.gmail.com","threadId":"57898","inReplyTo":"CABPp-BHToEDv=8W9nf4XDC8b+5=EVuc=OnoT6xCRGWxB9CKWfw@mail.gmail.com","subject":"Re: [PATCH v4 2/7] merge-resolve: abort if index does not match HEAD","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-23T05:44:35Z","receivedAt":"2022-07-23T06:03:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Jul 22 2022, Elijah Newren wrote:\n\n> On Fri, Jul 22, 2022 at 3:46 AM Ævar Arnfjörð Bjarmason\n> [...]\n>> I don't know enough about the context here, but given our *.sh->C\n>> migration elsewhere it's a bit unfortunate to see more *.sh code added\n>> back.\n>\n> This seems like a curious objection. \"We are trying to get rid of\n> shell scripts, so don't even fix bugs in any of the existing ones.\" ?\n>\n>> We have \"git merge\" driving this, isn't it OK to have it make this\n>> check before invoking \"resolve\" (may be a stupid question).\n>\n> Ah, I can kind of see where you're coming from now, but that seems to\n> me to be bending over backwards in attempting to fix a component\n> written in shell without actually modifying the shell.\n> builtin/merge.c is some glue code that can call multiple different\n> strategies, but isn't the place for the implementation of the\n> strategies themselves, and I'd hate to see us put half the\n> implementation in one place and half in another. In addition, besides\n> the separation of concerns issue::\n>\n> * We document that users can add their own merge strategies (a\n> shell or executable named git-merge-$USERNAME and \"git merge -s\n> $USERNAME\" will call them)\n> * git-merge-resolve and git-merge-octopus serve as examples\n> * Our examples should demonstrate correct behavior and perform\n> documented, required steps. This particular check is important:\n>\n> /*\n> * At this point, we need a real merge. No matter what strategy\n> * we use, it would operate on the index, possibly affecting the\n> * working tree, and when resolved cleanly, have the desired\n> * tree in the index -- this means that the index must be in\n> * sync with the head commit. The strategies are responsible\n> * to ensure this.\n> */\n>\n> So, even if someone were to reimplement git-merge-resolve.sh in C, and\n> start the deprecation process with some merge.useBuiltinResolve config\n> setting (similar to rebase.useBuiltin), I'd still want this shell fix\n> added to git-merge-resolve.sh in the meantime, both as an important\n> bugfix, and so that people looking for merge strategy examples who\n> find this script hopefully find a new enough version with this\n> important check included.\n>\n> In general, if merge strategies do not perform this check, we have\n> observed that they often will either (a) discard users' staged changes\n> (best case) or (b) smash staged changes into the created commit and\n> thus create some kind of evil merge (making it look like they created\n> a merge normally, and then amended the merge with additional changes).\n>\n> We're lucky that the way resolve was implemented, other git calls\n> would usually incidentally catch such issues for us without an\n> explicit check. We were also lucky that the observed behavior was\n> '(a)' rather than '(b)' for resolve. But the issue should still be\n> fixed.\n\nMakes sense I guess, yeah I was wondering if we could just assume that\n\"git merge\" would always invoke those, and therefore could offload more\nof the pre-flight checks over there.\n\n>> For this code in particular it:\n>>\n>> * Uses spaces, not tabs\n>\n> Yes, that's fair, but as I mentioned in the commit message, it was\n> copied from git-merge-octopus.sh. So, as you say below, \"so did the\n> older version\".\n\n*nod*\n\n>> * We lose the diff-index .. --name-only exit code (segfault), but so\n>> did the older version\n>\n> Um, I don't understand this objection. I think you are referring to\n> the pipe to sed, but if so...who cares? The exit code would be lost\n> anyway because we aren't running under errexit, and the next line of\n> code ignores any and all previous exit codes when it runs \"exit 2\".\n> And if you're not referring to the pipe to sed but the fact that it\n> unconditionally returns an exit code of 2 on the next line, then yes\n> that is the expected return code. Whatever the diff-index segfault\n> returns would be the wrong exit status and could fool the\n> builtin/merge.c into doing the wrong thing. It expects merge\n> strategies to return one of three exit codes: 0, 1, or 2:\n>\n> /*\n> * The backend exits with 1 when conflicts are\n> * left to be resolved, with 2 when it does not\n> * handle the given merge at all.\n> */\n>\n> So, ignoring the return code from diff-index is correct behavior here.\n>\n> Were you thinking this was a test script or something?\n\nWe can leave this for now.\n\nBut no. Whatever the merge driver is documenting as its normal return\nvalues we really should be ferrying up abort() and segfault, per the\n\"why do we miss...\" in:\nhttps://lore.kernel.org/git/patch-v2-11.14-8cc6ab390db-20220720T211221Z-avarab@gmail.com/\n\nI.e. this is one of the cases in the test suite where we haven't closed\nthat gap, and could hide segfaults as a \"normal\" exit 2.\n\nSo I think your v5 is fine as-is, but in general I'd be really\ninterested if you want to double-down on this view for the merge drivers\nfor some reason, because my current plan for addressing these blindspots\noutlined in the above wouldn't work then...\n\n >> * I wonder if bending over backwards to emit the exact message we\n>> emitted before is worth it\n>>\n>> If you just make this something like (untested):\n>>\n>> {\n>> gettext \"error: \" &&\n>> gettextln \"Your local...\"\n>> }\n>>\n>> You could re-use the translation from the *.c one (and the \"error: \" one\n>> we'll get from usage.c).\n>>\n>> That leaves \"\\n %s\" as the difference, but we could just remove that\n>> from the _() and emit it unconditionally, no?\n>\n> ??\n>\n> Copying a few lines from git-merge-octopus.sh to get the same fix it\n> has is \"bending over backwards\"? That's what I call \"doing the\n> easiest thing possible\" (and which _also_ has the benefit of being\n> battle tested code), and then you describe a bunch of gymnastics as an\n> alternative? I see your suggestion as running afoul of the objection\n> you are raising, and the code I'm adding as being a solution to that\n> particular objection. So this particular flag you are raising is\n> confusing to me.\n\nI wasn't aware of some greater context vis-as-vis octopus, but it just\nseemed to me that you were trying to maintain the \"Error\" v.s. \"error\"\ndistinction, i.e. the C code you're adding uses lower case, the *.sh\nupper-case.\n\nWhich I see is for consistency with some existing message we have a\ntranslation for, so if that was the main goal (and not bug-for-bug\nmessage compatibility) the above gettext/gettextln use would allow you\nto re-use the C i18n.\n\n"},{"id":"459860","messageId":"220725.864jz5qwms.gmgdl@evledraar.gmail.com","threadId":"57898","inReplyTo":"b41853e3f9908ab458bcb28684d817677e32367b.1658391391.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 3/7] merge: do not abort early if one strategy fails to handle the merge","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-25T10:38:15Z","receivedAt":"2022-07-25T11:06:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Jul 21 2022, Elijah Newren via GitGitGadget wrote:\n\n> From: Elijah Newren \n>\n> builtin/merge is setup to allow multiple strategies to be specified,\n> and it will find the \"best\" result and use it. This is defeated if\n> some of the merge strategies abort early when they cannot handle the\n> merge. Fix the logic that calls recursive and ort to not do such an\n> early abort, but instead return \"2\" or \"unhandled\" so that the next\n> strategy can try to handle the merge.\n>\n> Coming up with a testcase for this is somewhat difficult, since\n> recursive and ort both handle nearly any two-headed merge (there is\n> a separate code path that checks for non-two-headed merges and\n> already returns \"2\" for them). So use a somewhat synthetic testcase\n> of having the index not match HEAD before the merge starts, since all\n> merge strategies will abort for that.\n>\n> Signed-off-by: Elijah Newren \n> ---\n> builtin/merge.c | 6 ++++--\n> t/t6402-merge-rename.sh | 2 +-\n> t/t6424-merge-unrelated-index-changes.sh | 16 ++++++++++++++++\n> t/t6439-merge-co-error-msgs.sh | 1 +\n> 4 files changed, 22 insertions(+), 3 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 13884b8e836..dec7375bf2a 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -754,8 +754,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n> \t\telse\n> \t\t\tclean = merge_recursive(&o, head, remoteheads->item,\n> \t\t\t\t\t\treversed, &result);\n> -\t\tif (clean < 0)\n> -\t\t\texit(128);\n> +\t\tif (clean < 0) {\n> +\t\t\trollback_lock_file(&lock);\n> +\t\t\treturn 2;\n> +\t\t}\n> \t\tif (write_locked_index(&the_index, &lock,\n> \t\t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n> \t\t\tdie(_(\"unable to write %s\"), get_index_file());\n> diff --git a/t/t6402-merge-rename.sh b/t/t6402-merge-rename.sh\n> index 3a32b1a45cf..772238e582c 100755\n> --- a/t/t6402-merge-rename.sh\n> +++ b/t/t6402-merge-rename.sh\n> @@ -210,7 +210,7 @@ test_expect_success 'updated working tree file should prevent the merge' '\n> \techo >>M one line addition &&\n> \tcat M >M.saved &&\n> \tgit update-index M &&\n> -\ttest_expect_code 128 git pull --no-rebase . yellow &&\n> +\ttest_expect_code 2 git pull --no-rebase . yellow &&\n> \ttest_cmp M M.saved &&\n> \trm -f M.saved\n> '\n> diff --git a/t/t6424-merge-unrelated-index-changes.sh b/t/t6424-merge-unrelated-index-changes.sh\n> index f35d3182b86..8b749e19083 100755\n> --- a/t/t6424-merge-unrelated-index-changes.sh\n> +++ b/t/t6424-merge-unrelated-index-changes.sh\n> @@ -268,4 +268,20 @@ test_expect_success 'subtree' '\n> \ttest_path_is_missing .git/MERGE_HEAD\n> '\n> \n> +test_expect_success 'resolve && recursive && ort' '\n> +\tgit reset --hard &&\n> +\tgit checkout B^0 &&\n> +\n> +\ttest_seq 0 10 >a &&\n> +\tgit add a &&\n> +\n> +\tsane_unset GIT_TEST_MERGE_ALGORITHM &&\n> +\ttest_must_fail git merge -s resolve -s recursive -s ort C^0 >output 2>&1 &&\n> +\n> +\tgrep \"Trying merge strategy resolve...\" output &&\n> +\tgrep \"Trying merge strategy recursive...\" output &&\n> +\tgrep \"Trying merge strategy ort...\" output &&\n> +\tgrep \"No merge strategy handled the merge.\" output\n> +'\n> +\n> test_done\n> diff --git a/t/t6439-merge-co-error-msgs.sh b/t/t6439-merge-co-error-msgs.sh\n> index 5bfb027099a..52cf0c87690 100755\n> --- a/t/t6439-merge-co-error-msgs.sh\n> +++ b/t/t6439-merge-co-error-msgs.sh\n> @@ -47,6 +47,7 @@ test_expect_success 'untracked files overwritten by merge (fast and non-fast for\n> \t\texport GIT_MERGE_VERBOSITY &&\n> \t\ttest_must_fail git merge branch 2>out2\n> \t) &&\n> +\techo \"Merge with strategy ${GIT_TEST_MERGE_ALGORITHM:-ort} failed.\" >>expect &&\n> \ttest_cmp out2 expect &&\n> \tgit reset --hard HEAD^\n> '\n\nI'm re-rolling ab/leak-check, and came up with the below (at the very\nend) to \"fix\" a report in builtin/merge.c, reading your commit message\nyour fix seems obviously better.\n\nMine's early WIP, and I e.g. didn't notice that I forgot to unlock the\n&lock file, which is correct.\n\nI *could* say \"that's not my problem\", i.e. we didn't unlock it before\n(we rely on atexit). The truth is I just missed it, but having said that\nit *is* true that we could do without it, or do it as a separate chaneg.\n\nI'm just posting my version below to help move yours forward, i.e. to\nshow that someone else has carefully at least this part.\n\nBut it is worth noting from staring at the two that your version is\nmixing several different behavior changes into one, which *could* be\nsplit up (but whether you think that's worth it I leave to you).\n\nMaybe I'm the only one initially confused by it, and that's probably\njust from being mentally biased towards my own \"solution\". Those are (at\nleast):\n\n 1. Before we didn't explicitly unlock() before exit(), but had atexit()\n do it, that could be a one-line first commit. This change is\n obviously good.\n\n 2. A commit like mine could come next, i.e. we bug-for-bug do what we\n do do now, but just run the \"post-builtin\" logic when we return from\n cmd_merge().\n\n Doing it as an in-between would be some churn, as we'll need to get\n rid of \"early_exit\" again, but would allow us to incrementally move\n forward to...\n\n 3. ...then we'd say \"but it actually makes sense not to early abort\",\n i.e. you want to change this so that we'll run the logic between\n try_merge_strategy() exiting with 128 now and the return from\n cmd_merge().\n\n This bit is my main sticking point in reviewing your change,\n i.e. your \"a testcase for this is somewhat difficult\" somewhat\n addresses this, but (and maybe I'm wrong) it seems to me that \n\n Editing that code the post-image looks like this, with my\n commentary & most of the code removed, i.e. just focusing on the\n branches we do and don't potentially have tests for:\n\n \t\t/* Before this we fall through from ret == 128 (or ret == 2...) */\n\t\tif (automerge_was_ok) { // not tested?\n\t\tif (!best_strategy) {\n\t\t\t// we test this...\n\t\t\tif (use_strategies_nr > 1)\n\t\t\t\t// And this: _(\"No merge strategy handled the merge.\\n\"));\n\t\t\telse\n\t\t\t\t// And this: _(\"Merge with strategy %s failed.\\n\"),\n\t\t} else if (best_strategy == wt_strategy)\n\t\t\t// but not this?\n\t\telse\n\t\t\t// Or this, where we e.g. say \"Rewinding the tree to pristene...\"?\n\t\n\t\tif (squash) {\n\t\t\t// this?\n\t\t} else\n\t\t\t// this? (probably, yes)\n\t\t\twrite_merge_state(remoteheads);\n\t\n\t\tif (merge_was_ok)\n\t\t\t// this? (probably, yes, we just don't grep it?)\n\t\telse\n\t\t\t// this? maybe yes because it's covered by the\n\t\t\t// \"failed\" above too?\n\t\t\tret = suggest_conflicts();\n\t\n\tdone:\n\t\tif (!automerge_was_ok) {\n\t\t\t// this? ditto the first \"not tested?\"\n\t\t}\n\n I.e. are you confident that we want to continue now in these various\n cases, where we have squash, !automerge_was_ok etc. I think it would\n be really useful to comment on (perhaps by amending the above\n pseudocode) what test cases we're not testing / test already etc.\n\n 4. Having done all that (or maybe this can't be split up / needs to\n come earlier) you say that we'd like to not generically call this\n exit state 128, but have it under the \"exit(2)\" umbrella.\n\nAgain, all just food for thought, and a way to step-by-step go through\nhow I came about reviewing this in detail, I hope it and the below\nversion I came up with before seeing yours helps.\n\nP.s.: The last paragraph in my commit message does not point to some\nhidden edge case in the code behavior here, it's just that clang/gcc are\nfunny about exit() and die() control flow when combined with\n-fsanitize=address and higher optimization levels.\n\n-- >8 --\nSubject: [PATCH] merge: return, don't use exit()\n\nChange some of the builtin/merge.c code added in f241ff0d0a9 (prepare\nthe builtins for a libified merge_recursive(), 2016-07-26) to ferry up\nan \"early return\" state, rather than having try_merge_strategy() call\nexit() itself.\n\nThis is a follow-up to dda31145d79 (Merge branch\n'ab/usage-die-message' into gc/branch-recurse-submodules-fix,\n2022-03-31).\n\nThe only behavior change here is that we'll now properly catch other\nissues on our way out, see e.g. [1] and the interaction with /dev/full\nfor an example.\n\nThe immediate reason to do this change is because it's one of the\ncases where clang and gcc's SANITIZE=leak behavior differs. Under\nclang we don't detect that \"t/t6415-merge-dir-to-symlink.sh\" triggers\na leak, but gcc spots it.\n\n1. https://lore.kernel.org/git/87im2n3gje.fsf@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/merge.c | 27 +++++++++++++++++++++------\n 1 file changed, 21 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 23170f2d2a6..a8d5d04f622 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -709,10 +709,12 @@ static void write_tree_trivial(struct object_id *oid)\n \n static int try_merge_strategy(const char *strategy, struct commit_list *common,\n \t\t\t struct commit_list *remoteheads,\n-\t\t\t struct commit *head)\n+\t\t\t struct commit *head, int *early_exit)\n {\n \tconst char *head_arg = \"HEAD\";\n \n+\t*early_exit = 0;\n+\n \tif (refresh_and_write_cache(REFRESH_QUIET, SKIP_IF_UNCHANGED, 0) < 0)\n \t\treturn error(_(\"Unable to write index.\"));\n \n@@ -754,8 +756,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n \t\telse\n \t\t\tclean = merge_recursive(&o, head, remoteheads->item,\n \t\t\t\t\t\treversed, &result);\n-\t\tif (clean < 0)\n-\t\t\texit(128);\n+\t\tif (clean < 0) {\n+\t\t\t*early_exit = 1;\n+\t\t\treturn 128;\n+\t\t}\n \t\tif (write_locked_index(&the_index, &lock,\n \t\t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n \t\t\tdie(_(\"unable to write %s\"), get_index_file());\n@@ -1665,6 +1669,8 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \n \tfor (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n \t\tint ret, cnt;\n+\t\tint early_exit;\n+\n \t\tif (i) {\n \t\t\tprintf(_(\"Rewinding the tree to pristine...\\n\"));\n \t\t\trestore_state(&head_commit->object.oid, &stash);\n@@ -1680,7 +1686,10 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \n \t\tret = try_merge_strategy(use_strategies[i]->name,\n \t\t\t\t\t common, remoteheads,\n-\t\t\t\t\t head_commit);\n+\t\t\t\t\t head_commit, &early_exit);\n+\t\tif (early_exit)\n+\t\t\tgoto done;\n+\n \t\t/*\n \t\t * The backend exits with 1 when conflicts are\n \t\t * left to be resolved, with 2 when it does not\n@@ -1732,12 +1741,18 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n \t} else if (best_strategy == wt_strategy)\n \t\t; /* We already have its result in the working tree. */\n \telse {\n+\t\tint new_ret, early_exit;\n+\n \t\tprintf(_(\"Rewinding the tree to pristine...\\n\"));\n \t\trestore_state(&head_commit->object.oid, &stash);\n \t\tprintf(_(\"Using the %s strategy to prepare resolving by hand.\\n\"),\n \t\t\tbest_strategy);\n-\t\ttry_merge_strategy(best_strategy, common, remoteheads,\n-\t\t\t\t head_commit);\n+\t\tnew_ret = try_merge_strategy(best_strategy, common, remoteheads,\n+\t\t\t\t\t head_commit, &early_exit);\n+\t\tif (early_exit) {\n+\t\t\tret = new_ret;\n+\t\t\tgoto done;\n+\t\t}\n \t}\n \n \tif (squash) {\n-- \n2.36.1\n\n"},{"id":"459906","messageId":"xmqqh7353ti1.fsf@gitster.g","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 0/8] Fix merge restore state","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-25T19:03:02Z","receivedAt":"2022-07-25T19:03:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Elijah Newren via GitGitGadget\" writes:\n\n> This started as a simple series to fix restore_state() in builtin/merge.c,\n> fixing an issue reported by ZheNing Hu[3]. It now fixes several bugs and has\n> grown so much it's hard to call it simple. Anyway...\n\nThanks. Unless we hear any more comments, let's start merging it\ndown to 'next' soonish.\n\n"},{"id":"459952","messageId":"CABPp-BFVZy+TtYtaKe1d25Ua3HSMrQpzkUDgfnzQvo8xzpD-3g@mail.gmail.com","threadId":"57898","inReplyTo":"220725.864jz5qwms.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v3 3/7] merge: do not abort early if one strategy fails to handle the merge","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-26T01:31:16Z","receivedAt":"2022-07-26T01:31:34Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Jul 25, 2022 at 4:06 AM Ævar Arnfjörð Bjarmason\n wrote:\n>\n[...]\n>\n> I'm re-rolling ab/leak-check, and came up with the below (at the very\n> end) to \"fix\" a report in builtin/merge.c, reading your commit message\n> your fix seems obviously better.\n>\n> Mine's early WIP, and I e.g. didn't notice that I forgot to unlock the\n> &lock file, which is correct.\n>\n> I *could* say \"that's not my problem\", i.e. we didn't unlock it before\n> (we rely on atexit). The truth is I just missed it, but having said that\n> it *is* true that we could do without it, or do it as a separate chaneg.\n>\n> I'm just posting my version below to help move yours forward, i.e. to\n> show that someone else has carefully at least this part.\n\n\"has carefully ... at least this part\" ?\n\nI think you have a missing verb there.\n\n> But it is worth noting from staring at the two that your version is\n> mixing several different behavior changes into one, which *could* be\n> split up (but whether you think that's worth it I leave to you).\n>\n> Maybe I'm the only one initially confused by it, and that's probably\n> just from being mentally biased towards my own \"solution\". Those are (at\n> least):\n>\n> 1. Before we didn't explicitly unlock() before exit(), but had atexit()\n> do it, that could be a one-line first commit. This change is\n> obviously good.\n\nThat'd be fine. (Though at this point, I'd rather not mess with the\nseries more.)\n\n> 2. A commit like mine could come next, i.e. we bug-for-bug do what we\n> do do now, but just run the \"post-builtin\" logic when we return from\n> cmd_merge().\n>\n> Doing it as an in-between would be some churn, as we'll need to get\n> rid of \"early_exit\" again, but would allow us to incrementally move\n> forward to...\n\nSo, add a step that makes it glaringly obvious that the code is not\nonly buggy but totally at odds with itself?\n\nbuiltin/merge.c was designed to allow pluggable backends and to\nautomatically pick the \"best\" one if more than one is specified. We\nhad a bug in one line of code that defeated the design, by making it\nnot bother consulting beyond the first failed backend in some cases.\nThat's the bug I'm trying to address. Your patch would make the\ninconsistency with the design both bigger and more obvious; I don't\nsee how it's a useful step to take.\n\nNow, the existing design might be questionable. In fact, I'm not sure\nI like it. But I think we should either change the design, or fix\nthings in a way that improves towards the existing design.\n\n> 3. ...then we'd say \"but it actually makes sense not to early abort\",\n> i.e. you want to change this so that we'll run the logic between\n> try_merge_strategy() exiting with 128 now and the return from\n> cmd_merge().\n>\n> This bit is my main sticking point in reviewing your change,\n> i.e. your \"a testcase for this is somewhat difficult\" somewhat\n> addresses this, but (and maybe I'm wrong) it seems to me that\n>\n> Editing that code the post-image looks like this, with my\n> commentary & most of the code removed, i.e. just focusing on the\n> branches we do and don't potentially have tests for:\n>\n> /* Before this we fall through from ret == 128 (or ret == 2...) */\n> if (automerge_was_ok) { // not tested?\n> if (!best_strategy) {\n> // we test this...\n> if (use_strategies_nr > 1)\n> // And this: _(\"No merge strategy handled the merge.\\n\"));\n> else\n> // And this: _(\"Merge with strategy %s failed.\\n\"),\n> } else if (best_strategy == wt_strategy)\n> // but not this?\n> else\n> // Or this, where we e.g. say \"Rewinding the tree to pristene...\"?\n>\n> if (squash) {\n> // this?\n> } else\n> // this? (probably, yes)\n> write_merge_state(remoteheads);\n>\n> if (merge_was_ok)\n> // this? (probably, yes, we just don't grep it?)\n> else\n> // this? maybe yes because it's covered by the\n> // \"failed\" above too?\n> ret = suggest_conflicts();\n>\n> done:\n> if (!automerge_was_ok) {\n> // this? ditto the first \"not tested?\"\n> }\n>\n> I.e. are you confident that we want to continue now in these various\n> cases, where we have squash, !automerge_was_ok etc. I think it would\n> be really useful to comment on (perhaps by amending the above\n> pseudocode) what test cases we're not testing / test already etc.\n\nTo be honest, I'm confused by what looks like a make-work project.\nPerhaps if I understood your frame of reference better, maybe they\nwouldn't bother me, but there's a few things here that seem a little\nfunny to me.\n\nYou've highlighted that you are worried about the case where ret is 2\n(or 128) at the point of all these branches in question. However,\nthree of those branches can almost trivially be deduced to never be\ntaken unless ret is 0. One of the other codepaths, for freeing\nmemory, is correct regardless of the value of ret -- the memory is\nconditionally freed earlier and the \"if\"-check exists only to avoid a\ndouble free (and checking the recent commit message where those lines\nwere added would explain this, though I'm not sure why it'd even need\nexplaining separately for e.g. ret == 2 compared to any other value).\nThree of the other code paths involve nothing more than print\nstatements. Now, there are many codepaths you highlighted, and\nperhaps there are some where it's not trivial to determine whether\nthey are okay in combination with a different return value. And it\nmay also be easy to miss some of the \"almost trivial\" cases. I'd\nunderstand better if you asked about the tougher ones or only some of\nthe easier ones, but it feels like you didn't try to check any of them\nand instead wanted me to just spend time commenting on every single\ncode branch?\n\nI hope that doesn't come across harshly. I'm just struggling to\nunderstand where the detailed request is coming from.\n\nHowever, perhaps I can obviate the whole set of requests by just\npointing out that I don't think any of them are relevant. The premise\nfor the audit request seems to be that you are worrying that the\nchange from die() to \"return 2\" (or 128) in try_merge_strategy() will\nresult in the calling code getting into a state it has never\nexperienced before which might uncover latent bugs. We can trivially\npoint out that it's not a new state, though: such return values were\nalready possible from try_merge_strategy() via the final line of code\nin the function -- namely, the \"return try_merge_command(...)\" code\npath. And a return value of 2 from try_merge_command() and\ntry_merge_strategy() isn't merely theoretical either -- you can easily\ntrigger it multiple ways; the easiest is perhaps by passing `-s\noctopus` when doing a non-octopus merge. (You can also make the\ntestcase more complex by combining that with as many other additional\nmerge strategies as you want, e.g. \"git merge -s octopus -s resolve -s\nrecursive -s mySpecialStrategy $BRANCH\". You can also move the\noctopus to the end if you want to test what happens if it is tried\nlast. Lots of possibilities exist).\n\n> 4. Having done all that (or maybe this can't be split up / needs to\n> come earlier) you say that we'd like to not generically call this\n> exit state 128, but have it under the \"exit(2)\" umbrella.\n\nI don't see how reading your set of steps 1-4 logically restores the\ndesign of builtin/merge.c, though. An example: what if the user ran\n\"git merge -s resolve -s recursive $BRANCH\", and `resolve` handled the\nmerge but had some conflicts, while `recursive` just failed and\nreturned a value of clean < 0? In such a case, builtin/merge.c is\nsupposed to restore the tree to pristine after attempting the\nrecursive backend, and then redo using the best_strategy (which would\nbe `resolve` in this example case). Your steps 1-4 never address such\na thing. Your steps might incidentally address such a case as a side\neffect of their implementation, but that's not at all clear. Since\nthe whole point is fixing the code to match the existing design, it\nseems really odd to split things into a set of steps that obscures\nthat fix.\n\n> Again, all just food for thought, and a way to step-by-step go through\n> how I came about reviewing this in detail, I hope it and the below\n> version I came up with before seeing yours helps.\n>\n> P.s.: The last paragraph in my commit message does not point to some\n> hidden edge case in the code behavior here, it's just that clang/gcc are\n> funny about exit() and die() control flow when combined with\n> -fsanitize=address and higher optimization levels.\n>\n> -- >8 --\n> Subject: [PATCH] merge: return, don't use exit()\n>\n> Change some of the builtin/merge.c code added in f241ff0d0a9 (prepare\n> the builtins for a libified merge_recursive(), 2016-07-26) to ferry up\n> an \"early return\" state, rather than having try_merge_strategy() call\n> exit() itself.\n>\n> This is a follow-up to dda31145d79 (Merge branch\n> 'ab/usage-die-message' into gc/branch-recurse-submodules-fix,\n> 2022-03-31).\n>\n> The only behavior change here is that we'll now properly catch other\n> issues on our way out, see e.g. [1] and the interaction with /dev/full\n> for an example.\n>\n> The immediate reason to do this change is because it's one of the\n> cases where clang and gcc's SANITIZE=leak behavior differs. Under\n> clang we don't detect that \"t/t6415-merge-dir-to-symlink.sh\" triggers\n> a leak, but gcc spots it.\n>\n> 1. https://lore.kernel.org/git/87im2n3gje.fsf@evledraar.gmail.com/\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> builtin/merge.c | 27 +++++++++++++++++++++------\n> 1 file changed, 21 insertions(+), 6 deletions(-)\n>\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 23170f2d2a6..a8d5d04f622 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -709,10 +709,12 @@ static void write_tree_trivial(struct object_id *oid)\n>\n> static int try_merge_strategy(const char *strategy, struct commit_list *common,\n> struct commit_list *remoteheads,\n> - struct commit *head)\n> + struct commit *head, int *early_exit)\n> {\n> const char *head_arg = \"HEAD\";\n>\n> + *early_exit = 0;\n> +\n> if (refresh_and_write_cache(REFRESH_QUIET, SKIP_IF_UNCHANGED, 0) < 0)\n> return error(_(\"Unable to write index.\"));\n>\n> @@ -754,8 +756,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n> else\n> clean = merge_recursive(&o, head, remoteheads->item,\n> reversed, &result);\n> - if (clean < 0)\n> - exit(128);\n> + if (clean < 0) {\n> + *early_exit = 1;\n> + return 128;\n> + }\n> if (write_locked_index(&the_index, &lock,\n> COMMIT_LOCK | SKIP_IF_UNCHANGED))\n> die(_(\"unable to write %s\"), get_index_file());\n> @@ -1665,6 +1669,8 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n>\n> for (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n> int ret, cnt;\n> + int early_exit;\n> +\n> if (i) {\n> printf(_(\"Rewinding the tree to pristine...\\n\"));\n> restore_state(&head_commit->object.oid, &stash);\n> @@ -1680,7 +1686,10 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n>\n> ret = try_merge_strategy(use_strategies[i]->name,\n> common, remoteheads,\n> - head_commit);\n> + head_commit, &early_exit);\n> + if (early_exit)\n> + goto done;\n> +\n> /*\n> * The backend exits with 1 when conflicts are\n> * left to be resolved, with 2 when it does not\n> @@ -1732,12 +1741,18 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> } else if (best_strategy == wt_strategy)\n> ; /* We already have its result in the working tree. */\n> else {\n> + int new_ret, early_exit;\n> +\n> printf(_(\"Rewinding the tree to pristine...\\n\"));\n> restore_state(&head_commit->object.oid, &stash);\n> printf(_(\"Using the %s strategy to prepare resolving by hand.\\n\"),\n> best_strategy);\n> - try_merge_strategy(best_strategy, common, remoteheads,\n> - head_commit);\n> + new_ret = try_merge_strategy(best_strategy, common, remoteheads,\n> + head_commit, &early_exit);\n> + if (early_exit) {\n> + ret = new_ret;\n> + goto done;\n> + }\n\nIncidentally, this is essentially dead code being added in this last\nhunk. This final `else` block can only be triggered when\nbest_strategy has been set, and best_strategy will only be set after a\nmerge strategy works (possibly with conflicts, but was at least\nappropriate for the problem). Anyway, by the point of this `else`\nblock, we've run multiple strategies already, at least one worked, and\nthe \"best\" one was not the last one tried. So, at this point, we\nsimply rewind the tree and rerun the known-working merge strategy. (I\nguess you could say that maybe the merge strategy might not return the\nsame result despite being given the same inputs, so theoretically\nearly_exit could come back true and this hunk isn't quite dead. Just\nmostly dead, I guess.)\n\n> }\n>\n> if (squash) {\n> --\n> 2.36.1\n"},{"id":"459953","messageId":"CABPp-BE99NVBKRn9Uh3HMcgzV-egtmgnuVJdvT1Rk5VrEKnd4w@mail.gmail.com","threadId":"57898","inReplyTo":"220723.86h738qsr9.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v4 2/7] merge-resolve: abort if index does not match HEAD","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-26T01:58:53Z","receivedAt":"2022-07-26T01:59:08Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Jul 22, 2022 at 10:53 PM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> On Fri, Jul 22 2022, Elijah Newren wrote:\n>\n[...]\n> > So, ignoring the return code from diff-index is correct behavior here.\n> >\n> > Were you thinking this was a test script or something?\n>\n> We can leave this for now.\n>\n> But no. Whatever the merge driver is documenting as its normal return\n> values we really should be ferrying up abort() and segfault, per the\n> \"why do we miss...\" in:\n> https://lore.kernel.org/git/patch-v2-11.14-8cc6ab390db-20220720T211221Z-avarab@gmail.com/\n>\n> I.e. this is one of the cases in the test suite where we haven't closed\n> that gap, and could hide segfaults as a \"normal\" exit 2.\n>\n> So I think your v5 is fine as-is, but in general I'd be really\n> interested if you want to double-down on this view for the merge drivers\n> for some reason, because my current plan for addressing these blindspots\n> outlined in the above wouldn't work then...\n\nQuoting from there:\n\n> * We have in-tree shellscripts like \"git-merge-one-file.sh\" invoking\n> git commands, they'll usually return their own exit codes on \"git\"\n> failure, rather then ferrying up segfault or abort() exit code.\n>\n> E.g. these invocations in git-merge-one-file.sh leak, but aren't\n> reflected in the \"git merge\" exit code:\n>\n>src1=$(git unpack-file $2)\n>src2=$(git unpack-file $3)\n>\n> That case would be easily \"fixed\" by adding a line like this after\n> each assignment:\n>\n>test $? -ne 0 && exit $?\n>\n> But we'd then in e.g. \"t6407-merge-binary.sh\" run into\n> write_tree_trivial() in \"builtin/merge.c\" calling die() instead of\n> ferrying up the relevant exit code.\n\nSidenote, but I don't think t6407-merge-binary.sh calls into\nwrite_tree_trivial(). Doesn't in my testing, anyway.\n\nAre you really planning on auditing every line of git-bisect.sh,\ngit-merge*.sh, git-sh-setup.sh, git-submodule.sh, git-web--browse.sh,\nand others, and munging every area that invokes git to check the exit\nstatus? Yuck. A few points:\n\n * Will this really buy you anything? Don't we have other regression\ntests of all these commands (e.g. \"git unpack-file\") which ought to\nshow the same memory leaks? This seems like high lift, low value to\nme, and just fixing direct invocations in the regression tests is\nwhere the value comes. (If direct coverage is lacking in the\nregression tests, shouldn't the solution be to add coverage?)\n * Won't this be a huge review and support burden to maintain the\nextra checking?\n * Some of these scripts, such as git-merge-resolve.sh and\ngit-merge-octopus.sh are used as examples of e.g. merge drivers, and\ninvasive checks whose sole purpose is memory leak checking seems to\nrun counter to the purpose of being a simple example for users\n * Wouldn't using errexit and pipefail be an easier way to accomplish\nchecking the exit status (avoiding the problems from the last few\nbullets)? You'd still have to audit the code and write e.g.\nshutupgrep wrappers (since grep reports whether it found certain\npatterns in the input, rather than whether it was able to perform the\nsearch on the input, and we often only care about the latter), but it\nat least would automatically check future git invocations.\n * Are we running the risk of overloading special return codes (e.g.\n125 in git-bisect)\n\nI do still think that \"2\" is the correct return code for the\nshell-script merge strategies here, though I think it's feasible in\ntheir cases to change the documentation to relax the return code\nrequirements in such a way to allow those scripts to utilize errexit\nand pipefail.\n\n> >> * I wonder if bending over backwards to emit the exact message we\n> >> emitted before is worth it\n> >>\n> >> If you just make this something like (untested):\n> >>\n> >> {\n> >> gettext \"error: \" &&\n> >> gettextln \"Your local...\"\n> >> }\n> >>\n> >> You could re-use the translation from the *.c one (and the \"error: \" one\n> >> we'll get from usage.c).\n> >>\n> >> That leaves \"\\n %s\" as the difference, but we could just remove that\n> >> from the _() and emit it unconditionally, no?\n> >\n> > ??\n> >\n> > Copying a few lines from git-merge-octopus.sh to get the same fix it\n> > has is \"bending over backwards\"? That's what I call \"doing the\n> > easiest thing possible\" (and which _also_ has the benefit of being\n> > battle tested code), and then you describe a bunch of gymnastics as an\n> > alternative? I see your suggestion as running afoul of the objection\n> > you are raising, and the code I'm adding as being a solution to that\n> > particular objection. So this particular flag you are raising is\n> > confusing to me.\n>\n> I wasn't aware of some greater context vis-as-vis octopus,\n\nI didn't expect everyone to be, but that's why I put it in the commit\nmessage. ;-)\n"},{"id":"459954","messageId":"CABPp-BEMv6r=-Zs4AoachjZxWOVOL2DrZq7CpBEamCnq4vZTWg@mail.gmail.com","threadId":"57898","inReplyTo":"xmqqh7353ti1.fsf@gitster.g","subject":"Re: [PATCH v5 0/8] Fix merge restore state","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2022-07-26T01:59:55Z","receivedAt":"2022-07-26T02:00:19Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Jul 25, 2022 at 12:03 PM Junio C Hamano wrote:\n>\n> \"Elijah Newren via GitGitGadget\" writes:\n>\n> > This started as a simple series to fix restore_state() in builtin/merge.c,\n> > fixing an issue reported by ZheNing Hu[3]. It now fixes several bugs and has\n> > grown so much it's hard to call it simple. Anyway...\n>\n> Thanks. Unless we hear any more comments, let's start merging it\n> down to 'next' soonish.\n\nSounds good; let's plan on that.\n"},{"id":"459958","messageId":"CAOLTT8RCQtrPPcC4T8O57dN_ZxGOie7zP780xiuuyv4uwYaFqw@mail.gmail.com","threadId":"57898","inReplyTo":"pull.1231.v5.git.1658541198.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 0/8] Fix merge restore state","fromName":"ZheNing Hu","fromEmail":"adlternative@gmail.com","sentAt":"2022-07-26T04:03:52Z","receivedAt":"2022-07-26T04:04:09Z","isPatch":true,"sender":{"key":"adlternative@gmail.com","avatar":"https://avatars.githubusercontent.com/u/58138461?v=4"},"body":"Elijah Newren via GitGitGadget 于2022年7月23日周六 09:53写道：\n>\n> This started as a simple series to fix restore_state() in builtin/merge.c,\n> fixing an issue reported by ZheNing Hu[3]. It now fixes several bugs and has\n> grown so much it's hard to call it simple. Anyway...\n>\n\nThanks. This patch is good enough for me :)\n"},{"id":"459960","messageId":"220726.86k080nz3b.gmgdl@evledraar.gmail.com","threadId":"57898","inReplyTo":"CABPp-BE99NVBKRn9Uh3HMcgzV-egtmgnuVJdvT1Rk5VrEKnd4w@mail.gmail.com","subject":"Re: [PATCH v4 2/7] merge-resolve: abort if index does not match HEAD","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-26T06:35:36Z","receivedAt":"2022-07-26T06:54:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 25 2022, Elijah Newren wrote:\n\n> On Fri, Jul 22, 2022 at 10:53 PM Ævar Arnfjörð Bjarmason\n> wrote:\n>>\n>> On Fri, Jul 22 2022, Elijah Newren wrote:\n>>\n> [...]\n>> > So, ignoring the return code from diff-index is correct behavior here.\n>> >\n>> > Were you thinking this was a test script or something?\n>>\n>> We can leave this for now.\n>>\n>> But no. Whatever the merge driver is documenting as its normal return\n>> values we really should be ferrying up abort() and segfault, per the\n>> \"why do we miss...\" in:\n>> https://lore.kernel.org/git/patch-v2-11.14-8cc6ab390db-20220720T211221Z-avarab@gmail.com/\n>>\n>> I.e. this is one of the cases in the test suite where we haven't closed\n>> that gap, and could hide segfaults as a \"normal\" exit 2.\n>>\n>> So I think your v5 is fine as-is, but in general I'd be really\n>> interested if you want to double-down on this view for the merge drivers\n>> for some reason, because my current plan for addressing these blindspots\n>> outlined in the above wouldn't work then...\n>\n> Quoting from there:\n>\n>> * We have in-tree shellscripts like \"git-merge-one-file.sh\" invoking\n>> git commands, they'll usually return their own exit codes on \"git\"\n>> failure, rather then ferrying up segfault or abort() exit code.\n>>\n>> E.g. these invocations in git-merge-one-file.sh leak, but aren't\n>> reflected in the \"git merge\" exit code:\n>>\n>>src1=$(git unpack-file $2)\n>>src2=$(git unpack-file $3)\n>>\n>> That case would be easily \"fixed\" by adding a line like this after\n>> each assignment:\n>>\n>>test $? -ne 0 && exit $?\n>>\n>> But we'd then in e.g. \"t6407-merge-binary.sh\" run into\n>> write_tree_trivial() in \"builtin/merge.c\" calling die() instead of\n>> ferrying up the relevant exit code.\n>\n> Sidenote, but I don't think t6407-merge-binary.sh calls into\n> write_tree_trivial(). Doesn't in my testing, anyway.\n\nI haven't gone back & looked at that, but I vaguely recall that if you\n\"get past\" that one it was returning 128 somewhere, either directly or\nindirectly.\n\nIn any case, for the purposes of that summary it's a common pattern\nelsewhere...\n\n> Are you really planning on auditing every line of git-bisect.sh,\n> git-merge*.sh, git-sh-setup.sh, git-submodule.sh, git-web--browse.sh,\n> and others, and munging every area that invokes git to check the exit\n> status?\n\nNot really, but just for that change explaining why it's required to\nhave this log-on-the-side to munge the exit code.\n\nAlthough I think you might have not kept up with just how close we are\nto \"git rm\"-ing most of our non-trivial amounts of\nshellscript. git-{submodule,bisect}.sh is going away, hopefully in this\nrelease.\n\n*If* we go for that approach I'd think it would be relatively easy to\n add a helper to git-sh-setup.sh to wrap the various \"git\" callse.\n\n> Yuck. A few points:\n> * Will this really buy you anything? Don't we have other regression\n> tests of all these commands (e.g. \"git unpack-file\") which ought to\n> show the same memory leaks? This seems like high lift, low value to\n> me, and just fixing direct invocations in the regression tests is\n> where the value comes.\n\nIt's a long-tail problem, and we don't need to fix it all now. I'm just\ncommenting on it here because there's an *addition* of a hidden exit\ncode, and more importantly I wanted to clear up if you thought ferrying\nup abort() or segfaults wouldn't be desired in those cases.\n\n> (If direct coverage is lacking in the\n> regression tests, shouldn't the solution be to add coverage?)\n\n\nBut how do we find out what we're covering? Yes \"make coverage\", but\nthat's just going to give you all C lines we \"visit\", but you don't know\nif those lines were visited by parts of our test suite where we're\nchecking the exit code.\n\n> * Won't this be a huge review and support burden to maintain the\n> extra checking?\n\nI think the end result mostly makes things easier to deal with &\nmaintaine, e.g. consistently using &&-chaining, or test_cmp instead of\n\"test \"$(...)\" etc.\n\n> * Some of these scripts, such as git-merge-resolve.sh and\n> git-merge-octopus.sh are used as examples of e.g. merge drivers, and\n> invasive checks whose sole purpose is memory leak checking seems to\n> run counter to the purpose of being a simple example for users\n\nI'm not doing any of this now, but I'd think we could do something like\nthis (i.e. new helpers in git-sh-setup.sh):\n\t\n\tdiff --git a/git-merge-resolve.sh b/git-merge-resolve.sh\n\tindex 343fe7bccd0..f23344dfec8 100755\n\t--- a/git-merge-resolve.sh\n\t+++ b/git-merge-resolve.sh\n\t@@ -37,15 +37,16 @@ then\n\t \texit 2\n\t fi\n\t \n\t-git update-index -q --refresh\n\t-git read-tree -u -m --aggressive $bases $head $remotes || exit 2\n\t+run_git git update-index -q --refresh\n\t+run_git --exit-on-failure 2 git read-tree -u -m --aggressive $bases $head $remotes\n\t echo \"Trying simple merge.\"\n\t-if result_tree=$(git write-tree 2>/dev/null)\n\t+result_tree=$(git write-tree 2>/dev/null)\n\t+if check_last_git_cmd $?\n\t then\n\t \texit 0\n\t else\n\t \techo \"Simple merge failed, trying Automatic merge.\"\n\t-\tif git merge-index -o git-merge-one-file -a\n\t+\tif run_git git merge-index -o git-merge-one-file -a\n\t \tthen\n\t \t\texit 0\n\t \telse\n\n> * Wouldn't using errexit and pipefail be an easier way to accomplish\n> checking the exit status (avoiding the problems from the last few\n> bullets)? You'd still have to audit the code and write e.g.\n> shutupgrep wrappers (since grep reports whether it found certain\n> patterns in the input, rather than whether it was able to perform the\n> search on the input, and we often only care about the latter), but it\n> at least would automatically check future git invocations.\n\nSomewhat, but unless we're going to depend on \"bash\" I think we can at\nmost use those during development to locate various issues, e.g. as I\ndid in this series:\nhttps://lore.kernel.org/git/20210123130046.21975-1-avarab@gmail.com/\n\n> * Are we running the risk of overloading special return codes (e.g.\n> 125 in git-bisect)\n\nNo, we already deal with that as a potential problem in test_must_fail\nin the test suite, i.e. we just catch abort(), segfaults & the like. In\nbourn shells those are 134.\n\n> I do still think that \"2\" is the correct return code for the\n> shell-script merge strategies here, though I think it's feasible in\n> their cases to change the documentation to relax the return code\n> requirements in such a way to allow those scripts to utilize errexit\n> and pipefail.\n\nI think for any such interface it makes sense to exit with 0, 1 and 2 or\nwhatever during normal circumstances.\n\nBut if the program you just called segfaulted I think it makes sense to\ntreat that as an exception. I.e. it's not just that the merge failed,\nbut we should really abort() in the calling program too..\n\nAnyway, this is all quite academic at this point, but since you asked...\n"},{"id":"459961","messageId":"220726.86fsionynh.gmgdl@evledraar.gmail.com","threadId":"57898","inReplyTo":"CABPp-BFVZy+TtYtaKe1d25Ua3HSMrQpzkUDgfnzQvo8xzpD-3g@mail.gmail.com","subject":"Re: [PATCH v3 3/7] merge: do not abort early if one strategy fails to handle the merge","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-26T06:54:29Z","receivedAt":"2022-07-26T07:03:53Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Jul 25 2022, Elijah Newren wrote:\n\n> On Mon, Jul 25, 2022 at 4:06 AM Ævar Arnfjörð Bjarmason\n> wrote:\n>>\n> [...]\n>>\n>> I'm re-rolling ab/leak-check, and came up with the below (at the very\n>> end) to \"fix\" a report in builtin/merge.c, reading your commit message\n>> your fix seems obviously better.\n>>\n>> Mine's early WIP, and I e.g. didn't notice that I forgot to unlock the\n>> &lock file, which is correct.\n>>\n>> I *could* say \"that's not my problem\", i.e. we didn't unlock it before\n>> (we rely on atexit). The truth is I just missed it, but having said that\n>> it *is* true that we could do without it, or do it as a separate chaneg.\n>>\n>> I'm just posting my version below to help move yours forward, i.e. to\n>> show that someone else has carefully at least this part.\n>\n> \"has carefully ... at least this part\" ?\n>\n> I think you have a missing verb there.\n\n\"reviewed\", sorry.\n\n>> But it is worth noting from staring at the two that your version is\n>> mixing several different behavior changes into one, which *could* be\n>> split up (but whether you think that's worth it I leave to you).\n>>\n>> Maybe I'm the only one initially confused by it, and that's probably\n>> just from being mentally biased towards my own \"solution\". Those are (at\n>> least):\n>>\n>> 1. Before we didn't explicitly unlock() before exit(), but had atexit()\n>> do it, that could be a one-line first commit. This change is\n>> obviously good.\n>\n> That'd be fine. (Though at this point, I'd rather not mess with the\n> series more.)\n\nThat's completely fine, to be clear(er) what I'm walking through here is\nmy review process & potential edge cases I discovered, but I think your\npatch as-is does it all correctly.\n\n>> 2. A commit like mine could come next, i.e. we bug-for-bug do what we\n>> do do now, but just run the \"post-builtin\" logic when we return from\n>> cmd_merge().\n>>\n>> Doing it as an in-between would be some churn, as we'll need to get\n>> rid of \"early_exit\" again, but would allow us to incrementally move\n>> forward to...\n>\n> So, add a step that makes it glaringly obvious that the code is not\n> only buggy but totally at odds with itself?\n>\n> builtin/merge.c was designed to allow pluggable backends and to\n> automatically pick the \"best\" one if more than one is specified. We\n> had a bug in one line of code that defeated the design, by making it\n> not bother consulting beyond the first failed backend in some cases.\n> That's the bug I'm trying to address. Your patch would make the\n> inconsistency with the design both bigger and more obvious; I don't\n> see how it's a useful step to take.\n>\n> Now, the existing design might be questionable. In fact, I'm not sure\n> I like it. But I think we should either change the design, or fix\n> things in a way that improves towards the existing design.\n\nYes, I don't think it's useful, sorry about the confusion. I was walking\nthrough what the observable changes from the outside are here, and how\nthey *might* be split up.\n\nNot as a practical suggestion, but as a way to understand your commit...\n\n>> 3. ...then we'd say \"but it actually makes sense not to early abort\",\n>> i.e. you want to change this so that we'll run the logic between\n>> try_merge_strategy() exiting with 128 now and the return from\n>> cmd_merge().\n>>\n>> This bit is my main sticking point in reviewing your change,\n>> i.e. your \"a testcase for this is somewhat difficult\" somewhat\n>> addresses this, but (and maybe I'm wrong) it seems to me that\n>>\n>> Editing that code the post-image looks like this, with my\n>> commentary & most of the code removed, i.e. just focusing on the\n>> branches we do and don't potentially have tests for:\n>>\n>> /* Before this we fall through from ret == 128 (or ret == 2...) */\n>> if (automerge_was_ok) { // not tested?\n>> if (!best_strategy) {\n>> // we test this...\n>> if (use_strategies_nr > 1)\n>> // And this: _(\"No merge strategy handled the merge.\\n\"));\n>> else\n>> // And this: _(\"Merge with strategy %s failed.\\n\"),\n>> } else if (best_strategy == wt_strategy)\n>> // but not this?\n>> else\n>> // Or this, where we e.g. say \"Rewinding the tree to pristene...\"?\n>>\n>> if (squash) {\n>> // this?\n>> } else\n>> // this? (probably, yes)\n>> write_merge_state(remoteheads);\n>>\n>> if (merge_was_ok)\n>> // this? (probably, yes, we just don't grep it?)\n>> else\n>> // this? maybe yes because it's covered by the\n>> // \"failed\" above too?\n>> ret = suggest_conflicts();\n>>\n>> done:\n>> if (!automerge_was_ok) {\n>> // this? ditto the first \"not tested?\"\n>> }\n>>\n>> I.e. are you confident that we want to continue now in these various\n>> cases, where we have squash, !automerge_was_ok etc. I think it would\n>> be really useful to comment on (perhaps by amending the above\n>> pseudocode) what test cases we're not testing / test already etc.\n>\n> To be honest, I'm confused by what looks like a make-work project.\n> Perhaps if I understood your frame of reference better, maybe they\n> wouldn't bother me, but there's a few things here that seem a little\n> funny to me.\n>\n> You've highlighted that you are worried about the case where ret is 2\n> (or 128) at the point of all these branches in question. However,\n> three of those branches can almost trivially be deduced to never be\n> taken unless ret is 0. One of the other codepaths, for freeing\n> memory, is correct regardless of the value of ret -- the memory is\n> conditionally freed earlier and the \"if\"-check exists only to avoid a\n> double free (and checking the recent commit message where those lines\n> were added would explain this, though I'm not sure why it'd even need\n> explaining separately for e.g. ret == 2 compared to any other value).\n> Three of the other code paths involve nothing more than print\n> statements. Now, there are many codepaths you highlighted, and\n> perhaps there are some where it's not trivial to determine whether\n> they are okay in combination with a different return value. And it\n> may also be easy to miss some of the \"almost trivial\" cases. I'd\n> understand better if you asked about the tougher ones or only some of\n> the easier ones, but it feels like you didn't try to check any of them\n> and instead wanted me to just spend time commenting on every single\n> code branch?\n>\n> I hope that doesn't come across harshly. I'm just struggling to\n> understand where the detailed request is coming from.\n>\n> However, perhaps I can obviate the whole set of requests by just\n> pointing out that I don't think any of them are relevant. The premise\n> for the audit request seems to be that you are worrying that the\n> change from die() to \"return 2\" (or 128) in try_merge_strategy() will\n> result in the calling code getting into a state it has never\n> experienced before which might uncover latent bugs. We can trivially\n> point out that it's not a new state, though: such return values were\n> already possible from try_merge_strategy() via the final line of code\n> in the function -- namely, the \"return try_merge_command(...)\" code\n> path. And a return value of 2 from try_merge_command() and\n> try_merge_strategy() isn't merely theoretical either -- you can easily\n> trigger it multiple ways; the easiest is perhaps by passing `-s\n> octopus` when doing a non-octopus merge. (You can also make the\n> testcase more complex by combining that with as many other additional\n> merge strategies as you want, e.g. \"git merge -s octopus -s resolve -s\n> recursive -s mySpecialStrategy $BRANCH\". You can also move the\n> octopus to the end if you want to test what happens if it is tried\n> last. Lots of possibilities exist).\n\nThanks, that all makes sense & addresses any questions I had, which were\njust if we were sure that this behavior was expected.\n\n>> 4. Having done all that (or maybe this can't be split up / needs to\n>> come earlier) you say that we'd like to not generically call this\n>> exit state 128, but have it under the \"exit(2)\" umbrella.\n>\n> I don't see how reading your set of steps 1-4 logically restores the\n> design of builtin/merge.c, though. An example: what if the user ran\n> \"git merge -s resolve -s recursive $BRANCH\", and `resolve` handled the\n> merge but had some conflicts, while `recursive` just failed and\n> returned a value of clean < 0? In such a case, builtin/merge.c is\n> supposed to restore the tree to pristine after attempting the\n> recursive backend, and then redo using the best_strategy (which would\n> be `resolve` in this example case). Your steps 1-4 never address such\n> a thing. Your steps might incidentally address such a case as a side\n> effect of their implementation, but that's not at all clear. Since\n> the whole point is fixing the code to match the existing design, it\n> seems really odd to split things into a set of steps that obscures\n> that fix.\n\nYou know more about the wider context here, I was just poking at the\nnarrow code change & seeing what the side-effects were of the changes\nbeing made.\n\n>> Again, all just food for thought, and a way to step-by-step go through\n>> how I came about reviewing this in detail, I hope it and the below\n>> version I came up with before seeing yours helps.\n>>\n>> P.s.: The last paragraph in my commit message does not point to some\n>> hidden edge case in the code behavior here, it's just that clang/gcc are\n>> funny about exit() and die() control flow when combined with\n>> -fsanitize=address and higher optimization levels.\n>>\n>> -- >8 --\n>> Subject: [PATCH] merge: return, don't use exit()\n>>\n>> Change some of the builtin/merge.c code added in f241ff0d0a9 (prepare\n>> the builtins for a libified merge_recursive(), 2016-07-26) to ferry up\n>> an \"early return\" state, rather than having try_merge_strategy() call\n>> exit() itself.\n>>\n>> This is a follow-up to dda31145d79 (Merge branch\n>> 'ab/usage-die-message' into gc/branch-recurse-submodules-fix,\n>> 2022-03-31).\n>>\n>> The only behavior change here is that we'll now properly catch other\n>> issues on our way out, see e.g. [1] and the interaction with /dev/full\n>> for an example.\n>>\n>> The immediate reason to do this change is because it's one of the\n>> cases where clang and gcc's SANITIZE=leak behavior differs. Under\n>> clang we don't detect that \"t/t6415-merge-dir-to-symlink.sh\" triggers\n>> a leak, but gcc spots it.\n>>\n>> 1. https://lore.kernel.org/git/87im2n3gje.fsf@evledraar.gmail.com/\n>>\n>> Signed-off-by: Ævar Arnfjörð Bjarmason \n>> ---\n>> builtin/merge.c | 27 +++++++++++++++++++++------\n>> 1 file changed, 21 insertions(+), 6 deletions(-)\n>>\n>> diff --git a/builtin/merge.c b/builtin/merge.c\n>> index 23170f2d2a6..a8d5d04f622 100644\n>> --- a/builtin/merge.c\n>> +++ b/builtin/merge.c\n>> @@ -709,10 +709,12 @@ static void write_tree_trivial(struct object_id *oid)\n>>\n>> static int try_merge_strategy(const char *strategy, struct commit_list *common,\n>> struct commit_list *remoteheads,\n>> - struct commit *head)\n>> + struct commit *head, int *early_exit)\n>> {\n>> const char *head_arg = \"HEAD\";\n>>\n>> + *early_exit = 0;\n>> +\n>> if (refresh_and_write_cache(REFRESH_QUIET, SKIP_IF_UNCHANGED, 0) < 0)\n>> return error(_(\"Unable to write index.\"));\n>>\n>> @@ -754,8 +756,10 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,\n>> else\n>> clean = merge_recursive(&o, head, remoteheads->item,\n>> reversed, &result);\n>> - if (clean < 0)\n>> - exit(128);\n>> + if (clean < 0) {\n>> + *early_exit = 1;\n>> + return 128;\n>> + }\n>> if (write_locked_index(&the_index, &lock,\n>> COMMIT_LOCK | SKIP_IF_UNCHANGED))\n>> die(_(\"unable to write %s\"), get_index_file());\n>> @@ -1665,6 +1669,8 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n>>\n>> for (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n>> int ret, cnt;\n>> + int early_exit;\n>> +\n>> if (i) {\n>> printf(_(\"Rewinding the tree to pristine...\\n\"));\n>> restore_state(&head_commit->object.oid, &stash);\n>> @@ -1680,7 +1686,10 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n>>\n>> ret = try_merge_strategy(use_strategies[i]->name,\n>> common, remoteheads,\n>> - head_commit);\n>> + head_commit, &early_exit);\n>> + if (early_exit)\n>> + goto done;\n>> +\n>> /*\n>> * The backend exits with 1 when conflicts are\n>> * left to be resolved, with 2 when it does not\n>> @@ -1732,12 +1741,18 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n>> } else if (best_strategy == wt_strategy)\n>> ; /* We already have its result in the working tree. */\n>> else {\n>> + int new_ret, early_exit;\n>> +\n>> printf(_(\"Rewinding the tree to pristine...\\n\"));\n>> restore_state(&head_commit->object.oid, &stash);\n>> printf(_(\"Using the %s strategy to prepare resolving by hand.\\n\"),\n>> best_strategy);\n>> - try_merge_strategy(best_strategy, common, remoteheads,\n>> - head_commit);\n>> + new_ret = try_merge_strategy(best_strategy, common, remoteheads,\n>> + head_commit, &early_exit);\n>> + if (early_exit) {\n>> + ret = new_ret;\n>> + goto done;\n>> + }\n>\n> Incidentally, this is essentially dead code being added in this last\n> hunk. This final `else` block can only be triggered when\n> best_strategy has been set, and best_strategy will only be set after a\n> merge strategy works (possibly with conflicts, but was at least\n> appropriate for the problem). Anyway, by the point of this `else`\n> block, we've run multiple strategies already, at least one worked, and\n> the \"best\" one was not the last one tried. So, at this point, we\n> simply rewind the tree and rerun the known-working merge strategy. (I\n> guess you could say that maybe the merge strategy might not return the\n> same result despite being given the same inputs, so theoretically\n> early_exit could come back true and this hunk isn't quite dead. Just\n> mostly dead, I guess.)\n\nThe idea here was to leave the \"is it really dead?\" question aside, and\njust move the code towards running the post-builtin code we run when we\ndon't call an early exit().\n\n"},{"id":"472915","messageId":"ZABNceY3cSWNb75u@F407C5W6RY.office.atlassian.com","threadId":"57898","inReplyTo":"xmqq35eul95b.fsf@gitster.g","subject":"Re: [PATCH v3 6/7] merge: ensure we can actually restore pre-merge state","fromName":"Ben Humphreys","fromEmail":"behumphreys@atlassian.com","sentAt":"2023-03-02T07:17:05Z","receivedAt":"2023-03-02T07:17:16Z","isPatch":true,"sender":{"key":"behumphreys@atlassian.com","avatar":"https://gravatar.com/avatar/5cb49575477c33416331403d493bb02758031b2a4e5183491395b02cea9e8236?d=mp&s=160"},"body":"Hi Junio / Elijah,\n\nOn Thu, Jul 21, 2022 at 09:31:44AM -0700, Junio C Hamano wrote:\n> \"Elijah Newren via GitGitGadget\" writes:\n> \n> > diff --git a/builtin/merge.c b/builtin/merge.c\n> > index f807bf335bd..11bb4bab0a1 100644\n> > --- a/builtin/merge.c\n> > +++ b/builtin/merge.c\n> > @@ -1686,12 +1686,12 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> > \t * tree in the index -- this means that the index must be in\n> > \t * sync with the head commit. The strategies are responsible\n> > \t * to ensure this.\n> > +\t *\n> > +\t * Stash away the local changes so that we can try more than one\n> > +\t * and/or recover from merge strategies bailing while leaving the\n> > +\t * index and working tree polluted.\n> > \t */\n> \n> Makes sense. We may want to special-case strategies that are known\n> not to have the buggy \"leave contaminated tree when bailing out\"\n> behaviour to avoid waste. I expect that more than 99.99% of the\n> time people are feeding a single other commit to ort or recursive,\n> and if these are known to be safe, a lot will be saved by not saving\n> \"just in case\". But that can be left for later, after the series\n> solidifies.\n\nThis may stretch your memory a bit since the above was many months ago,\nbut I'm wondering if you know of any effort since to build the above\ndescribed optimisations?\n\nWe've seen when Git 2.38.0 (which introduced this change) is used with\nBitbucket Server it results in a severe performance regression due to an\nsharp increase in disk and CPU load. Our code that tests the mergeability\nof a pull request is one such affected codepath.\n\nIf there isn't any existing efforts to build the optimisations you\nmention above I will have a shot at it.\n\n> > -\tif (use_strategies_nr == 1 ||\n> > -\t /*\n> > -\t * Stash away the local changes so that we can try more than one.\n> > -\t */\n> > -\t save_state(&stash))\n> > +\tif (save_state(&stash))\n> > \t\toidclr(&stash);\n> > \n> > \tfor (i = 0; !merge_was_ok && i < use_strategies_nr; i++) {\n\nBest Regards,\nBen Humphreys\n"},{"id":"472920","messageId":"CABPp-BEJmnAB4QX_HMnNaZ9vEh_mKiqka_uZRm_E_LNuOBiFsg@mail.gmail.com","threadId":"57898","inReplyTo":"ZABNceY3cSWNb75u@F407C5W6RY.office.atlassian.com","subject":"Re: [PATCH v3 6/7] merge: ensure we can actually restore pre-merge state","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2023-03-02T15:35:30Z","receivedAt":"2023-03-02T15:35:49Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi Ben,\n\nOn Wed, Mar 1, 2023 at 11:17 PM Ben Humphreys wrote:\n>\n> Hi Junio / Elijah,\n>\n> On Thu, Jul 21, 2022 at 09:31:44AM -0700, Junio C Hamano wrote:\n> > \"Elijah Newren via GitGitGadget\" writes:\n> >\n> > > diff --git a/builtin/merge.c b/builtin/merge.c\n> > > index f807bf335bd..11bb4bab0a1 100644\n> > > --- a/builtin/merge.c\n> > > +++ b/builtin/merge.c\n> > > @@ -1686,12 +1686,12 @@ int cmd_merge(int argc, const char **argv, const char *prefix)\n> > > * tree in the index -- this means that the index must be in\n> > > * sync with the head commit. The strategies are responsible\n> > > * to ensure this.\n> > > + *\n> > > + * Stash away the local changes so that we can try more than one\n> > > + * and/or recover from merge strategies bailing while leaving the\n> > > + * index and working tree polluted.\n> > > */\n> >\n> > Makes sense. We may want to special-case strategies that are known\n> > not to have the buggy \"leave contaminated tree when bailing out\"\n> > behaviour to avoid waste. I expect that more than 99.99% of the\n> > time people are feeding a single other commit to ort or recursive,\n> > and if these are known to be safe, a lot will be saved by not saving\n> > \"just in case\". But that can be left for later, after the series\n> > solidifies.\n>\n> This may stretch your memory a bit since the above was many months ago,\n> but I'm wondering if you know of any effort since to build the above\n> described optimisations?\n>\n> We've seen when Git 2.38.0 (which introduced this change) is used with\n> Bitbucket Server it results in a severe performance regression due to an\n> sharp increase in disk and CPU load. Our code that tests the mergeability\n> of a pull request is one such affected codepath.\n>\n> If there isn't any existing efforts to build the optimisations you\n> mention above I will have a shot at it.\n\nI've got bad news for you and great news for you.\n\nThe bad news: there have not yet been any efforts to build these\noptimizations mentioned above.\n\nThe great news: the fact that this affects you means you are using\nnon-bare clones in your mergeability checks, and being forced with\nevery merge to first checkout the appropriate branch, and pay for the\npenalty of updating both the index and the working tree both in that\ncheckout and during the merge (and perhaps in doing a hard reset\nafterwards) in your mergeability check, despite the fact that a\nmergeability check really only needs a boolean: \"does it merge\ncleanly?\". Doing a full worktree-tied merge like this is really\nexpensive, and while the above Git changes may have made it even more\nexpensive for you, the real savings comes from switching to a bare\nclone and not writing any working tree files or the index. That's\navailable via running `git merge-tree`; see the documentation for the\n--write-tree option in particular. GitHub switched over to it last\nyear and GitLab should be switching soon (or may have already\ncompleted it; I haven't checked in a bit).\n\nYou are, of course, more than welcome to build the optimizations Junio\nalludes to. It'd help out various end users. But for improving\nserver side operations, I think switching to `git merge-tree` would\nprovide you _much_ bigger benefits.\n\n\nHope that helps,\nElijah\n"},{"id":"472922","messageId":"xmqqmt4vktx7.fsf@gitster.g","threadId":"57898","inReplyTo":"CABPp-BEJmnAB4QX_HMnNaZ9vEh_mKiqka_uZRm_E_LNuOBiFsg@mail.gmail.com","subject":"Re: [PATCH v3 6/7] merge: ensure we can actually restore pre-merge state","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-03-02T16:19:16Z","receivedAt":"2023-03-02T16:19:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n> The great news: the fact that this affects you means you are using\n> non-bare clones in your mergeability checks, and being forced with\n> every merge to first checkout the appropriate branch, and pay for the\n> penalty of updating both the index and the working tree both in that\n> checkout and during the merge (and perhaps in doing a hard reset\n> afterwards) in your mergeability check, despite the fact that a\n> mergeability check really only needs a boolean: \"does it merge\n> cleanly?\". Doing a full worktree-tied merge like this is really\n> expensive, and while the above Git changes may have made it even more\n> expensive for you, the real savings comes from switching to a bare\n> clone and not writing any working tree files or the index. That's\n> available via running `git merge-tree`; see the documentation for the\n> --write-tree option in particular. GitHub switched over to it last\n> year and GitLab should be switching soon (or may have already\n> completed it; I haven't checked in a bit).\n\nNice.\n"},{"id":"473006","messageId":"CANaDLWLaOm-oow_-wMS3w0z=5O3uau3xZrXHboxLC7CZV+iFQQ@mail.gmail.com","threadId":"57898","inReplyTo":"xmqqmt4vktx7.fsf@gitster.g","subject":"Re: [PATCH v3 6/7] merge: ensure we can actually restore pre-merge state","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2023-03-04T16:18:40Z","receivedAt":"2023-03-04T16:18:57Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> I think switching to `git merge-tree` would provide you _much_ bigger benefits.\n\nSupporting evidence: we have an automerge pipeline running\non our gigantic monolith at Salesforce to keep some shared\nbranches automatically up-to-date with their upstream. We first\nbuilt it with a version of Git older than 2.38, and we were doing\ncheckouts and running good old 'git merge' and 'git push'\ncommands. We didn't measure since it wasn't in production yet,\nbut from memory, each merge could take maybe 30s to a minute,\nthe checkouts could be really heavy sometimes. And of course,\nafter our initial fetch (which we still do of course), there was also an\nincompressible multi-minute initial checkout.\n\nWe updated our pipeline to run Git 2.39, and we switched to doing\nno checkout and running 'git merge-tree', followed by 'git commit-tree',\nand then 'git update-ref'. Now, each of those is about 5 seconds,\nincluding re-fetching the branches before starting in case they very\nrecently changed. It made a gigantic difference to us, thanks a lot\nto everyone who worked on this.\n"},{"id":"473088","messageId":"ZAZm51aZEfuTkdTm@F407C5W6RY.office.atlassian.com","threadId":"57898","inReplyTo":"CABPp-BEJmnAB4QX_HMnNaZ9vEh_mKiqka_uZRm_E_LNuOBiFsg@mail.gmail.com","subject":"Re: [PATCH v3 6/7] merge: ensure we can actually restore pre-merge state","fromName":"Ben Humphreys","fromEmail":"behumphreys@atlassian.com","sentAt":"2023-03-06T22:19:19Z","receivedAt":"2023-03-06T22:19:31Z","isPatch":true,"sender":{"key":"behumphreys@atlassian.com","avatar":"https://gravatar.com/avatar/5cb49575477c33416331403d493bb02758031b2a4e5183491395b02cea9e8236?d=mp&s=160"},"body":"On Thu, Mar 02, 2023 at 07:35:30AM -0800, Elijah Newren wrote:\n> I've got bad news for you and great news for you.\n> \n> The bad news: there have not yet been any efforts to build these\n> optimizations mentioned above.\n> \n> The great news: the fact that this affects you means you are using\n> non-bare clones in your mergeability checks, and being forced with\n> every merge to first checkout the appropriate branch, and pay for the\n> penalty of updating both the index and the working tree both in that\n> checkout and during the merge (and perhaps in doing a hard reset\n> afterwards) in your mergeability check, despite the fact that a\n> mergeability check really only needs a boolean: \"does it merge\n> cleanly?\". Doing a full worktree-tied merge like this is really\n> expensive, and while the above Git changes may have made it even more\n> expensive for you, the real savings comes from switching to a bare\n> clone and not writing any working tree files or the index. That's\n> available via running `git merge-tree`; see the documentation for the\n> --write-tree option in particular. GitHub switched over to it last\n> year and GitLab should be switching soon (or may have already\n> completed it; I haven't checked in a bit).\n> \n> You are, of course, more than welcome to build the optimizations Junio\n> alludes to. It'd help out various end users. But for improving\n> server side operations, I think switching to `git merge-tree` would\n> provide you _much_ bigger benefits.\n\nMany thanks for the detailed reply Elijah; indeed the good news\noutweighs the bad news! I've started migrating to merge-tree and it\nlooks great. Once complete I might take a look at the other\noptimizations anyway, as a fun project.\n\nThanks again!\n\nBest Regards,\nBen Humphreys\n"}]}