{"thread":{"id":"24670","subject":"[PATCHv2 0/2] Fix spurious conflicts with pull --rebase","startedAt":"2010-08-08T19:04:44Z","lastAt":"2010-08-08T21:43:58Z","messageCount":8,"participants":["Elijah Newren","Ævar Arnfjörð Bjarmason","Johannes Sixt","Jonathan Nieder"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"147450","messageId":"1281294286-27709-1-git-send-email-newren@gmail.com","threadId":"24670","inReplyTo":null,"subject":"[PATCHv2 0/2] Fix spurious conflicts with pull --rebase","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-08T19:04:44Z","receivedAt":"2010-08-08T19:04:44Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"This patch series fixes spurious conflict issues with git pull\n--rebase for the case where the upstream repository is *not* rebased.\n(There is no change in the case where the upstream repository is\nrebased.)\n\nIn c85c79279d and d44e71261f, the call to git-rebase was modified in\nan effort to reduce the number of commits being reapplied, by trying\nto avoid commits that upstream already had or has.  It was\nspecifically designed with an upstream that is rebased in mind.\nUnfortunately, it had two side effects for the non-rebased upstream\ncase: (1) it prevented detecting if \"local\" patches are already\nupstream, and (2) it could in some cases cause more patches known to\nbe upstream to be reapplied rather than less.  This series fixes both\nof these issues for the non-rebased upstream case.  See the commit\nmessage of the second patch for details.\n\nIt's worth noting that issue (1) above also affects the case where the\nupstream repository has been rebased, which this patch series does not\naddress.  As far as I can tell, fixing it would require changes\n(including new syntax) to format-patch to allow it to be told what\n'upstream' is, and some changes to git-pull.sh/git-rebase.sh to pass\nit this information.\n\nChanges since the last posting of these series:\n  * Rewrite cover letter and commit messages for clarity\n  * Avoid sed -i in the testsuite to improve portability\n\nElijah Newren (2):\n  t5520-pull: Add testcases showing spurious conflicts from git pull\n    --rebase\n  pull --rebase: Avoid spurious conflicts and reapplying unnecessary\n    patches\n\n git-pull.sh     |   34 ++++++++++++++++++++-----------\n t/t5520-pull.sh |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 80 insertions(+), 12 deletions(-)\n\n-- \n1.7.2.1\n"},{"id":"147451","messageId":"1281294286-27709-2-git-send-email-newren@gmail.com","threadId":"24670","inReplyTo":"1281294286-27709-1-git-send-email-newren@gmail.com","subject":"[PATCHv2 1/2] t5520-pull: Add testcases showing spurious conflicts from git pull --rebase","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-08T19:04:45Z","receivedAt":"2010-08-08T19:04:45Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\n t/t5520-pull.sh |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 58 insertions(+), 0 deletions(-)\n\ndiff --git a/t/t5520-pull.sh b/t/t5520-pull.sh\nindex 319e389..1624dd3 100755\n--- a/t/t5520-pull.sh\n+++ b/t/t5520-pull.sh\n@@ -160,4 +160,62 @@ test_expect_success 'pull --rebase works on branch yet to be born' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'setup for detecting upstreamed changes' '\n+\tmkdir src &&\n+\t(cd src &&\n+\t git init &&\n+\t for i in $(seq 1 10); do echo $i; done > stuff &&\n+\t git add stuff &&\n+\t git commit -m \"Initial revision\"\n+\t) &&\n+\tgit clone src dst &&\n+\t(cd src &&\n+\t perl -pi -e s/5/43/ stuff &&\n+\t git commit -a -m \"5->43\" &&\n+\t perl -pi -e s/6/42/ stuff &&\n+\t git commit -a -m \"Make it bigger\" &&\n+\t correct=$(git rev-parse HEAD)\n+\t) &&\n+\t(cd dst &&\n+\t perl -pi -e s/5/43/ stuff &&\n+\t git commit -a -m \"Independent discovery of 5->43\"\n+\t)\n+'\n+\n+test_expect_failure 'git pull --rebase detects upstreamed changes' '\n+\t(cd dst &&\n+\t git pull --rebase &&\n+\t test -z \"$(git ls-files -u)\"\n+\t)\n+'\n+\n+test_expect_success 'setup for avoiding reapplying old patches' '\n+\t(cd dst &&\n+\t (git rebase --abort || true) &&\n+\t git reset --hard origin/master\n+\t) &&\n+\tgit clone --bare src src-replace.git &&\n+\trm -rf src &&\n+\tmv src-replace.git src &&\n+\t(cd dst &&\n+\t perl -pi -e s/2/22/ stuff &&\n+\t git commit -a -m \"Change 2\" &&\n+\t perl -pi -e s/3/33/ stuff &&\n+\t git commit -a -m \"Change 3\" &&\n+\t perl -pi -e s/4/44/ stuff &&\n+\t git commit -a -m \"Change 4\" &&\n+\t git push &&\n+\n+\t perl -pi -e s/44/55/ stuff &&\n+\t git commit --amend -a -m \"Modified Change 4\"\n+\t)\n+'\n+\n+test_expect_failure 'git pull --rebase does not reapply old patches' '\n+\t(cd dst &&\n+\t (git pull --rebase || true) &&\n+\t test 1 = $(find .git/rebase-apply -name \"000*\" | wc -l)\n+\t)\n+'\n+\n test_done\n-- \n1.7.2.1\n"},{"id":"147452","messageId":"1281294286-27709-3-git-send-email-newren@gmail.com","threadId":"24670","inReplyTo":"1281294286-27709-1-git-send-email-newren@gmail.com","subject":"[PATCHv2 2/2] pull --rebase: Avoid spurious conflicts and reapplying unnecessary patches","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-08T19:04:46Z","receivedAt":"2010-08-08T19:04:46Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Prior to c85c79279df2c8a583d95449d1029baba41f8660, pull --rebase would run\n  git rebase $merge_head\nwhich resulted in a call to\n  git format-patch ... --ignore-if-in-upstream $merge_head..$cur_branch\n\nThis resulted in patches from $merge_head..$cur_branch being applied, as\nlong as they did not already exist in $cur_branch..$merge_head.\nUnfortunately, when upstream is rebased, $merge_head..$cur_branch also\nrefers to \"old\" commits that have already been rebased upstream, meaning\nthat many patches that were already fixed upstream would be reapplied.\nThis could result in many spurious conflicts, as well as reintroduce\npatches that were intentionally dropped upstream.\n\nSo the algorithm was changed in c85c79279df2c8a583d95449d1029baba41f8660\nand d44e71261f91d3cc81293e0976bb40daa8abb583.  Defining $old_remote_ref to\nbe the most recent entry in the reflog for @{upstream} that is an ancestor\nof $cur_branch, pull --rebase was changed to run\n  git rebase --onto $merge_head $old_remote_ref\nwhich results in a call to\n  git format-patch ... --ignore-if-in-upstream $old_remote_ref..$cur_branch\n\nThe whole point of this change was to reduce the number of commits being\nreapplied, by avoiding commits that upstream already has or had.\n\nIn the rebased upstream case, this change achieved that purpose.  It is\nworth noting, though, that since $old_remote_ref is always an ancestor of\n$cur_branch (by its definition), format-patch will not know what upstream\nis and thus will not be able to determine if any patches are already\nupstream; they will all be reapplied.\n\nIn the non-rebased upstream case, this new form is usually the same as the\noriginal code but in some cases $old_remote_ref can be an ancestor of\n   $(git merge-base $merge_head $cur_branch)\nmeaning that instead of avoiding reapplying commits that upstream already\nhas, it actually includes more such commits.  Combined with the fact that\nformat-patch can no longer detect commits that are already upstream (since\nit is no longer told what upstream is), results in lots of confusion for\nusers (e.g. \"git is giving me lots of conflicts in stuff I didn't even\nchange since my last push.\")\n\nFix the non-rebased upstream case by ignoring $old_remote_ref whenever it\nis contained in $(git merge-base $merge_head $cur_branch).  This should\nhave no affect on the rebased upstream case.\n\nSigned-off-by: Elijah Newren <newren@gmail.com>\n---\nNote that 24 lines of the patch are simply due to moving 12 lines; I\nconsidered leaving those 12 lines where they were, but it'd split the\ncode that handles the setting of oldremoteref, which I thought should\nbe together.  I couldn't place the new code earlier, because it\ndepends on the definition of $merge_head.  I'd be happy to reshuffle\ndifferently if anyone has any strong preferences.\n\n git-pull.sh     |   34 ++++++++++++++++++++++------------\n t/t5520-pull.sh |    4 ++--\n 2 files changed, 24 insertions(+), 14 deletions(-)\n\ndiff --git a/git-pull.sh b/git-pull.sh\nindex a09a44e..54da07b 100755\n--- a/git-pull.sh\n+++ b/git-pull.sh\n@@ -206,18 +206,6 @@ test true = \"$rebase\" && {\n \t\tgit diff-index --ignore-submodules --cached --quiet HEAD -- ||\n \t\tdie \"refusing to pull with rebase: your working tree is not up-to-date\"\n \tfi\n-\toldremoteref= &&\n-\t. git-parse-remote &&\n-\tremoteref=\"$(get_remote_merge_branch \"$@\" 2>/dev/null)\" &&\n-\toldremoteref=\"$(git rev-parse -q --verify \"$remoteref\")\" &&\n-\tfor reflog in $(git rev-list -g $remoteref 2>/dev/null)\n-\tdo\n-\t\tif test \"$reflog\" = \"$(git merge-base $reflog $curr_branch)\"\n-\t\tthen\n-\t\t\toldremoteref=\"$reflog\"\n-\t\t\tbreak\n-\t\tfi\n-\tdone\n }\n orig_head=$(git rev-parse -q --verify HEAD)\n git fetch $verbosity $progress $dry_run --update-head-ok \"$@\" || exit 1\n@@ -273,6 +261,28 @@ then\n \texit\n fi\n \n+if test true = \"$rebase\"\n+then\n+\toldremoteref= &&\n+\t. git-parse-remote &&\n+\tremoteref=\"$(get_remote_merge_branch \"$@\" 2>/dev/null)\" &&\n+\toldremoteref=\"$(git rev-parse -q --verify \"$remoteref\")\" &&\n+\tfor reflog in $(git rev-list -g $remoteref 2>/dev/null)\n+\tdo\n+\t\tif test \"$reflog\" = \"$(git merge-base $reflog $curr_branch)\"\n+\t\tthen\n+\t\t\toldremoteref=\"$reflog\"\n+\t\t\tbreak\n+\t\tfi\n+\tdone\n+\n+\to=$(git show-branch --merge-base $curr_branch $merge_head $oldremoteref)\n+\tif test \"$oldremoteref\" = \"$o\"\n+\tthen\n+\t\tunset oldremoteref\n+\tfi\n+fi\n+\n merge_name=$(git fmt-merge-msg $log_arg <\"$GIT_DIR/FETCH_HEAD\") || exit\n case \"$rebase\" in\n true)\ndiff --git a/t/t5520-pull.sh b/t/t5520-pull.sh\nindex 1624dd3..ec4d36a 100755\n--- a/t/t5520-pull.sh\n+++ b/t/t5520-pull.sh\n@@ -182,7 +182,7 @@ test_expect_success 'setup for detecting upstreamed changes' '\n \t)\n '\n \n-test_expect_failure 'git pull --rebase detects upstreamed changes' '\n+test_expect_success 'git pull --rebase detects upstreamed changes' '\n \t(cd dst &&\n \t git pull --rebase &&\n \t test -z \"$(git ls-files -u)\"\n@@ -211,7 +211,7 @@ test_expect_success 'setup for avoiding reapplying old patches' '\n \t)\n '\n \n-test_expect_failure 'git pull --rebase does not reapply old patches' '\n+test_expect_success 'git pull --rebase does not reapply old patches' '\n \t(cd dst &&\n \t (git pull --rebase || true) &&\n \t test 1 = $(find .git/rebase-apply -name \"000*\" | wc -l)\n-- \n1.7.2.1\n"},{"id":"147454","messageId":"AANLkTikhsFfCfmR2V8iSRvU73B5yW=_kqJAEfAGgxpLY@mail.gmail.com","threadId":"24670","inReplyTo":"1281294286-27709-2-git-send-email-newren@gmail.com","subject":"Re: [PATCHv2 1/2] t5520-pull: Add testcases showing spurious conflicts from git pull --rebase","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-08-08T20:01:52Z","receivedAt":"2010-08-08T20:01:52Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Sun, Aug 8, 2010 at 19:04, Elijah Newren <newren@gmail.com> wrote:\n>\n> Signed-off-by: Elijah Newren <newren@gmail.com>\n> ---\n>  t/t5520-pull.sh |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++\n>  1 files changed, 58 insertions(+), 0 deletions(-)\n>\n> diff --git a/t/t5520-pull.sh b/t/t5520-pull.sh\n> index 319e389..1624dd3 100755\n> --- a/t/t5520-pull.sh\n> +++ b/t/t5520-pull.sh\n> @@ -160,4 +160,62 @@ test_expect_success 'pull --rebase works on branch yet to be born' '\n>        test_cmp expect actual\n>  '\n>\n> +test_expect_success 'setup for detecting upstreamed changes' '\n> +       mkdir src &&\n> +       (cd src &&\n> +        git init &&\n> +        for i in $(seq 1 10); do echo $i; done > stuff &&\n> +        git add stuff &&\n> +        git commit -m \"Initial revision\"\n> +       ) &&\n> +       git clone src dst &&\n> +       (cd src &&\n> +        perl -pi -e s/5/43/ stuff &&\n> +        git commit -a -m \"5->43\" &&\n> +        perl -pi -e s/6/42/ stuff &&\n\nPlease use sed so the test doesn't depend on perl being present.\n\n> +        git commit -a -m \"Make it bigger\" &&\n> +        correct=$(git rev-parse HEAD)\n> +       ) &&\n> +       (cd dst &&\n> +        perl -pi -e s/5/43/ stuff &&\n> +        git commit -a -m \"Independent discovery of 5->43\"\n> +       )\n> +'\n> +\n> +test_expect_failure 'git pull --rebase detects upstreamed changes' '\n> +       (cd dst &&\n> +        git pull --rebase &&\n> +        test -z \"$(git ls-files -u)\"\n> +       )\n> +'\n> +\n> +test_expect_success 'setup for avoiding reapplying old patches' '\n> +       (cd dst &&\n> +        (git rebase --abort || true) &&\n\nIf you're ignoring the git rebase --abort return value:\n\n  (cd dst &&\n    git rebase --abort;\n    git reset ...)\n\n> +        git reset --hard origin/master\n> +       ) &&\n> +       git clone --bare src src-replace.git &&\n> +       rm -rf src &&\n> +       mv src-replace.git src &&\n> +       (cd dst &&\n> +        perl -pi -e s/2/22/ stuff &&\n> +        git commit -a -m \"Change 2\" &&\n> +        perl -pi -e s/3/33/ stuff &&\n> +        git commit -a -m \"Change 3\" &&\n> +        perl -pi -e s/4/44/ stuff &&\n> +        git commit -a -m \"Change 4\" &&\n> +        git push &&\n> +\n> +        perl -pi -e s/44/55/ stuff &&\n> +        git commit --amend -a -m \"Modified Change 4\"\n> +       )\n> +'\n> +\n> +test_expect_failure 'git pull --rebase does not reapply old patches' '\n> +       (cd dst &&\n> +        (git pull --rebase || true) &&\n> +        test 1 = $(find .git/rebase-apply -name \"000*\" | wc -l)\n> +       )\n> +'\n> +\n>  test_done\n> --\n> 1.7.2.1\n>\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>\n"},{"id":"147456","messageId":"AANLkTimsfcSB7GMcNnZSBiwH4kwZeEazo5svW96_Nf9J@mail.gmail.com","threadId":"24670","inReplyTo":"AANLkTikhsFfCfmR2V8iSRvU73B5yW=_kqJAEfAGgxpLY@mail.gmail.com","subject":"Re: [PATCHv2 1/2] t5520-pull: Add testcases showing spurious conflicts from git pull --rebase","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2010-08-08T20:17:45Z","receivedAt":"2010-08-08T20:17:45Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nThanks for taking a look!\n\nOn Sun, Aug 8, 2010 at 2:01 PM, Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n> On Sun, Aug 8, 2010 at 19:04, Elijah Newren <newren@gmail.com> wrote:\n<snip>\n>> +        perl -pi -e s/5/43/ stuff &&\n>> +        git commit -a -m \"5->43\" &&\n>> +        perl -pi -e s/6/42/ stuff &&\n>\n> Please use sed so the test doesn't depend on perl being present.\n\nThe original series used 'sed -i', but I found out that is apparently\nnot portable.  Should I use sed + a temporary file + mv?\n\n<snip>\n>> +test_expect_success 'setup for avoiding reapplying old patches' '\n>> +       (cd dst &&\n>> +        (git rebase --abort || true) &&\n>\n> If you're ignoring the git rebase --abort return value:\n>\n>  (cd dst &&\n>    git rebase --abort;\n>    git reset ...)\n\nMakes sense; I'll do that.\n"},{"id":"147458","messageId":"AANLkTinkT5nZqJW=4ufU=oVp0GjUzh-HkLky_6S5RcJo@mail.gmail.com","threadId":"24670","inReplyTo":"AANLkTimsfcSB7GMcNnZSBiwH4kwZeEazo5svW96_Nf9J@mail.gmail.com","subject":"Re: [PATCHv2 1/2] t5520-pull: Add testcases showing spurious conflicts from git pull --rebase","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-08-08T20:33:03Z","receivedAt":"2010-08-08T20:33:03Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Sun, Aug 8, 2010 at 20:17, Elijah Newren <newren@gmail.com> wrote:\n> Hi,\n>\n> Thanks for taking a look!\n>\n> On Sun, Aug 8, 2010 at 2:01 PM, Ævar Arnfjörð Bjarmason\n> <avarab@gmail.com> wrote:\n>> On Sun, Aug 8, 2010 at 19:04, Elijah Newren <newren@gmail.com> wrote:\n> <snip>\n>>> +        perl -pi -e s/5/43/ stuff &&\n>>> +        git commit -a -m \"5->43\" &&\n>>> +        perl -pi -e s/6/42/ stuff &&\n>>\n>> Please use sed so the test doesn't depend on perl being present.\n>\n> The original series used 'sed -i', but I found out that is apparently\n> not portable.  Should I use sed + a temporary file + mv?\n\nYeah, a few tests do that already. E.g.:\n\n    sed s/6/42/ <stuff >stuff.tmp &&\n    mv stuff.tmp stuff\n\nYou could also just declare a dependency on perl:\n\n    test_expect_success PERL 'setup for detecting upstreamed changes' '\n\nBut then you'd also have to use \"$PERL_PATH\" instead of just perl, see\nt9700-perl-git.sh.\n"},{"id":"147464","messageId":"201008082336.18270.j6t@kdbg.org","threadId":"24670","inReplyTo":"1281294286-27709-2-git-send-email-newren@gmail.com","subject":"Re: [PATCHv2 1/2] t5520-pull: Add testcases showing spurious conflicts from git pull --rebase","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2010-08-08T21:36:18Z","receivedAt":"2010-08-08T21:36:18Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"On Sonntag, 8. August 2010, Elijah Newren wrote:\n> +test_expect_success 'setup for detecting upstreamed changes' '\n> +\tmkdir src &&\n> +\t(cd src &&\n> +\t git init &&\n> +\t for i in $(seq 1 10); do echo $i; done > stuff &&\n\nWe don't have seq on Windows. The sequence is short enough to list it \nexplicitly.\n\n> +\t git add stuff &&\n> +\t git commit -m \"Initial revision\"\n> +\t) &&\n> +\tgit clone src dst &&\n> +\t(cd src &&\n> +\t perl -pi -e s/5/43/ stuff &&\n\nperl -pi is no better than sed -i; it does not work on Windows because it \ntries to remove an open file (and that is a no-go on Windows). perl -pi.bak \ndoes work though.\n\n> +\t git commit -a -m \"5->43\" &&\n> +\t perl -pi -e s/6/42/ stuff &&\n> +\t git commit -a -m \"Make it bigger\" &&\n> +\t correct=$(git rev-parse HEAD)\n> +\t) &&\n\nThis assignment at the end of the subshell is pointless.\n\n> +test_expect_success 'setup for avoiding reapplying old patches' '\n> +\t(cd dst &&\n> +\t (git rebase --abort || true) &&\n\nPerhaps:\n\n\ttest_might_fail git rebase --abort &&\n\nbut I'm not sure whether that's the intended use of test_might_fail.\n\n> +\t git reset --hard origin/master\n> +\t) &&\n...\n> +test_expect_failure 'git pull --rebase does not reapply old patches' '\n> +\t(cd dst &&\n> +\t (git pull --rebase || true) &&\n\nDitto.\n\n> +\t test 1 = $(find .git/rebase-apply -name \"000*\" | wc -l)\n> +\t)\n> +'\n> +\n>  test_done\n\n-- Hannes\n"},{"id":"147465","messageId":"20100808214358.GB6962@burratino","threadId":"24670","inReplyTo":"201008082336.18270.j6t@kdbg.org","subject":"Re: [PATCHv2 1/2] t5520-pull: Add testcases showing spurious conflicts from git pull --rebase","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-08T21:43:58Z","receivedAt":"2010-08-08T21:43:58Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Johannes Sixt wrote:\n> On Sonntag, 8. August 2010, Elijah Newren wrote:\n\n>> +test_expect_success 'setup for avoiding reapplying old patches' '\n>> +\t(cd dst &&\n>> +\t (git rebase --abort || true) &&\n>\n> Perhaps:\n>\n> \ttest_might_fail git rebase --abort &&\n>\n> but I'm not sure whether that's the intended use of test_might_fail.\n\nIt is.  Thanks for noticing.\n"}]}