{"thread":{"id":"55273","subject":"[PATCH] contrib/rebase-catchup: helper for updating old branches","startedAt":"2021-03-08T23:04:54Z","lastAt":"2021-03-09T01:04:44Z","messageCount":3,"participants":["Mike Frysinger","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"418546","messageId":"20210308230345.28498-1-vapier@gentoo.org","threadId":"55273","inReplyTo":null,"subject":"[PATCH] contrib/rebase-catchup: helper for updating old branches","fromName":"Mike Frysinger","fromEmail":"vapier@gentoo.org","sentAt":"2021-03-08T23:03:45Z","receivedAt":"2021-03-08T23:04:54Z","isPatch":true,"sender":{"key":"vapier@gentoo.org","avatar":"https://gravatar.com/avatar/e84e90ef943ceda7d1138ced7f3cc3f866dc1fdf9cb76313ffba167fb73fa057?d=mp&s=160"},"body":"For people who want to rebase their work onto the latest branch\n(instead of merging), but there's many conflicting changes.  This\nallows you to address those conflicts one-by-one and work through\neach issue instead of trying to take them all on at once.\n\nSigned-off-by: Mike Frysinger <vapier@gentoo.org>\n---\nIf there's no interest in merging this into contrib, then this is more spam,\nand anyone interested can use https://github.com/vapier/git-rebase-catchup\n\n contrib/rebase-catchup/README.md             |  61 ++++++\n contrib/rebase-catchup/git-rebase-catchup.py | 187 +++++++++++++++++++\n 2 files changed, 248 insertions(+)\n create mode 100644 contrib/rebase-catchup/README.md\n create mode 100755 contrib/rebase-catchup/git-rebase-catchup.py\n\ndiff --git a/contrib/rebase-catchup/README.md b/contrib/rebase-catchup/README.md\nnew file mode 100644\nindex 000000000000..083d2012f833\n--- /dev/null\n+++ b/contrib/rebase-catchup/README.md\n@@ -0,0 +1,61 @@\n+# Rebase Catchup\n+\n+Helpful when you have a branch tracking an old commit, and a lot of conflicting\n+changes have landed in the latest branch, but you still want to update.\n+\n+A single rebase to the latest commit will require addressing all the different\n+changes at once which can be difficult, overwhelming, and error-prone.  Instead,\n+if you rebased onto each intermediate conflicting point, you'd break up the work\n+into smaller pieces, and be able to run tests to make sure things were still OK.\n+\n+## Example\n+\n+Let's say you have a branch that is currently 357 commits behind.  When you try\n+rebasing onto the latest, it hits a lot of conflicts.  The tool will bisect down\n+to find the most recent commit it can cleanly rebase onto.\n+\n+```sh\n+$ git rebase-catchup\n+Local branch resolved to \"s-logs\"\n+Tracking branch resolved to \"origin/master\"\n+Branch is 2 commits ahead and 357 commits behind\n+Trying to rebase onto latest origin/master ... failed; falling back to bisect\n+Rebasing onto origin/master~178 ... failed\n+Rebasing onto origin/master~267 ... failed\n+Rebasing onto origin/master~312 ... failed\n+Rebasing onto origin/master~334 ... failed\n+Rebasing onto origin/master~345 ... OK\n+Rebasing onto origin/master~339 ... OK\n+Rebasing onto origin/master~336 ... failed\n+Rebasing onto origin/master~337 ... OK\n+Rebasing onto origin/master~336 ... failed\n+Found first failure origin/master~336\n+```\n+\n+Now you know the first conflicting change is `origin/master~336`.  Rebase onto\n+that directly and address all the problems (and run tests/etc...).  Then restart\n+the process.\n+\n+```sh\n+$ git rebase origin/master~336\n+... address all the conflicts ...\n+$ git rebase --continue\n+$ git rebase-catchup\n+Local branch resolved to \"s-logs\"\n+Tracking branch resolved to \"origin/master\"\n+Branch is 2 commits ahead and 335 commits behind\n+Trying to rebase onto latest origin/master ... failed; falling back to bisect\n+Rebasing onto origin/master~167 ... OK\n+Rebasing onto origin/master~83 ... OK\n+Rebasing onto origin/master~41 ... failed\n+Rebasing onto origin/master~62 ... OK\n+Rebasing onto origin/master~51 ... OK\n+Rebasing onto origin/master~46 ... OK\n+Rebasing onto origin/master~43 ... failed\n+Rebasing onto origin/master~44 ... failed\n+Rebasing onto origin/master~45 ... OK\n+Rebasing onto origin/master~44 ... failed\n+Found first failure origin/master~44\n+```\n+\n+Now you're only 44 commits behind.  Keep doing this until you catchup!\ndiff --git a/contrib/rebase-catchup/git-rebase-catchup.py b/contrib/rebase-catchup/git-rebase-catchup.py\nnew file mode 100755\nindex 000000000000..2cc5c5381616\n--- /dev/null\n+++ b/contrib/rebase-catchup/git-rebase-catchup.py\n@@ -0,0 +1,187 @@\n+#!/usr/bin/env python3\n+# Distributed under the terms of the GNU General Public License v2 or later.\n+\n+\"\"\"Helper to automatically rebase onto latest commit possible.\n+\n+Helpful when you have a branch tracking an old commit, and a lot of conflicting\n+changes have landed in the latest branch, but you still want to update.\n+\n+A single rebase to the latest commit will require addressing all the different\n+changes at once which can be difficult, overwhelming, and error-prone.  Instead,\n+if you rebased onto each intermediate conflicting point, you'd break up the work\n+into smaller pieces, and be able to run tests to make sure things were still OK.\n+\"\"\"\n+\n+import argparse\n+import subprocess\n+import sys\n+from typing import List, Tuple, Union\n+\n+\n+assert sys.version_info >= (3, 7), f'Need Python 3.7+, not {sys.version_info}'\n+\n+\n+def git(args: List[str], **kwargs) -> subprocess.CompletedProcess:\n+    \"\"\"Run git.\"\"\"\n+    kwargs.setdefault('check', True)\n+    kwargs.setdefault('capture_output', True)\n+    kwargs.setdefault('encoding', 'utf-8')\n+    # pylint: disable=subprocess-run-check\n+    return subprocess.run(['git'] + args, **kwargs)\n+\n+\n+def rebase(target: str) -> bool:\n+    \"\"\"Try to rebase onto |target|.\"\"\"\n+    try:\n+        git(['rebase', target])\n+        return True\n+    except KeyboardInterrupt:\n+        git(['rebase', '--abort'])\n+        print('aborted')\n+        sys.exit(1)\n+    except:\n+        git(['rebase', '--abort'])\n+        return False\n+\n+\n+def rebase_bisect(lbranch: str,\n+                  rbranch: str,\n+                  behind: int,\n+                  leave_rebase: bool = False,\n+                  force_checkout: bool = False):\n+    \"\"\"Try to rebase branch as close to |rbranch| as possible.\"\"\"\n+    def attempt(pos: int) -> bool:\n+        target = f'{rbranch}~{pos}'\n+        print(f'Rebasing onto {target} ', end='')\n+        print('.', end='', flush=True)\n+        # Checking out these branches directly helps clobber orphaned files,\n+        # but is usually unnessary, and can slow down the overall process.\n+        if force_checkout:\n+            git(['checkout', '-f', target])\n+        print('.', end='', flush=True)\n+        if force_checkout:\n+            git(['checkout', '-f', lbranch])\n+        print('. ', end='', flush=True)\n+        ret = rebase(target)\n+        print('OK' if ret else 'failed')\n+        return ret\n+\n+    # \"pmin\" is the latest branch position while \"pmax\" is where we're now.\n+    pmin = 0\n+    pmax = behind\n+    old_mid = None\n+    first_fail = 0\n+    while True:\n+        mid = pmin + (pmax - pmin) // 2\n+        if mid == old_mid or mid < pmin or mid >= pmax:\n+            break\n+        if attempt(mid):\n+            pmax = mid\n+        else:\n+            first_fail = max(first_fail, mid)\n+            pmin = mid\n+        old_mid = mid\n+\n+    if pmin or pmax:\n+        last_target = f'{rbranch}~{first_fail}'\n+        if leave_rebase:\n+            print('Restarting', last_target)\n+            result = git(['rebase', last_target], check=False)\n+            print(result.stdout.strip())\n+        else:\n+            print('Found first failure', last_target)\n+    else:\n+        print('All caught up!')\n+\n+\n+def get_ahead_behind(lbranch: str, rbranch: str) -> Tuple[int, int]:\n+    \"\"\"Return number of commits |lbranch| is ahead & behind relative to |rbranch|.\"\"\"\n+    output = git(\n+        ['rev-list', '--left-right', '--count', f'{lbranch}...{rbranch}']).stdout\n+    return [int(x) for x in output.split()]\n+\n+\n+def get_tracking_branch(branch: str) -> Union[str, None]:\n+    \"\"\"Return branch that |branch| is tracking.\"\"\"\n+    merge = git(['config', '--local', f'branch.{branch}.merge']).stdout.strip()\n+    if not merge:\n+        return None\n+\n+    remote = git(['config', '--local', f'branch.{branch}.remote']).stdout.strip()\n+    if remote:\n+        if merge.startswith('refs/heads/'):\n+            merge = merge[11:]\n+        return f'{remote}/{merge}'\n+    else:\n+        return merge\n+\n+\n+def get_local_branch() -> str:\n+    \"\"\"Return the name of the local checked out branch.\"\"\"\n+    return git(['branch', '--show-current']).stdout.strip()\n+\n+\n+def get_parser() -> argparse.ArgumentParser:\n+    \"\"\"Get CLI parser.\"\"\"\n+    parser = argparse.ArgumentParser(\n+        description=__doc__,\n+        formatter_class=argparse.RawDescriptionHelpFormatter)\n+    parser.add_argument(\n+        '--skip-initial-rebase-latest', dest='initial_rebase',\n+        action='store_false', default=True,\n+        help='skip initial rebase attempt onto the latest branch')\n+    parser.add_argument(\n+        '--leave-at-last-failed-rebase', dest='leave_rebase',\n+        action='store_true', default=False,\n+        help='leave tree state at last failing rebase')\n+    parser.add_argument(\n+        '--checkout-before-rebase', dest='force_checkout',\n+        action='store_true', default=False,\n+        help='force checkout before rebasing to target (to cleanup orphans)')\n+    parser.add_argument(\n+        'branch', nargs='?',\n+        help='branch to rebase onto')\n+    return parser\n+\n+\n+def main(argv: List[str]) -> int:\n+    \"\"\"The main entry point for scripts.\"\"\"\n+    parser = get_parser()\n+    opts = parser.parse_args(argv)\n+\n+    lbranch = get_local_branch()\n+    print(f'Local branch resolved to \"{lbranch}\"')\n+    if not lbranch:\n+        print('Unable to resolve local branch', file=sys.stderr)\n+        return 1\n+\n+    if opts.branch:\n+        rbranch = opts.branch\n+    else:\n+        rbranch = get_tracking_branch(lbranch)\n+    print(f'Tracking branch resolved to \"{rbranch}\"')\n+\n+    ahead, behind = get_ahead_behind(lbranch, rbranch)\n+    print(f'Branch is {ahead} commits ahead and {behind} commits behind')\n+\n+    if not behind:\n+        print('Up-to-date!')\n+    elif not ahead:\n+        print('Fast forwarding ...')\n+        git(['merge'])\n+    else:\n+        if opts.initial_rebase:\n+            print(f'Trying to rebase onto latest {rbranch} ... ',\n+                  end='', flush=True)\n+            if rebase(rbranch):\n+                print('OK!')\n+                return 0\n+            print('failed; falling back to bisect')\n+        rebase_bisect(lbranch, rbranch, behind, leave_rebase=opts.leave_rebase,\n+                      force_checkout=opts.force_checkout)\n+\n+    return 0\n+\n+\n+if __name__ == '__main__':\n+    sys.exit(main(sys.argv[1:]))\n-- \n2.30.0\n\n"},{"id":"418548","messageId":"xmqqk0qh43ob.fsf@gitster.c.googlers.com","threadId":"55273","inReplyTo":"20210308230345.28498-1-vapier@gentoo.org","subject":"Re: [PATCH] contrib/rebase-catchup: helper for updating old branches","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-08T23:38:12Z","receivedAt":"2021-03-08T23:38:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Mike Frysinger <vapier@gentoo.org> writes:\n\n> For people who want to rebase their work onto the latest branch\n> (instead of merging), but there's many conflicting changes.  This\n> allows you to address those conflicts one-by-one and work through\n> each issue instead of trying to take them all on at once.\n\nI wonder how well this compares or complements with Michael\nHaggerty's \"git imerge\".\n\n> If there's no interest in merging this into contrib, then this is more spam,\n> and anyone interested can use https://github.com/vapier/git-rebase-catchup\n\nThe thinking during the past several years is that the Git ecosystem\nand userbase have grown large enough, and unlike our earlier years,\nindividual add-on's like this (and \"imerge\" I mentioned earlier) can\nthrive without being in-tree to gain an undue exposure boost over\nits competitors, so I doubt that adding more stuff to contrib/ would\nbe a good direction to go in the longer term.\n\n>  create mode 100755 contrib/rebase-catchup/git-rebase-catchup.py\n\nAnd it does not help for this to be written in Python (which we've\nbeen moving away from).\n\nHaving said all that, the end-user community may benefit from having\na well curated set of third-party tools advertised at a single\nautoritative-sounding place, but it is dubious that contrib/\nsubdirectory of my tree should be such a place (I won't be its\ncurator anyway).\n\nPerhaps there is an opening for non-code contribution for those\nwithout much programming experience but with great organizational\nskills.\n\nThanks.\n"},{"id":"418564","messageId":"YEbJd3Y/8ep1d5gc@vapier","threadId":"55273","inReplyTo":"xmqqk0qh43ob.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] contrib/rebase-catchup: helper for updating old branches","fromName":"Mike Frysinger","fromEmail":"vapier@gentoo.org","sentAt":"2021-03-09T01:03:51Z","receivedAt":"2021-03-09T01:04:44Z","isPatch":true,"sender":{"key":"vapier@gentoo.org","avatar":"https://gravatar.com/avatar/e84e90ef943ceda7d1138ced7f3cc3f866dc1fdf9cb76313ffba167fb73fa057?d=mp&s=160"},"body":"On 08 Mar 2021 15:38, Junio C Hamano wrote:\n> Mike Frysinger <vapier@gentoo.org> writes:\n> > For people who want to rebase their work onto the latest branch\n> > (instead of merging), but there's many conflicting changes.  This\n> > allows you to address those conflicts one-by-one and work through\n> > each issue instead of trying to take them all on at once.\n> \n> I wonder how well this compares or complements with Michael\n> Haggerty's \"git imerge\".\n\nthanks, hadn't heard of that before\n\n> > If there's no interest in merging this into contrib, then this is more spam,\n> > and anyone interested can use https://github.com/vapier/git-rebase-catchup\n> \n> The thinking during the past several years is that the Git ecosystem\n> and userbase have grown large enough, and unlike our earlier years,\n> individual add-on's like this (and \"imerge\" I mentioned earlier) can\n> thrive without being in-tree to gain an undue exposure boost over\n> its competitors, so I doubt that adding more stuff to contrib/ would\n> be a good direction to go in the longer term.\n\ni'm totally fine with \"no\".  thanks for the info.\n-mike\n"}]}