{"thread":{"id":"31876","subject":"[PATCH v2] Add new remote-hg transport helper","startedAt":"2012-10-20T15:00:06Z","lastAt":"2012-10-28T04:24:29Z","messageCount":4,"participants":["Felipe Contreras","Mike Hommey"],"isPatch":true,"patchVersion":2,"patchTotal":null},"messages":[{"id":"201593","messageId":"1350745206-28955-1-git-send-email-felipe.contreras@gmail.com","threadId":"31876","inReplyTo":null,"subject":"[PATCH v2] Add new remote-hg transport helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-20T15:00:06Z","receivedAt":"2012-10-20T15:00:06Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>\n---\n\nI've looked at many hg<->git tools and none satisfy me. Too complicated, or too\nslow, or to difficult to setup, etc.\n\nThe only one I've liked so far is hg-fast-export[1], which is indeed fast,\nrelatively simple, and relatively easy to use. But it's not properly maintained\nany more.\n\nSo, I decided to write my own from scratch, using hg-fast-export as\ninspiration, and voila.\n\nThis one doesn't have any dependencies, just put it into your $PATH, and you\ncan clone and fetch hg repositories. More importantly to me; the code is\nsimple, and easy to maintain.\n\nOne important remote-hg alternative is the one written by Sverre Rabbelier that\nis now maintained and distributed in msysgit, however, in my opinion the code\nis bloated, and there isn't even a standalone branch to take a look at the\npatches, and give them a try.\n\nThis version has some features that Sverre's version doesn't:\n\n * Support for tags\n * Support to specify branchesto pull\n\nSverre's version has some features this one doesn't:\n\n * Support for pushing\n * Tests\n\n[1] http://repo.or.cz/w/fast-export.git\n\nChanges since v1:\n\n * Improved documentation\n * Use more common 'python' binary\n * Warn, don't barf when a branch has multiple heads\n * Fixed marks to fetch after cloned\n * Support for cloning/pulling remote repositories\n * Use a more appropriate internal directory (e.g. .git/hg/origin)\n * Fixes for python3\n\n contrib/remote-hg/git-remote-hg | 254 ++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 254 insertions(+)\n create mode 100755 contrib/remote-hg/git-remote-hg\n\ndiff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg\nnew file mode 100755\nindex 0000000..cc53091\n--- /dev/null\n+++ b/contrib/remote-hg/git-remote-hg\n@@ -0,0 +1,254 @@\n+#!/usr/bin/python\n+\n+# Inspired by Rocco Rutte's hg-fast-export\n+\n+# Just copy to your ~/bin, or anywhere in your $PATH.\n+# Then you can clone with:\n+# git clone hg::/path/to/mercurial/repo/\n+\n+from mercurial import hg, ui\n+\n+import re\n+import sys\n+import os\n+import json\n+\n+first = True\n+\n+def die(msg, *args):\n+    sys.stderr.write('ERROR: %s\\n' % (msg % args))\n+    sys.exit(1)\n+\n+def warn(msg, *args):\n+    sys.stderr.write('WARNING: %s\\n' % (msg % args))\n+\n+def gitmode(flags):\n+    return 'l' in flags and '120000' or 'x' in flags and '100755' or '100644'\n+\n+def export_file(fc):\n+    if fc.path() == '.hgtags':\n+        return\n+    d = fc.data()\n+    print \"M %s inline %s\" % (gitmode(fc.flags()), fc.path())\n+    print \"data %d\" % len(d)\n+    print d\n+\n+def get_filechanges(repo, ctx, parents):\n+    l = [repo.status(p, ctx)[:3] for p in parents]\n+    changed, added, removed = [sum(e, []) for e in zip(*l)]\n+    return added + changed, removed\n+\n+author_re = re.compile('^((.+?) )?(<.+?>)$')\n+\n+def fixup_user(user):\n+    user = user.replace('\"', '')\n+    m = author_re.match(user)\n+    if m:\n+        name = m.group(1)\n+        mail = m.group(3)\n+    else:\n+        name = user\n+        mail = None\n+\n+    if not name:\n+        name = 'Unknown'\n+    if not mail:\n+        mail = '<unknown>'\n+\n+    return '%s %s' % (name, mail)\n+\n+def get_repo(path, alias):\n+    global dirname\n+\n+    myui = ui.ui()\n+    myui.setconfig('ui', 'interactive', 'off')\n+\n+    if hg.islocal(path):\n+        repo = hg.repository(myui, path)\n+    else:\n+        local_path = os.path.join(dirname, 'clone')\n+        if not os.path.exists(local_path):\n+            srcpeer, dstpeer = hg.clone(myui, {}, path, local_path, update=False, pull=True)\n+            repo = dstpeer.local()\n+        else:\n+            repo = hg.repository(myui, local_path)\n+            peer = hg.peer(myui, {}, path)\n+            repo.pull(peer, heads=None, force=True)\n+\n+    return repo\n+\n+def hg_branch(b):\n+    if b == 'master':\n+        return 'default'\n+    return b\n+\n+def git_branch(b):\n+    if b == 'default':\n+        return 'master'\n+    return b\n+\n+def export_tag(repo, tag):\n+    global prefix\n+    print \"reset %s/tags/%s\" % (prefix, tag)\n+    print \"from :%s\" % (repo[tag].rev() + 1)\n+    print\n+\n+def export_branch(repo, branch):\n+    global prefix, marks, cache, branches\n+\n+    heads = branches[hg_branch(branch)]\n+\n+    # verify there's only one head\n+    if (len(heads) > 1):\n+        warn(\"Branch '%s' has more than one head, consider merging\" % hg_branch(branch))\n+\n+    head = repo[heads[0]]\n+    tip = marks.get(branch, 0)\n+    # mercurial takes too much time checking this\n+    if tip == head.rev():\n+        # nothing to do\n+        return\n+    revs = repo.revs('%u:%u' % (tip, head))\n+    count = 0\n+\n+    revs = [rev for rev in revs if not cache.get(rev, False)]\n+\n+    for rev in revs:\n+\n+        c = repo[rev]\n+        (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(c.node())\n+        rev_branch = git_branch(extra['branch'])\n+\n+        tz = '%+03d%02d' % (-tz / 3600, -tz % 3600 / 60)\n+\n+        print \"commit %s/branches/%s\" % (prefix, rev_branch)\n+        print \"mark :%d\" % (rev + 1)\n+        print \"committer %s %d %s\" % (fixup_user(user), time, tz)\n+        print \"data %d\" % (len(desc) + 1)\n+        print desc\n+        print\n+\n+        parents = [p for p in repo.changelog.parentrevs(rev) if p >= 0]\n+\n+        if len(parents) == 0:\n+            modified = c.manifest().keys()\n+            removed = []\n+        else:\n+            added = []\n+            changed = []\n+            print \"from :%s\" % (parents[0] + 1)\n+            if len(parents) > 1:\n+                print \"merge :%s\" % (parents[1] + 1)\n+            modified, removed = get_filechanges(repo, c, parents)\n+\n+        for f in removed:\n+            print \"D %s\" % (f)\n+        for f in modified:\n+            export_file(c.filectx(f))\n+        print\n+\n+        count += 1\n+        if (count % 100 == 0):\n+            print \"progress revision %d '%s' (%d/%d)\" % (rev, branch, count, len(revs))\n+            print \"#############################################################\"\n+\n+        cache[rev] = True\n+\n+    # store the latest revision\n+    marks[branch] = rev\n+\n+def do_capabilities(repo, args):\n+    global prefix, dirname\n+\n+    print \"import\"\n+    print \"refspec refs/heads/*:%s/branches/*\" % prefix\n+    print \"refspec refs/tags/*:%s/tags/*\" % prefix\n+    print\n+\n+def do_list(repo, args):\n+    global branches\n+\n+    head = repo.dirstate.branch()\n+    for branch in repo.branchmap():\n+        heads = repo.branchheads(branch)\n+        if len(heads):\n+            branches[branch] = heads\n+\n+    for branch in branches:\n+        print \"? refs/heads/%s\" % git_branch(branch)\n+    for tag, node in repo.tagslist():\n+        if tag == 'tip':\n+            continue\n+        print \"? refs/tags/%s\" % tag\n+    print \"@refs/heads/%s HEAD\" % git_branch(head)\n+    print\n+\n+def do_import(repo, args):\n+    global first\n+\n+    ref = args[1]\n+\n+    if first:\n+        path = os.path.join(dirname, 'marks-git')\n+\n+        if os.path.exists(path):\n+            print \"feature import-marks=%s\" % path\n+        print \"feature export-marks=%s\" % path\n+        sys.stdout.flush()\n+        first = False\n+\n+    if (ref == 'HEAD'):\n+        return\n+\n+    if ref.startswith('refs/heads/'):\n+        branch = ref[len('refs/heads/'):]\n+        export_branch(repo, branch)\n+    elif ref.startswith('refs/tags/'):\n+        tag = ref[len('refs/tags/'):]\n+        export_tag(repo, tag)\n+\n+def main(args):\n+    global prefix, dirname, marks, cache, branches\n+\n+    alias = args[1]\n+    url = args[2]\n+\n+    gitdir = os.environ['GIT_DIR']\n+    dirname = os.path.join(gitdir, 'hg', alias)\n+    cache = {}\n+    branches = {}\n+\n+    repo = get_repo(url, alias)\n+    prefix = 'refs/hg/%s' % alias\n+\n+    if not os.path.exists(dirname):\n+        os.makedirs(dirname)\n+\n+    marks_path = os.path.join(dirname, 'marks-hg')\n+    try:\n+        fp = open(marks_path, 'r')\n+        marks = json.load(fp)\n+        fp.close()\n+    except IOError:\n+        marks = {}\n+\n+    line = True\n+    while (line):\n+        line = sys.stdin.readline().strip()\n+        if line == '':\n+            break\n+        args = line.split()\n+        cmd = args[0]\n+        if cmd == 'capabilities':\n+            do_capabilities(repo, args)\n+        elif cmd == 'list':\n+            do_list(repo, args)\n+        elif cmd == 'import':\n+            do_import(repo, args)\n+        sys.stdout.flush()\n+\n+    fp = open(marks_path, 'w')\n+    json.dump(marks, fp)\n+    fp.close()\n+\n+sys.exit(main(sys.argv))\n-- \n1.8.0.rc2.7.g0961fdf.dirty\n"},{"id":"201594","messageId":"20121020154301.GA31829@glandium.org","threadId":"31876","inReplyTo":"1350745206-28955-1-git-send-email-felipe.contreras@gmail.com","subject":"Re: [PATCH v2] Add new remote-hg transport helper","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2012-10-20T15:43:01Z","receivedAt":"2012-10-20T15:43:01Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Sat, Oct 20, 2012 at 05:00:06PM +0200, Felipe Contreras wrote:\n> Changes since v1:\n> \n>  * Improved documentation\n>  * Use more common 'python' binary\n>  * Warn, don't barf when a branch has multiple heads\n>  * Fixed marks to fetch after cloned\n>  * Support for cloning/pulling remote repositories\n>  * Use a more appropriate internal directory (e.g. .git/hg/origin)\n>  * Fixes for python3\n\nAre the resulting commits identical to what you'd get from using hg-git?\n\nMike\n"},{"id":"201595","messageId":"CAMP44s14LeQY5OumDrMRN9foQPTnAUxtYoyf0Z5YHzjxOybcsA@mail.gmail.com","threadId":"31876","inReplyTo":"20121020154301.GA31829@glandium.org","subject":"Re: [PATCH v2] Add new remote-hg transport helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-20T16:04:06Z","receivedAt":"2012-10-20T16:04:06Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Sat, Oct 20, 2012 at 5:43 PM, Mike Hommey <mh@glandium.org> wrote:\n> On Sat, Oct 20, 2012 at 05:00:06PM +0200, Felipe Contreras wrote:\n>> Changes since v1:\n>>\n>>  * Improved documentation\n>>  * Use more common 'python' binary\n>>  * Warn, don't barf when a branch has multiple heads\n>>  * Fixed marks to fetch after cloned\n>>  * Support for cloning/pulling remote repositories\n>>  * Use a more appropriate internal directory (e.g. .git/hg/origin)\n>>  * Fixes for python3\n>\n> Are the resulting commits identical to what you'd get from using hg-git?\n\nNo, but should be easy to implement.\n\n-- \nFelipe Contreras\n"},{"id":"202031","messageId":"CAMP44s1a4ZjhQySTFbFzvgPhTwNJNoqLchP0xqmGekwHk5kqXw@mail.gmail.com","threadId":"31876","inReplyTo":"20121020154301.GA31829@glandium.org","subject":"Re: [PATCH v2] Add new remote-hg transport helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-28T04:24:29Z","receivedAt":"2012-10-28T04:24:29Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Sat, Oct 20, 2012 at 5:43 PM, Mike Hommey <mh@glandium.org> wrote:\n> On Sat, Oct 20, 2012 at 05:00:06PM +0200, Felipe Contreras wrote:\n>> Changes since v1:\n>>\n>>  * Improved documentation\n>>  * Use more common 'python' binary\n>>  * Warn, don't barf when a branch has multiple heads\n>>  * Fixed marks to fetch after cloned\n>>  * Support for cloning/pulling remote repositories\n>>  * Use a more appropriate internal directory (e.g. .git/hg/origin)\n>>  * Fixes for python3\n>\n> Are the resulting commits identical to what you'd get from using hg-git?\n\nNow they are :)\n\n-- \nFelipe Contreras\n"}]}