{"thread":{"id":"31849","subject":"[PATCH] Add new git-remote-hd helper","startedAt":"2012-10-17T12:58:41Z","lastAt":"2012-10-26T09:02:18Z","messageCount":27,"participants":["Felipe Contreras","Johannes Schindelin","Sverre Rabbelier","Jeff King","Junio C Hamano","Matthieu Moy","Michael J Gruber"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"201425","messageId":"1350478721-3685-1-git-send-email-felipe.contreras@gmail.com","threadId":"31849","inReplyTo":null,"subject":"[PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-17T12:58:41Z","receivedAt":"2012-10-17T12:58:41Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"Signed-off-by: Felipe Contreras \n---\n\nI've looked at many hg<->git tools and none satisfy me. Too complicated, or too\nslow, or to difficult to setup, etc.\n\nThe only one I've liked so far is hg-fast-export[1], which is indeed fast,\nrelatively simple, and relatively easy to use. But it's not properly maintained\nany more.\n\nSo, I decided to write my own from scratch, using hg-fast-export as\ninspiration, and voila.\n\nThis one doesn't have any dependencies, just put it into your $PATH, and you\ncan clone and fetch hg repositories. More importantly to me; the code is\nsimple, and easy to maintain.\n\n[1] http://repo.or.cz/w/fast-export.git\n\n contrib/remote-hd/git-remote-hg | 231 ++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 231 insertions(+)\n create mode 100755 contrib/remote-hd/git-remote-hg\n\ndiff --git a/contrib/remote-hd/git-remote-hg b/contrib/remote-hd/git-remote-hg\nnew file mode 100755\nindex 0000000..9157b30\n--- /dev/null\n+++ b/contrib/remote-hd/git-remote-hg\n@@ -0,0 +1,231 @@\n+#!/usr/bin/python2\n+\n+# Inspired by Rocco Rutte's hg-fast-export\n+\n+# Just copy to your ~/bin, or anywhere in your $PATH.\n+# Then you can clone with:\n+# hg::file:///path/to/mercurial/repo/\n+\n+from mercurial import hg, ui\n+\n+import re\n+import sys\n+import os\n+import json\n+\n+def die(msg, *args):\n+ print >> sys.stderr, 'ERROR:', msg % args\n+ sys.exit(1)\n+\n+def gitmode(flags):\n+ return 'l' in flags and '120000' or 'x' in flags and '100755' or '100644'\n+\n+def export_file(fc):\n+ if fc.path() == '.hgtags':\n+ return\n+ d = fc.data()\n+ print \"M %s inline %s\" % (gitmode(fc.flags()), fc.path())\n+ print \"data %d\" % len(d)\n+ print d\n+\n+def get_filechanges(repo, ctx, parents):\n+ l = [repo.status(p, ctx)[:3] for p in parents]\n+ changed, added, removed = [sum(e, []) for e in zip(*l)]\n+ return added + changed, removed\n+\n+author_re = re.compile('^((.+?) )?(<.+?>)$')\n+\n+def fixup_user(user):\n+ user = user.replace('\"', '')\n+ m = author_re.match(user)\n+ if m:\n+ name = m.group(1)\n+ mail = m.group(3)\n+ else:\n+ name = user\n+ mail = None\n+\n+ if not name:\n+ name = 'Unknown'\n+ if not mail:\n+ mail = ''\n+\n+ return '%s %s' % (name, mail)\n+\n+def get_repo(path, alias):\n+ myui = ui.ui()\n+ myui.setconfig('ui', 'interactive', 'off')\n+ repo = hg.repository(myui, path)\n+ return repo\n+\n+def hg_branch(b):\n+ if b == 'master':\n+ return 'default'\n+ return b\n+\n+def git_branch(b):\n+ if b == 'default':\n+ return 'master'\n+ return b\n+\n+def export_tag(repo, tag):\n+ global prefix\n+ print \"reset %s/tags/%s\" % (prefix, tag)\n+ print \"from :%s\" % (repo[tag].rev() + 1)\n+ print\n+\n+def export_branch(repo, branch):\n+ global prefix, marks, cache, branches\n+\n+ heads = branches[hg_branch(branch)]\n+\n+ # verify there's only one head\n+ if (len(heads) > 1):\n+ die(\"Branch '%s' has more than one head\" % hg_branch(branch))\n+\n+ head = repo[heads[0]]\n+ tip = marks.get(branch, 0)\n+ # mercurial takes too much time checking this\n+ if tip == head.rev():\n+ # nothing to do\n+ return\n+ revs = repo.revs('%u:%u' % (tip, head))\n+ count = 0\n+\n+ revs = [rev for rev in revs if not cache.get(rev, False)]\n+\n+ for rev in revs:\n+\n+ c = repo[rev]\n+ (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(c.node())\n+ rev_branch = git_branch(extra['branch'])\n+\n+ tz = '%+03d%02d' % (-tz / 3600, -tz % 3600 / 60)\n+\n+ print \"commit %s/branches/%s\" % (prefix, rev_branch)\n+ print \"mark :%d\" % (rev + 1)\n+ print \"committer %s %d %s\" % (fixup_user(user), time, tz)\n+ print \"data %d\" % (len(desc) + 1)\n+ print desc\n+ print\n+\n+ parents = [p for p in repo.changelog.parentrevs(rev) if p >= 0]\n+\n+ if len(parents) == 0:\n+ modified = c.manifest().keys()\n+ removed = []\n+ else:\n+ added = []\n+ changed = []\n+ print \"from :%s\" % (parents[0] + 1)\n+ if len(parents) > 1:\n+ print \"merge :%s\" % (parents[1] + 1)\n+ modified, removed = get_filechanges(repo, c, parents)\n+\n+ for f in removed:\n+ print \"D %s\" % (f)\n+ for f in modified:\n+ export_file(c.filectx(f))\n+ print\n+\n+ count += 1\n+ if (count % 100 == 0):\n+ print \"progress revision %d '%s' (%d/%d)\" % (rev, branch, count, len(revs))\n+ print \"#############################################################\"\n+\n+ cache[rev] = True\n+\n+ # store the latest revision\n+ marks[branch] = rev\n+\n+def do_capabilities(repo, args):\n+ global prefix, dirname\n+\n+ print \"import\"\n+ print \"refspec refs/heads/*:%s/branches/*\" % prefix\n+ print \"refspec refs/tags/*:%s/tags/*\" % prefix\n+\n+ path = os.path.join(dirname, 'marks-git')\n+\n+ print \"*export-marks %s\" % path\n+ if os.path.exists(path):\n+ print \"*import-marks %s\" % path\n+\n+ print\n+\n+def do_list(repo, args):\n+ global branches\n+\n+ head = repo.dirstate.branch()\n+ for branch in repo.branchmap():\n+ heads = repo.branchheads(branch)\n+ if len(heads):\n+ branches[branch] = heads\n+\n+ for branch in branches:\n+ print \"? refs/heads/%s\" % git_branch(branch)\n+ for tag, node in repo.tagslist():\n+ if tag == 'tip':\n+ continue\n+ print \"? refs/tags/%s\" % tag\n+ print \"@refs/heads/%s HEAD\" % git_branch(head)\n+ print\n+\n+def do_import(repo, args):\n+ ref = args[1]\n+\n+ if (ref == 'HEAD'):\n+ return\n+\n+ if ref.startswith('refs/heads/'):\n+ branch = ref[len('refs/heads/'):]\n+ export_branch(repo, branch)\n+ elif ref.startswith('refs/tags/'):\n+ tag = ref[len('refs/tags/'):]\n+ export_tag(repo, tag)\n+\n+def main(args):\n+ global prefix, dirname, marks, cache, branches\n+\n+ alias = args[1]\n+ url = args[2]\n+\n+ gitdir = os.environ['GIT_DIR']\n+ dirname = os.path.join(gitdir, 'hg')\n+ cache = {}\n+ branches = {}\n+\n+ repo = get_repo(url, alias)\n+ prefix = 'refs/hg/%s' % alias\n+\n+ if not os.path.exists(dirname):\n+ os.makedirs(dirname)\n+\n+ marks_path = os.path.join(dirname, 'marks-hg')\n+ try:\n+ fp = open(marks_path, 'r')\n+ marks = json.load(fp)\n+ fp.close()\n+ except IOError:\n+ marks = {}\n+\n+ line = True\n+ while (line):\n+ line = sys.stdin.readline().strip()\n+ if line == '':\n+ break\n+ args = line.split()\n+ cmd = args[0]\n+ if cmd == 'capabilities':\n+ do_capabilities(repo, args)\n+ elif cmd == 'list':\n+ do_list(repo, args)\n+ elif cmd == 'import':\n+ do_import(repo, args)\n+ sys.stdout.flush()\n+\n+ fp = open(marks_path, 'w')\n+ json.dump(marks, fp)\n+ fp.close()\n+\n+sys.exit(main(sys.argv))\n-- \n1.8.0.rc2.5.gccf4c94\n"},{"id":"201428","messageId":"alpine.DEB.1.00.1210171759230.3049@bonsai2","threadId":"31849","inReplyTo":"1350478721-3685-1-git-send-email-felipe.contreras@gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2012-10-17T16:03:14Z","receivedAt":"2012-10-17T16:03:14Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 17 Oct 2012, Felipe Contreras wrote:\n\n> I've looked at many hg<->git tools and none satisfy me. Too complicated,\n> or too slow, or to difficult to setup, etc.\n\nThe one I merged into Git for Windows (since that is what I install on all\nmy machines even if they run Linux) is rock-solid. It also comes with\ntests. And it requires a fix I tried to get into git.git (but failed,\nsince I was asked to do much more in addition to what I needed for myself,\nand I lack the time to address such requests these days).\n\nSo I have to admit that I do not quite see the point of avoiding to\nenhance the existing work of Sverre (and a little bit of me, too, in a\nhackathon for which I traveled half the continent back in July 2011).\n\nCiao,\nJohannes\n"},{"id":"201429","messageId":"CAMP44s1WY+Q7jyy4PQvwff7JSxHsnkhrPWozOLnNuNOrn7FGBA@mail.gmail.com","threadId":"31849","inReplyTo":"alpine.DEB.1.00.1210171759230.3049@bonsai2","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-17T16:38:23Z","receivedAt":"2012-10-17T16:38:23Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Wed, Oct 17, 2012 at 6:03 PM, Johannes Schindelin\n wrote:\n> On Wed, 17 Oct 2012, Felipe Contreras wrote:\n>\n>> I've looked at many hg<->git tools and none satisfy me. Too complicated,\n>> or too slow, or to difficult to setup, etc.\n>\n> The one I merged into Git for Windows (since that is what I install on all\n> my machines even if they run Linux) is rock-solid. It also comes with\n> tests. And it requires a fix I tried to get into git.git (but failed,\n> since I was asked to do much more in addition to what I needed for myself,\n> and I lack the time to address such requests these days).\n\nMaybe, but who uses it? It's quite a lot of code, and it's quite\ndifficult to setup--you would need a non-vanilla version of git.\n\nCompare this:\n32 files changed, 3351 insertions(+), 289 deletions(-)\n\nTo this:\n1 file changed, 231 insertions(+)\n\nI would like to first get something that works in, and then step by\nstep work on top of that.\n\nAnyway, I'm not even sure which version you are talking about, because\nthere's plenty out there:\nhttps://github.com/SRabbelier/git/network\n\n> So I have to admit that I do not quite see the point of avoiding to\n> enhance the existing work of Sverre (and a little bit of me, too, in a\n> hackathon for which I traveled half the continent back in July 2011).\n\nIt's way too much code, to be specific; 15x the code I just submitted.\nIt would be better to work together, but to me the code-styles are way\ntoo different, the difference between night and day. If you are\ninterested in simplifying that code, get rid of the classes of classes\nof classes and have something more consolidated, I could try to\ncontribute, but I doubt that's the case.\n\nAnyway, this is 231 lines of code, and works just fine, which is\nbetter than what we have in git.git for mercurial: basically nothing.\n\nCheers.\n\n-- \nFelipe Contreras\n"},{"id":"201433","messageId":"alpine.DEB.1.00.1210171936170.3049@bonsai2","threadId":"31849","inReplyTo":"CAMP44s1WY+Q7jyy4PQvwff7JSxHsnkhrPWozOLnNuNOrn7FGBA@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2012-10-17T17:39:44Z","receivedAt":"2012-10-17T17:39:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 17 Oct 2012, Felipe Contreras wrote:\n\n> On Wed, Oct 17, 2012 at 6:03 PM, Johannes Schindelin\n> wrote:\n> > On Wed, 17 Oct 2012, Felipe Contreras wrote:\n> >\n> >> I've looked at many hg<->git tools and none satisfy me. Too\n> >> complicated, or too slow, or to difficult to setup, etc.\n> >\n> > The one I merged into Git for Windows (since that is what I install on\n> > all my machines even if they run Linux) is rock-solid. It also comes\n> > with tests. And it requires a fix I tried to get into git.git (but\n> > failed, since I was asked to do much more in addition to what I needed\n> > for myself, and I lack the time to address such requests these days).\n> \n> Maybe, but who uses it? It's quite a lot of code, and it's quite\n> difficult to setup--you would need a non-vanilla version of git.\n\nOkay, so the difficulty of setting it up is because it is not in mainline\ngit.git?\n\n> Compare this:\n> 32 files changed, 3351 insertions(+), 289 deletions(-)\n> \n> To this:\n> 1 file changed, 231 insertions(+)\n\nYeah, and that's also because of the severe lack of tests. And the lack of\npossible code-sharing with other remote helpers.\n\nAs for who uses it:\n\n\thttps://github.com/dscho/hg\n\n> It would be better to work together, but to me the code-styles are way\n> too different, the difference between night and day.\n\nAha. Well, okay, it was an offer to collaborate.\n\nCiao,\nJohannes\n"},{"id":"201436","messageId":"CAMP44s2=hhMz35SpYXMBLcQPfVaPxY4pBWZDF7zbqVwz=tbedA@mail.gmail.com","threadId":"31849","inReplyTo":"alpine.DEB.1.00.1210171936170.3049@bonsai2","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-17T18:12:43Z","receivedAt":"2012-10-17T18:12:43Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Wed, Oct 17, 2012 at 7:39 PM, Johannes Schindelin\n wrote:\n> On Wed, 17 Oct 2012, Felipe Contreras wrote:\n>\n>> On Wed, Oct 17, 2012 at 6:03 PM, Johannes Schindelin\n>> wrote:\n>> > On Wed, 17 Oct 2012, Felipe Contreras wrote:\n>> >\n>> >> I've looked at many hg<->git tools and none satisfy me. Too\n>> >> complicated, or too slow, or to difficult to setup, etc.\n>> >\n>> > The one I merged into Git for Windows (since that is what I install on\n>> > all my machines even if they run Linux) is rock-solid. It also comes\n>> > with tests. And it requires a fix I tried to get into git.git (but\n>> > failed, since I was asked to do much more in addition to what I needed\n>> > for myself, and I lack the time to address such requests these days).\n>>\n>> Maybe, but who uses it? It's quite a lot of code, and it's quite\n>> difficult to setup--you would need a non-vanilla version of git.\n>\n> Okay, so the difficulty of setting it up is because it is not in mainline\n> git.git?\n\nMy version:\n1) cp contrib/remote-hg/git-remote-hg ~/bin\n\nYour version? I don't know, but it certainly will be more than one\nstep... may more.\n\n>> Compare this:\n>> 32 files changed, 3351 insertions(+), 289 deletions(-)\n>>\n>> To this:\n>> 1 file changed, 231 insertions(+)\n>\n> Yeah, and that's also because of the severe lack of tests. And the lack of\n> possible code-sharing with other remote helpers.\n\nWhat is there to share? It's 230 lines of code. And share it with\nwhich remote helpers? And trying to do so will certainly make it\nharder to setup.\n\nAs for the tests, I don't see any tests checking that the import of\ntags succeeds. Oh, that's right, that is not implemented, so not\nsurprisingly; there are no tests for that. What does a dozen passing\ntests tell you about the code? Not much. If my code had tests, the\ntest for importing tags would succeed, but I chose to spend my time\nimplementing features, and trying to make them accessible to as many\npeople as possible... rather than writing tests.\n\nBut fine, lets remove the tests out of the equation (150 lines), the\nnumber of lines of code still exceeds 3000.\n\nCheers.\n\n-- \nFelipe Contreras\n"},{"id":"201437","messageId":"CAGdFq_i4c=ei4ni5bv9nNEbCfNG4sbRkqyip2voW17GhkRAKPQ@mail.gmail.com","threadId":"31849","inReplyTo":"CAMP44s2=hhMz35SpYXMBLcQPfVaPxY4pBWZDF7zbqVwz=tbedA@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2012-10-17T18:18:05Z","receivedAt":"2012-10-17T18:18:05Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"On Wed, Oct 17, 2012 at 11:12 AM, Felipe Contreras\n wrote:\n> But fine, lets remove the tests out of the equation (150 lines), the\n> number of lines of code still exceeds 3000.\n\nI don't think it's fair to just look at LOC, git-remote-hg when it was\njust parsing was fairly simple. Most of the current code is our copy\nof the python fast-import library which is only used to support\npushing to mercurial.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"201443","messageId":"CAMP44s2-BCuA5v7jE8S3d-Gg+DxKf6Yte9MvTDBDr4jEkLKZ9g@mail.gmail.com","threadId":"31849","inReplyTo":"CAGdFq_i4c=ei4ni5bv9nNEbCfNG4sbRkqyip2voW17GhkRAKPQ@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-17T18:33:07Z","receivedAt":"2012-10-17T18:33:07Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Wed, Oct 17, 2012 at 8:18 PM, Sverre Rabbelier wrote:\n> On Wed, Oct 17, 2012 at 11:12 AM, Felipe Contreras\n> wrote:\n>> But fine, lets remove the tests out of the equation (150 lines), the\n>> number of lines of code still exceeds 3000.\n>\n> I don't think it's fair to just look at LOC, git-remote-hg when it was\n> just parsing was fairly simple. Most of the current code is our copy\n> of the python fast-import library which is only used to support\n> pushing to mercurial.\n\nWell, as a rule of thumb more code means more places for bugs to hide.\nIt is also quite frankly rather difficult to navigate; very\nspaghetti-like. I have the feeling I can implement the same\nfast-import functionality in a much simpler way, but for now I want to\nconcentrate on fetching, and hopefully making it easy for people to\nactually use it.\n\nI would like to cooperate as much as possible, but as I said, I would\nrewrite a lot of that code. And also, I'm not even sure which\nrepository contains the latest version of this code. I've seen a\ncouple of them that are quite different, and none which are based on a\nrecent version of git.\n\nCheers.\n\n-- \nFelipe Contreras\n"},{"id":"201454","messageId":"20121017225913.GC21742@sigill.intra.peff.net","threadId":"31849","inReplyTo":"1350478721-3685-1-git-send-email-felipe.contreras@gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-10-17T22:59:13Z","receivedAt":"2012-10-17T22:59:13Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Oct 17, 2012 at 02:58:41PM +0200, Felipe Contreras wrote:\n\n> I've looked at many hg<->git tools and none satisfy me. Too complicated, or too\n> slow, or to difficult to setup, etc.\n\nI run into this every few months, evaluate all of the options, and come\nto the same conclusion. So I am excited at the prospect of something\nsimple that just works out of the box.\n\nUnfortunately, when I tried it, it did not work for me. :(\n\nDetails below.\n\n> contrib/remote-hd/git-remote-hg | 231 ++++++++++++++++++++++++++++++++++++++++\n> 1 file changed, 231 insertions(+)\n> create mode 100755 contrib/remote-hd/git-remote-hg\n\nIs this \"hd\" a typo, or is there something clever I am missing?\n\n> --- /dev/null\n> +++ b/contrib/remote-hd/git-remote-hg\n> @@ -0,0 +1,231 @@\n> +#!/usr/bin/python2\n\nI do not have /usr/bin/python2. I do have (on my Debian box):\n\n $ ls -l /usr/bin/python* | perl -lne 'print $& if m{/.*}'\n /usr/bin/python -> python2.7\n /usr/bin/python2.6\n /usr/bin/python2.7\n /usr/bin/python3 -> python3.2\n /usr/bin/python3.2 -> python3.2mu\n /usr/bin/python3.2mu\n /usr/bin/python3mu -> python3.2mu\n\nObviously a minor, easily fixable issue, but I wonder if it should ship\nwith a more portable default (like just \"/usr/bin/python\", or even\n\"/usr/bin/env python\").\n\n> +# Inspired by Rocco Rutte's hg-fast-export\n> +\n> +# Just copy to your ~/bin, or anywhere in your $PATH.\n> +# Then you can clone with:\n> +# hg::file:///path/to/mercurial/repo/\n\nThe first thing I tried was:\n\n $ git clone hg::https://code.google.com/p/dactyl/ \n Cloning into 'dactyl'...\n fatal: Unable to find remote helper for 'hg'\n sigill:~/compile/dactyl$ git clone hg::https://code.google.com/p/dactyl/ \n Cloning into 'dactyl'...\n Traceback (most recent call last):\n File \"/home/peff/local/bin/git-remote-hg\", line 231, in \n sys.exit(main(sys.argv))\n File \"/home/peff/local/bin/git-remote-hg\", line 222, in main\n do_list(repo, args)\n File \"/home/peff/local/bin/git-remote-hg\", line 159, in do_list\n head = repo.dirstate.branch()\n AttributeError: 'httpsrepository' object has no attribute 'dirstate'\n\nSo we are failing here:\n\n> +def do_list(repo, args):\n> + global branches\n> +\n> + head = repo.dirstate.branch()\n> + for branch in repo.branchmap():\n> + heads = repo.branchheads(branch)\n> + if len(heads):\n> + branches[branch] = heads\n\nIs there a way to get this information for remote repos?\n\nI worked around it by doing an hg-clone and trying to git-clone from\nthat local clone. But that didn't work either:\n\n $ hg clone https://code.google.com/p/dactyl/ hg\n [... clone eventually completes ...]\n\n $ git clone hg::$PWD/hg git\n Cloning into 'git'...\n progress revision 99 'pentadactyl-1.0b5-branch' (100/5367)\n [... many more progress updates ...]\n progress revision 6766 'cpg-hack' (1400/1467)\n ERROR: Branch 'default' has more than one head\n error: refs/tags/VIMPERATOR_2_2_b1 does not point to a valid object!\n error: refs/tags/muttator-0.5 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b1 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b2 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b3 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b4 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b4.1 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b4.2 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b4.3 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b5 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b5.1 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b6 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b7 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0b7.1 does not point to a valid object!\n error: refs/tags/pentadactyl-1.0rc1 does not point to a valid object!\n error: refs/tags/vimperator-0.4.1 does not point to a valid object!\n error: refs/tags/vimperator-0.5 does not point to a valid object!\n error: refs/tags/vimperator-0.5-branch-HEAD-merge-1 does not point to a valid object!\n error: refs/tags/vimperator-0.5.1 does not point to a valid object!\n error: refs/tags/vimperator-0.5.2 does not point to a valid object!\n error: refs/tags/vimperator-0.5.3 does not point to a valid object!\n error: refs/tags/vimperator-1.0 does not point to a valid object!\n error: refs/tags/vimperator-1.1 does not point to a valid object!\n error: refs/tags/vimperator-1.2 does not point to a valid object!\n error: refs/tags/vimperator-2.0 does not point to a valid object!\n error: refs/tags/vimperator-2.0a1 does not point to a valid object!\n error: refs/tags/vimperator-2.1 does not point to a valid object!\n error: refs/tags/vimperator-2.2 does not point to a valid object!\n error: refs/tags/vimperator-2.2b1 does not point to a valid object!\n error: refs/tags/xulmus-0.1 does not point to a valid object!\n\nI seem to remember getting this with other importers, too (probably\nbecause they were also based on the same script).\n\nWe do not need to fix every bug before bringing a script into git\n(especially into contrib/), but I am wondering if this script errs too\nmuch on the side of \"simple\" and not enough on \"works out of the box\".\nMaybe this repo is really complex and unusual, and the multi-heads thing\nis not common enough to worry about. But I feel cloning a remote is the\nfirst thing most people are going to try, and it doesn't work.\n\n-Peff\n"},{"id":"201463","messageId":"CAMP44s0+Fhtj2rMQ1Av-49Koa=DumX8JZs5angOFSRzqtDc+9Q@mail.gmail.com","threadId":"31849","inReplyTo":"20121017225913.GC21742@sigill.intra.peff.net","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-18T03:44:30Z","receivedAt":"2012-10-18T03:44:30Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 12:59 AM, Jeff King wrote:\n> On Wed, Oct 17, 2012 at 02:58:41PM +0200, Felipe Contreras wrote:\n>\n>> I've looked at many hg<->git tools and none satisfy me. Too complicated, or too\n>> slow, or to difficult to setup, etc.\n>\n> I run into this every few months, evaluate all of the options, and come\n> to the same conclusion. So I am excited at the prospect of something\n> simple that just works out of the box.\n>\n> Unfortunately, when I tried it, it did not work for me. :(\n>\n> Details below.\n>\n>> contrib/remote-hd/git-remote-hg | 231 ++++++++++++++++++++++++++++++++++++++++\n>> 1 file changed, 231 insertions(+)\n>> create mode 100755 contrib/remote-hd/git-remote-hg\n>\n> Is this \"hd\" a typo, or is there something clever I am missing?\n\nYeah, I've fixed that now.\n\n>> --- /dev/null\n>> +++ b/contrib/remote-hd/git-remote-hg\n>> @@ -0,0 +1,231 @@\n>> +#!/usr/bin/python2\n>\n> I do not have /usr/bin/python2. I do have (on my Debian box):\n>\n> $ ls -l /usr/bin/python* | perl -lne 'print $& if m{/.*}'\n> /usr/bin/python -> python2.7\n> /usr/bin/python2.6\n> /usr/bin/python2.7\n> /usr/bin/python3 -> python3.2\n> /usr/bin/python3.2 -> python3.2mu\n> /usr/bin/python3.2mu\n> /usr/bin/python3mu -> python3.2mu\n>\n> Obviously a minor, easily fixable issue, but I wonder if it should ship\n> with a more portable default (like just \"/usr/bin/python\", or even\n> \"/usr/bin/env python\").\n\nYeah, this has always been an issue in Arch Linux; I have to compile\ngit by exporting PYTHON_PATH.\n\nI'm OK with using any of the two above suggestions above, as they are\nmore standard.\n\n>> +# Inspired by Rocco Rutte's hg-fast-export\n>> +\n>> +# Just copy to your ~/bin, or anywhere in your $PATH.\n>> +# Then you can clone with:\n>> +# hg::file:///path/to/mercurial/repo/\n>\n> The first thing I tried was:\n>\n> $ git clone hg::https://code.google.com/p/dactyl/\n\nRight, doesn't look like it works for remote repositories. I think\nthat's the next feature I want to implement, but to be honest, I don't\nthink it's a big issue. To replace this:\n\ngit clone hg::https://code.google.com/p/dactyl/\n\nWith this\n\nhg clone https://code.google.com/p/dactyl/\ngit clone hg::dactyl dactyl-git\n\nWe could do what other tools do, manually clone the repository and\nstore it internally, but I'll rather not trick the users this way.\n\n> I worked around it by doing an hg-clone and trying to git-clone from\n> that local clone. But that didn't work either:\n>\n> $ hg clone https://code.google.com/p/dactyl/ hg\n> [... clone eventually completes ...]\n>\n> $ git clone hg::$PWD/hg git\n> Cloning into 'git'...\n> progress revision 99 'pentadactyl-1.0b5-branch' (100/5367)\n> [... many more progress updates ...]\n> progress revision 6766 'cpg-hack' (1400/1467)\n> ERROR: Branch 'default' has more than one head\n\nYes, this is deliberate, we can't have more than one head per branch in git.\n\nWhat you should do is go to the mercurial repo, and 'hg merge' (I think).\n\nWe could just pick the first head, and warn the user instead.\n\nBut at the moment it should fail at this point, I wonder why you get\nthe errors below.\n\n> error: refs/tags/VIMPERATOR_2_2_b1 does not point to a valid object!\n> error: refs/tags/muttator-0.5 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b1 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b2 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b3 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b4 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b4.1 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b4.2 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b4.3 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b5 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b5.1 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b6 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b7 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0b7.1 does not point to a valid object!\n> error: refs/tags/pentadactyl-1.0rc1 does not point to a valid object!\n> error: refs/tags/vimperator-0.4.1 does not point to a valid object!\n> error: refs/tags/vimperator-0.5 does not point to a valid object!\n> error: refs/tags/vimperator-0.5-branch-HEAD-merge-1 does not point to a valid object!\n> error: refs/tags/vimperator-0.5.1 does not point to a valid object!\n> error: refs/tags/vimperator-0.5.2 does not point to a valid object!\n> error: refs/tags/vimperator-0.5.3 does not point to a valid object!\n> error: refs/tags/vimperator-1.0 does not point to a valid object!\n> error: refs/tags/vimperator-1.1 does not point to a valid object!\n> error: refs/tags/vimperator-1.2 does not point to a valid object!\n> error: refs/tags/vimperator-2.0 does not point to a valid object!\n> error: refs/tags/vimperator-2.0a1 does not point to a valid object!\n> error: refs/tags/vimperator-2.1 does not point to a valid object!\n> error: refs/tags/vimperator-2.2 does not point to a valid object!\n> error: refs/tags/vimperator-2.2b1 does not point to a valid object!\n> error: refs/tags/xulmus-0.1 does not point to a valid object!\n\nThis is weird.\n\n> I seem to remember getting this with other importers, too (probably\n> because they were also based on the same script).\n\nhg-fast-import also fails if there are multiple heads, also deliberately.\n\n> We do not need to fix every bug before bringing a script into git\n> (especially into contrib/), but I am wondering if this script errs too\n> much on the side of \"simple\" and not enough on \"works out of the box\".\n> Maybe this repo is really complex and unusual, and the multi-heads thing\n> is not common enough to worry about. But I feel cloning a remote is the\n> first thing most people are going to try, and it doesn't work.\n\nWell, maybe the next time you try the branch will be merged. In\nmercurial they even warn you about this when you do 'hg push', so it's\nnot a good practice that they are doing that in their own repo.\n\nAs for remote repos, I don't know what we should be doing at the moment.\n\nCheers.\n\n-- \nFelipe Contreras\n"},{"id":"201465","messageId":"CAMP44s1b=dNaCbm1WkFBv6368Y+jDYdkPuscEFdiUnVJBStfVg@mail.gmail.com","threadId":"31849","inReplyTo":"CAMP44s0+Fhtj2rMQ1Av-49Koa=DumX8JZs5angOFSRzqtDc+9Q@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-18T05:18:49Z","receivedAt":"2012-10-18T05:18:49Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 5:44 AM, Felipe Contreras\n wrote:\n> On Thu, Oct 18, 2012 at 12:59 AM, Jeff King wrote:\n>> On Wed, Oct 17, 2012 at 02:58:41PM +0200, Felipe Contreras wrote:\n>>\n>>> I've looked at many hg<->git tools and none satisfy me. Too complicated, or too\n>>> slow, or to difficult to setup, etc.\n>>\n>> I run into this every few months, evaluate all of the options, and come\n>> to the same conclusion. So I am excited at the prospect of something\n>> simple that just works out of the box.\n>>\n>> Unfortunately, when I tried it, it did not work for me. :(\n\nOk, I've fixed all those issues:\nhttp://github.com/felipec/git/blob/fc-remote-hg/contrib/remote-hg/git-remote-hg\n\nRight now I've just added an error when using remote repositories. But\nit seems there's no way around it; if we want to have support for\nremote repos, we need to make a local clone.\n\n> But at the moment it should fail at this point, I wonder why you get\n> the errors below.\n>\n>> error: refs/tags/VIMPERATOR_2_2_b1 does not point to a valid object!\n>> error: refs/tags/muttator-0.5 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b1 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b2 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b3 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b4 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b4.1 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b4.2 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b4.3 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b5 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b5.1 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b6 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b7 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0b7.1 does not point to a valid object!\n>> error: refs/tags/pentadactyl-1.0rc1 does not point to a valid object!\n>> error: refs/tags/vimperator-0.4.1 does not point to a valid object!\n>> error: refs/tags/vimperator-0.5 does not point to a valid object!\n>> error: refs/tags/vimperator-0.5-branch-HEAD-merge-1 does not point to a valid object!\n>> error: refs/tags/vimperator-0.5.1 does not point to a valid object!\n>> error: refs/tags/vimperator-0.5.2 does not point to a valid object!\n>> error: refs/tags/vimperator-0.5.3 does not point to a valid object!\n>> error: refs/tags/vimperator-1.0 does not point to a valid object!\n>> error: refs/tags/vimperator-1.1 does not point to a valid object!\n>> error: refs/tags/vimperator-1.2 does not point to a valid object!\n>> error: refs/tags/vimperator-2.0 does not point to a valid object!\n>> error: refs/tags/vimperator-2.0a1 does not point to a valid object!\n>> error: refs/tags/vimperator-2.1 does not point to a valid object!\n>> error: refs/tags/vimperator-2.2 does not point to a valid object!\n>> error: refs/tags/vimperator-2.2b1 does not point to a valid object!\n>> error: refs/tags/xulmus-0.1 does not point to a valid object!\n>\n> This is weird.\n\nI think I know why the errors above show up; even though the helper\ndies, transport-helper doesn't check the status until the very end.\n\nSomething like this should do the trick:\n\ndiff --git a/run-command.c b/run-command.c\nindex 1101ef7..0a859ca 100644\n--- a/run-command.c\n+++ b/run-command.c\n@@ -559,6 +559,21 @@ int run_command(struct child_process *cmd)\n \treturn finish_command(cmd);\n }\n\n+int check_command(struct child_process *cmd)\n+{\n+\tint status;\n+\tpid_t pid;\n+\n+\tpid = waitpid(cmd->pid, &status, WNOHANG);\n+\n+\tif (pid != cmd->pid)\n+\t\treturn -1;\n+\tif (WIFSIGNALED(status))\n+\t\treturn WTERMSIG(status);\n+\tif (WIFEXITED(status))\n+\t\treturn WEXITSTATUS(status);\n+}\n+\n static void prepare_run_command_v_opt(struct child_process *cmd,\n \t\t\t\t const char **argv,\n \t\t\t\t int opt)\ndiff --git a/run-command.h b/run-command.h\nindex 44f7d2b..9019e38 100644\n--- a/run-command.h\n+++ b/run-command.h\n@@ -45,6 +45,7 @@ struct child_process {\n int start_command(struct child_process *);\n int finish_command(struct child_process *);\n int run_command(struct child_process *);\n+int check_command(struct child_process *cmd);\n\n extern int run_hook(const char *index_file, const char *name, ...);\n\ndiff --git a/transport-helper.c b/transport-helper.c\nindex cfe0988..bc1349d 100644\n--- a/transport-helper.c\n+++ b/transport-helper.c\n@@ -441,6 +441,10 @@ static int fetch_with_import(struct transport *transport,\n\n \tif (finish_command(&fastimport))\n \t\tdie(\"Error while running fast-import\");\n+\n+\tif (check_command(data->helper))\n+\t\tdie(\"Error while running helper\");\n+\n \tfree(fastimport.argv);\n \tfastimport.argv = NULL;\n\n@@ -784,6 +788,10 @@ static int push_refs_with_export(struct transport\n*transport,\n\n \tif (finish_command(&exporter))\n \t\tdie(\"Error while running fast-export\");\n+\n+\tif (check_command(data->helper))\n+\t\tdie(\"Error while running helper\");\n+\n \tpush_update_refs_status(data, remote_refs);\n \treturn 0;\n }\n\n-- \nFelipe Contreras\n"},{"id":"201477","messageId":"CAGdFq_hsCjL5uunwg=N-y+FkFt-ZxORtrEcxAAHt-rDfA04cZw@mail.gmail.com","threadId":"31849","inReplyTo":"CAMP44s1b=dNaCbm1WkFBv6368Y+jDYdkPuscEFdiUnVJBStfVg@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2012-10-18T06:12:16Z","receivedAt":"2012-10-18T06:12:16Z","isPatch":true,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"On Wed, Oct 17, 2012 at 10:18 PM, Felipe Contreras\n wrote:\n> Right now I've just added an error when using remote repositories. But\n> it seems there's no way around it; if we want to have support for\n> remote repos, we need to make a local clone.\n\nMy git-remote-hg does the local clone into .git/ using a hash of the\nurl (although you could just as well use urlencode, basically any way\nto safely use a url as a directory name). Have a look if you want.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"201485","messageId":"alpine.DEB.1.00.1210181031320.3049@bonsai2","threadId":"31849","inReplyTo":"CAMP44s2-BCuA5v7jE8S3d-Gg+DxKf6Yte9MvTDBDr4jEkLKZ9g@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2012-10-18T08:47:07Z","receivedAt":"2012-10-18T08:47:07Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Felipe,\n\nOn Wed, 17 Oct 2012, Felipe Contreras wrote:\n\n> On Wed, Oct 17, 2012 at 8:18 PM, Sverre Rabbelier wrote:\n> > On Wed, Oct 17, 2012 at 11:12 AM, Felipe Contreras\n> > wrote:\n> >> But fine, lets remove the tests out of the equation (150 lines), the\n> >> number of lines of code still exceeds 3000.\n> >\n> > I don't think it's fair to just look at LOC, git-remote-hg when it was\n> > just parsing was fairly simple. Most of the current code is our copy\n> > of the python fast-import library which is only used to support\n> > pushing to mercurial.\n> \n> Well, as a rule of thumb more code means more places for bugs to hide.\n\nEverybody on this list knows that. But it is equally true that more\nfunctionality requires more code.\n\nBesides, we are talking about concrete code, so there is no need at all\nfor handwaving arguments. GitHub makes it easy to point at exact line\nnumbers in exact file names in exact revisions, as you know, and we should\nuse that to discuss code.\n\n> It is also quite frankly rather difficult to navigate; very\n> spaghetti-like. I have the feeling [...]\n\nYours truly always welcomes constructive criticism. Other types of\ncriticism, not so much.\n\nAs to the functionality you seek: git-remote-hg found in\ngit://github.com/msysgit/git works. It has the following advantages over\nevery other solution, including the one proposed in this thread:\n\n- it works\n\n- no really, it works\n\n- it supports pushes, too\n\n- it matured over a long time\n\n- there are tests\n\n- whenever we fixed bugs, we also added tests for the bug fixes\n\n- it is rock solid\n\n- it is in constant use\n\nWithout push support, remote-hg is useless to me. Without regression tests\nproving that it is rock solid, I will not use remote-hg. And I will not\nindulge in efforts to duplicate work.\n\nIf there are concerns about code style or unnecessary code (insofar it is\nreally unnecessary, testgit for example is not, unless you want to avoid\nrobust regression tests), I will discuss issues and collaborate. If the\nidea was not to collaborate, but to show off how much shorter code can be\nwhen it lacks functionality and proof of robustness I require for my\neveryday use of the program, dismissing existing code and concepts, less\nso.\n\nHth,\nJohannes\n"},{"id":"201486","messageId":"CAMP44s3=pv4kHiZ2KM+e1tnGo=38oy76Sm3i3VMDFseWXZH85Q@mail.gmail.com","threadId":"31849","inReplyTo":"CAMP44s0+Fhtj2rMQ1Av-49Koa=DumX8JZs5angOFSRzqtDc+9Q@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-18T08:48:51Z","receivedAt":"2012-10-18T08:48:51Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 5:44 AM, Felipe Contreras\n wrote:\n> On Thu, Oct 18, 2012 at 12:59 AM, Jeff King wrote:\n\n>> The first thing I tried was:\n>>\n>> $ git clone hg::https://code.google.com/p/dactyl/\n>\n> Right, doesn't look like it works for remote repositories. I think\n> that's the next feature I want to implement, but to be honest, I don't\n> think it's a big issue. To replace this:\n\nDone, now you should be able to clone and fetch remote repositories :)\nhttps://github.com/felipec/git/commit/783e4b380ab4fabb4e2fb200722c92afc8494a83\n\n-- \nFelipe Contreras\n"},{"id":"201487","messageId":"CAMP44s2wOX0-UrTM5t5ewZ4yhOJJU=+SfB9PoTQKdsP4Pzsnpg@mail.gmail.com","threadId":"31849","inReplyTo":"alpine.DEB.1.00.1210181031320.3049@bonsai2","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-18T09:03:56Z","receivedAt":"2012-10-18T09:03:56Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 10:47 AM, Johannes Schindelin\n wrote:\n> Hi Felipe,\n>\n> On Wed, 17 Oct 2012, Felipe Contreras wrote:\n>\n>> On Wed, Oct 17, 2012 at 8:18 PM, Sverre Rabbelier wrote:\n>> > On Wed, Oct 17, 2012 at 11:12 AM, Felipe Contreras\n>> > wrote:\n>> >> But fine, lets remove the tests out of the equation (150 lines), the\n>> >> number of lines of code still exceeds 3000.\n>> >\n>> > I don't think it's fair to just look at LOC, git-remote-hg when it was\n>> > just parsing was fairly simple. Most of the current code is our copy\n>> > of the python fast-import library which is only used to support\n>> > pushing to mercurial.\n>>\n>> Well, as a rule of thumb more code means more places for bugs to hide.\n>\n> Everybody on this list knows that. But it is equally true that more\n> functionality requires more code.\n\nNot necessarily. There's projects with more code and less\nfunctionality than their alternatives.\n\n>> It is also quite frankly rather difficult to navigate; very\n>> spaghetti-like. I have the feeling [...]\n>\n> Yours truly always welcomes constructive criticism. Other types of\n> criticism, not so much.\n>\n> As to the functionality you seek: git-remote-hg found in\n> git://github.com/msysgit/git works. It has the following advantages over\n> every other solution, including the one proposed in this thread:\n>\n> - it works\n>\n> - no really, it works\n\nNot for me.\n\n> - it supports pushes, too\n\nI don't care. When I need that I'll implement that with probably much less code.\n\n> - it matured over a long time\n\nSo has CVS.\n\n> - there are tests\n\nOnly a dozen of them. If I write the same tests for my solution would\nyou be happier? I didn't think so.\n\n> - whenever we fixed bugs, we also added tests for the bug fixes\n\nLike this one?\nhttps://github.com/msysgit/git/commit/9f934c9987981cbecf4ebaf8eb4a8e9f1d002caf\n\nI don't see any tests for that.\n\n> - it is in constant use\n\nSo you say, my impression is different.\n\n> Without push support, remote-hg is useless to me.\n\nDifferent people have different needs.\n\nWithout an easy way to setup, remote-hg is useless to me.\n\n> If there are concerns about code style or unnecessary code (insofar it is\n> really unnecessary, testgit for example is not, unless you want to avoid\n> robust regression tests), I will discuss issues and collaborate. If the\n> idea was not to collaborate, but to show off how much shorter code can be\n> when it lacks functionality and proof of robustness I require for my\n> everyday use of the program, dismissing existing code and concepts, less\n> so.\n\nSo your idea of collaboration is accept that your code is awesome, and\nmy code sucks, and that I should fix your code, and throw my code to\nthe trash, while you do absolutely nothing but complain about the\nwhole situation. I have at least looked at your code. Have you even\nlooked at mine?\n\nI've done my part in making my code easily available and ready for\nreview. I will not reply to you anymore until you show your\nwillingness to collaborate that you seem to demand for me, and:\n\n1) Point to a remote-hg branch that is independent of msysgit stuff,\nor any other irrelevant stuff\n2) Is based on top of a recent version of git\n\nCheers.\n\n-- \nFelipe Contreras\n"},{"id":"201490","messageId":"alpine.DEB.1.00.1210181109120.3049@bonsai2","threadId":"31849","inReplyTo":"CAMP44s2wOX0-UrTM5t5ewZ4yhOJJU=+SfB9PoTQKdsP4Pzsnpg@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2012-10-18T09:10:02Z","receivedAt":"2012-10-18T09:10:02Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Felipe,\n\nOn Thu, 18 Oct 2012, Felipe Contreras wrote:\n\n> On Thu, Oct 18, 2012 at 10:47 AM, Johannes Schindelin\n> wrote:\n>\n> So your idea of collaboration is accept that your code is awesome, and\n> my code sucks, and that I should fix your code, and throw my code to the\n> trash, while you do absolutely nothing but complain about the whole\n> situation.\n\nI said no such thing. I like to keep things professional and keep emotions\nout.\n\nHth,\nJohannes\n"},{"id":"201488","messageId":"CAMP44s3+FP2N4iEmxzcu_ayUAF+XWmKdBFDCvFvj6D5RQmN+5Q@mail.gmail.com","threadId":"31849","inReplyTo":"CAGdFq_hsCjL5uunwg=N-y+FkFt-ZxORtrEcxAAHt-rDfA04cZw@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-18T09:10:35Z","receivedAt":"2012-10-18T09:10:35Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 8:12 AM, Sverre Rabbelier wrote:\n> On Wed, Oct 17, 2012 at 10:18 PM, Felipe Contreras\n> wrote:\n>> Right now I've just added an error when using remote repositories. But\n>> it seems there's no way around it; if we want to have support for\n>> remote repos, we need to make a local clone.\n>\n> My git-remote-hg does the local clone into .git/ using a hash of the\n> url (although you could just as well use urlencode, basically any way\n> to safely use a url as a directory name). Have a look if you want.\n\nCan you point to the version you are talking about? I've been checking\nthe remote-hg branch of fingolfin.\n\nhttps://github.com/fingolfin/git/tree/remote-hg/\n\n-- \nFelipe Contreras\n"},{"id":"201492","messageId":"alpine.DEB.1.00.1210181112050.3049@bonsai2","threadId":"31849","inReplyTo":"CAMP44s3+FP2N4iEmxzcu_ayUAF+XWmKdBFDCvFvj6D5RQmN+5Q@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2012-10-18T09:13:19Z","receivedAt":"2012-10-18T09:13:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 18 Oct 2012, Felipe Contreras wrote:\n\n> On Thu, Oct 18, 2012 at 8:12 AM, Sverre Rabbelier wrote:\n> > On Wed, Oct 17, 2012 at 10:18 PM, Felipe Contreras\n> > wrote:\n> >> Right now I've just added an error when using remote repositories.\n> >> But it seems there's no way around it; if we want to have support for\n> >> remote repos, we need to make a local clone.\n> >\n> > My git-remote-hg does the local clone into .git/ using a hash of the\n> > url (although you could just as well use urlencode, basically any way\n> > to safely use a url as a directory name). Have a look if you want.\n> \n> Can you point to the version you are talking about? I've been checking\n> the remote-hg branch of fingolfin.\n> \n> https://github.com/fingolfin/git/tree/remote-hg/\n\nThe code is in https://github.com/msysgit/git/ (Sverre does not have\nenough time to work on remote-hg, and was okay with me hosting it in Git\nfor Windows, for all the reasons I mentioned earlier in this thread).\n\nHth,\nJohannes\n"},{"id":"201493","messageId":"CAMP44s2Fd2aO_CcNrPJYGjo6thp3YXmN-dv4agd5qQEJQvqp0w@mail.gmail.com","threadId":"31849","inReplyTo":"alpine.DEB.1.00.1210181112050.3049@bonsai2","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-18T09:22:01Z","receivedAt":"2012-10-18T09:22:01Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 11:13 AM, Johannes Schindelin\n wrote:\n> Hi,\n>\n> On Thu, 18 Oct 2012, Felipe Contreras wrote:\n>\n>> On Thu, Oct 18, 2012 at 8:12 AM, Sverre Rabbelier wrote:\n>> > On Wed, Oct 17, 2012 at 10:18 PM, Felipe Contreras\n>> > wrote:\n>> >> Right now I've just added an error when using remote repositories.\n>> >> But it seems there's no way around it; if we want to have support for\n>> >> remote repos, we need to make a local clone.\n>> >\n>> > My git-remote-hg does the local clone into .git/ using a hash of the\n>> > url (although you could just as well use urlencode, basically any way\n>> > to safely use a url as a directory name). Have a look if you want.\n>>\n>> Can you point to the version you are talking about? I've been checking\n>> the remote-hg branch of fingolfin.\n>>\n>> https://github.com/fingolfin/git/tree/remote-hg/\n>\n> The code is in https://github.com/msysgit/git/ (Sverre does not have\n> enough time to work on remote-hg, and was okay with me hosting it in Git\n> for Windows, for all the reasons I mentioned earlier in this thread).\n\nWhich branch? I don't see any remote-hg branch.\n\n-- \nFelipe Contreras\n"},{"id":"201494","messageId":"7vbog0m8r0.fsf@alter.siamese.dyndns.org","threadId":"31849","inReplyTo":"CAMP44s2wOX0-UrTM5t5ewZ4yhOJJU=+SfB9PoTQKdsP4Pzsnpg@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-18T09:26:11Z","receivedAt":"2012-10-18T09:26:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Felipe Contreras writes:\n\n>> As to the functionality you seek: git-remote-hg found in\n>> git://github.com/msysgit/git works. It has the following advantages over\n>> every other solution, including the one proposed in this thread:\n>>\n>> - it works\n>>\n>> - no really, it works\n>\n> Not for me.\n\nFelipe, an argument along this line would not help very much.\n\nPlease elaborate a bit to describe what does not work and where you\nare having problems more concretely. Even for people who may want\nto see if they agree with your \"It does not work\", \"Not for me\"\nalone is too little for them to try out. Others who may want to and\nare capable of helping you won't know where to start.\n\nThanks.\n"},{"id":"201495","messageId":"CAMP44s0yqBFWFyYx6VdNhq43U1u2S-OPrbKAyMWBGS1n=ffQMg@mail.gmail.com","threadId":"31849","inReplyTo":"7vbog0m8r0.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-18T09:38:16Z","receivedAt":"2012-10-18T09:38:16Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 11:26 AM, Junio C Hamano wrote:\n> Felipe Contreras writes:\n>\n>>> As to the functionality you seek: git-remote-hg found in\n>>> git://github.com/msysgit/git works. It has the following advantages over\n>>> every other solution, including the one proposed in this thread:\n>>>\n>>> - it works\n>>>\n>>> - no really, it works\n>>\n>> Not for me.\n>\n> Felipe, an argument along this line would not help very much.\n>\n> Please elaborate a bit to describe what does not work and where you\n> are having problems more concretely. Even for people who may want\n> to see if they agree with your \"It does not work\", \"Not for me\"\n> alone is too little for them to try out. Others who may want to and\n> are capable of helping you won't know where to start.\n\nBasically what I already described:\n1) You need a non-vanilla version of git\n2) It's not easy to set up\n3) It's not even clear which branch you should be using (in case you\nare not using msysgit)\n\n-- \nFelipe Contreras\n"},{"id":"201496","messageId":"vpqpq4gi0b7.fsf@grenoble-inp.fr","threadId":"31849","inReplyTo":"CAMP44s0yqBFWFyYx6VdNhq43U1u2S-OPrbKAyMWBGS1n=ffQMg@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@grenoble-inp.fr","sentAt":"2012-10-18T09:42:04Z","receivedAt":"2012-10-18T09:42:04Z","isPatch":true,"sender":{"key":"matthieu.moy@grenoble-inp.fr","avatar":"https://gravatar.com/avatar/72c8a2705971a25dfaff23cece15130d405685845d911aedd5667ace277f3fc5?d=mp&s=160"},"body":"Felipe Contreras writes:\n\n> Basically what I already described:\n> 1) You need a non-vanilla version of git\n> 2) It's not easy to set up\n> 3) It's not even clear which branch you should be using (in case you\n> are not using msysgit)\n\nI do not read that as \"it does not work\", but instead as \"no one took\nthe time to push this code into git.git (although someone did in\nmsysgit)\".\n\n-- \nMatthieu Moy\nhttp://www-verimag.imag.fr/~moy/\n"},{"id":"201511","messageId":"50800197.90105@drmicha.warpmail.net","threadId":"31849","inReplyTo":"1350478721-3685-1-git-send-email-felipe.contreras@gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2012-10-18T13:18:15Z","receivedAt":"2012-10-18T13:18:15Z","isPatch":true,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Felipe Contreras venit, vidit, dixit 17.10.2012 14:58:\n> Signed-off-by: Felipe Contreras \n> ---\n> \n> I've looked at many hg<->git tools and none satisfy me. Too complicated, or too\n> slow, or to difficult to setup, etc.\n\nIt's in an unsatisfying state, I agree. We have a great remote helper\ninfrastructure, but neither git-svn nor git-cvsimport/export use it, and\nremote-hg is not in git.git.\n\ngit-svn used to be our killer feature! (It's becoming hard to maintain,\nthough.)\n\ngit-hg (in the shape of a remote helper) could be our next killer\nfeature, finally leading to our world domination ;)\n\n> The only one I've liked so far is hg-fast-export[1], which is indeed fast,\n> relatively simple, and relatively easy to use. But it's not properly maintained\n> any more.\n> \n> So, I decided to write my own from scratch, using hg-fast-export as\n> inspiration, and voila.\n> \n> This one doesn't have any dependencies, just put it into your $PATH, and you\n> can clone and fetch hg repositories. More importantly to me; the code is\n> simple, and easy to maintain.\n\nWell, it still has hg as a dependency. All our remote\nintegration/helpers suffer from that. At least, every hg install comes\nwith the modules, whereas svn is a beast (apr and such) that often comes\nwithout the language bindings.\n\n> [1] http://repo.or.cz/w/fast-export.git\n> \n> contrib/remote-hd/git-remote-hg | 231 ++++++++++++++++++++++++++++++++++++++++\n> 1 file changed, 231 insertions(+)\n> create mode 100755 contrib/remote-hd/git-remote-hg\n\nThat diffstat looks great (sans tests, of course; it's contrib), but\nthere's no push functionality, and that is usually the most difficult\npart all helpers have to implement: Not only the push interaction, but\nalso making sure that commits don't get duped in a roundtrip (git fetch\nfrom vcs, push to vcs, git fetch from vcs).\n\nJust cloning and fetching can be done most easily with a shared worktree\nand scripting around \"hg up\" and \"git commit -A\" in some flavour.\n\n> diff --git a/contrib/remote-hd/git-remote-hg b/contrib/remote-hd/git-remote-hg\n> new file mode 100755\n> index 0000000..9157b30\n> --- /dev/null\n> +++ b/contrib/remote-hd/git-remote-hg\n> @@ -0,0 +1,231 @@\n> +#!/usr/bin/python2\n> +\n> +# Inspired by Rocco Rutte's hg-fast-export\n> +\n> +# Just copy to your ~/bin, or anywhere in your $PATH.\n> +# Then you can clone with:\n> +# hg::file:///path/to/mercurial/repo/\n> +\n> +from mercurial import hg, ui\n> +\n> +import re\n> +import sys\n> +import os\n> +import json\n> +\n> +def die(msg, *args):\n> + print >> sys.stderr, 'ERROR:', msg % args\n> + sys.exit(1)\n\nWhile we don't have to code for py3, avoiding '>>' will help us later.\n(It got removed in py3.) sys.sdterr.write() should be most portable.\n\n> +def gitmode(flags):\n> + return 'l' in flags and '120000' or 'x' in flags and '100755' or '100644'\n> +\n> +def export_file(fc):\n> + if fc.path() == '.hgtags':\n\nIs this always relative? Just wondering, dunno.\n\n> + return\n> + d = fc.data()\n> + print \"M %s inline %s\" % (gitmode(fc.flags()), fc.path())\n> + print \"data %d\" % len(d)\n> + print d\n> +\n> +def get_filechanges(repo, ctx, parents):\n> + l = [repo.status(p, ctx)[:3] for p in parents]\n> + changed, added, removed = [sum(e, []) for e in zip(*l)]\n> + return added + changed, removed\n> +\n> +author_re = re.compile('^((.+?) )?(<.+?>)$')\n> +\n> +def fixup_user(user):\n> + user = user.replace('\"', '')\n> + m = author_re.match(user)\n> + if m:\n> + name = m.group(1)\n> + mail = m.group(3)\n> + else:\n> + name = user\n> + mail = None\n> +\n> + if not name:\n> + name = 'Unknown'\n> + if not mail:\n> + mail = ''\n> +\n> + return '%s %s' % (name, mail)\n> +\n> +def get_repo(path, alias):\n> + myui = ui.ui()\n> + myui.setconfig('ui', 'interactive', 'off')\n> + repo = hg.repository(myui, path)\n> + return repo\n\nIs there a reason to spell this out, e.g.: Why not\n\nreturn hg.repository(myui, path)\n\n> +\n> +def hg_branch(b):\n> + if b == 'master':\n> + return 'default'\n> + return b\n> +\n> +def git_branch(b):\n> + if b == 'default':\n> + return 'master'\n> + return b\n> +\n> +def export_tag(repo, tag):\n> + global prefix\n> + print \"reset %s/tags/%s\" % (prefix, tag)\n> + print \"from :%s\" % (repo[tag].rev() + 1)\n> + print\n> +\n> +def export_branch(repo, branch):\n> + global prefix, marks, cache, branches\n> +\n> + heads = branches[hg_branch(branch)]\n> +\n> + # verify there's only one head\n> + if (len(heads) > 1):\n> + die(\"Branch '%s' has more than one head\" % hg_branch(branch))\n\nWe have to deal with this at some point... Do you support \"hg\nbookmarks\"? They'd be an option, or we implement better detached head\nhandling in git...\n\n> +\n> + head = repo[heads[0]]\n> + tip = marks.get(branch, 0)\n> + # mercurial takes too much time checking this\n> + if tip == head.rev():\n> + # nothing to do\n> + return\n> + revs = repo.revs('%u:%u' % (tip, head))\n> + count = 0\n> +\n\n> + revs = [rev for rev in revs if not cache.get(rev, False)]\n> +\n> + for rev in revs:\n\nThose lines set up the list just so that you iterate over it later.\nPlease don't do this (unless you know that revs is very small).\n\nfor rev in revs:\n if cache.get(rev, False):\n continue\n\nis more performant. You can reduce this further to\n\ncount=0\nfor rev in repo.revs('%u:%u' % (tip, head)):\n if cache.get(rev, False):\n continue\n\nwhich is even more performant generally, and especially if repo.revs()\nreturns an iterator rather than a list.\n\n[Yes, you could use lambda+filter, but let's not get religious. The\nabove is simple and pythonic.]\n\n> +\n> + c = repo[rev]\n> + (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(c.node())\n\nSame here, you introduce c just for the next line (unless I'm mistaken).\n\n> + rev_branch = git_branch(extra['branch'])\n> +\n> + tz = '%+03d%02d' % (-tz / 3600, -tz % 3600 / 60)\n> +\n> + print \"commit %s/branches/%s\" % (prefix, rev_branch)\n> + print \"mark :%d\" % (rev + 1)\n> + print \"committer %s %d %s\" % (fixup_user(user), time, tz)\n> + print \"data %d\" % (len(desc) + 1)\n> + print desc\n> + print\n> +\n> + parents = [p for p in repo.changelog.parentrevs(rev) if p >= 0]\n> +\n> + if len(parents) == 0:\n> + modified = c.manifest().keys()\n> + removed = []\n> + else:\n> + added = []\n> + changed = []\n> + print \"from :%s\" % (parents[0] + 1)\n\nWhat is this +1 offset that is appearing here and there?\n\n> + if len(parents) > 1:\n> + print \"merge :%s\" % (parents[1] + 1)\n> + modified, removed = get_filechanges(repo, c, parents)\n> +\n> + for f in removed:\n> + print \"D %s\" % (f)\n> + for f in modified:\n> + export_file(c.filectx(f))\n> + print\n> +\n> + count += 1\n> + if (count % 100 == 0):\n> + print \"progress revision %d '%s' (%d/%d)\" % (rev, branch, count, len(revs))\n> + print \"#############################################################\"\n> +\n> + cache[rev] = True\n> +\n> + # store the latest revision\n> + marks[branch] = rev\n> +\n> +def do_capabilities(repo, args):\n> + global prefix, dirname\n> +\n> + print \"import\"\n> + print \"refspec refs/heads/*:%s/branches/*\" % prefix\n> + print \"refspec refs/tags/*:%s/tags/*\" % prefix\n> +\n> + path = os.path.join(dirname, 'marks-git')\n> +\n> + print \"*export-marks %s\" % path\n> + if os.path.exists(path):\n> + print \"*import-marks %s\" % path\n> +\n> + print\n> +\n> +def do_list(repo, args):\n> + global branches\n> +\n> + head = repo.dirstate.branch()\n> + for branch in repo.branchmap():\n> + heads = repo.branchheads(branch)\n> + if len(heads):\n> + branches[branch] = heads\n\nI'm a bit confused here. repo.brancheads() is a list, no? Is this the\nsingle head case only? I'd expect [0] of that, but you seem to be\ngetting branch names (strings).\n\nAlso, if len(heads) == 0 then branches[branch] is undefined or stale. No?\n\n> +\n> + for branch in branches:\n> + print \"? refs/heads/%s\" % git_branch(branch)\n> + for tag, node in repo.tagslist():\n> + if tag == 'tip':\n> + continue\n> + print \"? refs/tags/%s\" % tag\n> + print \"@refs/heads/%s HEAD\" % git_branch(head)\n> + print\n> +\n> +def do_import(repo, args):\n> + ref = args[1]\n> +\n> + if (ref == 'HEAD'):\n> + return\n> +\n> + if ref.startswith('refs/heads/'):\n> + branch = ref[len('refs/heads/'):]\n> + export_branch(repo, branch)\n> + elif ref.startswith('refs/tags/'):\n> + tag = ref[len('refs/tags/'):]\n> + export_tag(repo, tag)\n> +\n> +def main(args):\n> + global prefix, dirname, marks, cache, branches\n> +\n> + alias = args[1]\n> + url = args[2]\n> +\n> + gitdir = os.environ['GIT_DIR']\n> + dirname = os.path.join(gitdir, 'hg')\n> + cache = {}\n> + branches = {}\n> +\n> + repo = get_repo(url, alias)\n> + prefix = 'refs/hg/%s' % alias\n> +\n> + if not os.path.exists(dirname):\n> + os.makedirs(dirname)\n> +\n> + marks_path = os.path.join(dirname, 'marks-hg')\n> + try:\n> + fp = open(marks_path, 'r')\n> + marks = json.load(fp)\n> + fp.close()\n> + except IOError:\n> + marks = {}\n> +\n> + line = True\n> + while (line):\n> + line = sys.stdin.readline().strip()\n> + if line == '':\n> + break\n> + args = line.split()\n> + cmd = args[0]\n> + if cmd == 'capabilities':\n> + do_capabilities(repo, args)\n> + elif cmd == 'list':\n> + do_list(repo, args)\n> + elif cmd == 'import':\n> + do_import(repo, args)\n> + sys.stdout.flush()\n> +\n> + fp = open(marks_path, 'w')\n> + json.dump(marks, fp)\n> + fp.close()\n> +\n> +sys.exit(main(sys.argv))\n> \n\nOverall, this looks like plain scripting in python rather than anything\noo'ish, but that's okay and probably dictated by the remote helper\ninterface and/or the existing exporter.\n\nI'm all for an improvement in that area, but still feel we'd need a\ncombined effort rather than yet another start.\n\nMichael\n"},{"id":"201513","messageId":"CAMP44s2_jZRtG5bRCDsc8kO5EqFYLDWFQ5d6F9URrSQQNDqvNA@mail.gmail.com","threadId":"31849","inReplyTo":"50800197.90105@drmicha.warpmail.net","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-18T14:26:12Z","receivedAt":"2012-10-18T14:26:12Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 3:18 PM, Michael J Gruber\n wrote:\n> Felipe Contreras venit, vidit, dixit 17.10.2012 14:58:\n>> Signed-off-by: Felipe Contreras \n>> ---\n>>\n>> I've looked at many hg<->git tools and none satisfy me. Too complicated, or too\n>> slow, or to difficult to setup, etc.\n>\n> It's in an unsatisfying state, I agree. We have a great remote helper\n> infrastructure, but neither git-svn nor git-cvsimport/export use it, and\n> remote-hg is not in git.git.\n>\n> git-svn used to be our killer feature! (It's becoming hard to maintain,\n> though.)\n>\n> git-hg (in the shape of a remote helper) could be our next killer\n> feature, finally leading to our world domination ;)\n>\n>> The only one I've liked so far is hg-fast-export[1], which is indeed fast,\n>> relatively simple, and relatively easy to use. But it's not properly maintained\n>> any more.\n>>\n>> So, I decided to write my own from scratch, using hg-fast-export as\n>> inspiration, and voila.\n>>\n>> This one doesn't have any dependencies, just put it into your $PATH, and you\n>> can clone and fetch hg repositories. More importantly to me; the code is\n>> simple, and easy to maintain.\n>\n> Well, it still has hg as a dependency. All our remote\n> integration/helpers suffer from that. At least, every hg install comes\n> with the modules, whereas svn is a beast (apr and such) that often comes\n> without the language bindings.\n\nYeah, but there's no way around that, even if we write some code to\naccess the repository, it's quite likely that it will change. Unlike\ngit, mercurial developers expect people to access the repository\nthrough mercurial API, not directly, and that's probably what we\nshould do if we want to avoid problems.\n\n>> contrib/remote-hd/git-remote-hg | 231 ++++++++++++++++++++++++++++++++++++++++\n>> 1 file changed, 231 insertions(+)\n>> create mode 100755 contrib/remote-hd/git-remote-hg\n>\n> That diffstat looks great (sans tests, of course; it's contrib), but\n> there's no push functionality, and that is usually the most difficult\n> part all helpers have to implement: Not only the push interaction, but\n> also making sure that commits don't get duped in a roundtrip (git fetch\n> from vcs, push to vcs, git fetch from vcs).\n\nYeah, it will probably require much more code, but I think by far the\nmost important feature is fetching.\n\n> Just cloning and fetching can be done most easily with a shared worktree\n> and scripting around \"hg up\" and \"git commit -A\" in some flavour.\n\nYea, but that will be dead slow.\n\n>> diff --git a/contrib/remote-hd/git-remote-hg b/contrib/remote-hd/git-remote-hg\n>> new file mode 100755\n>> index 0000000..9157b30\n>> --- /dev/null\n>> +++ b/contrib/remote-hd/git-remote-hg\n>> @@ -0,0 +1,231 @@\n>> +#!/usr/bin/python2\n>> +\n>> +# Inspired by Rocco Rutte's hg-fast-export\n>> +\n>> +# Just copy to your ~/bin, or anywhere in your $PATH.\n>> +# Then you can clone with:\n>> +# hg::file:///path/to/mercurial/repo/\n>> +\n>> +from mercurial import hg, ui\n>> +\n>> +import re\n>> +import sys\n>> +import os\n>> +import json\n>> +\n>> +def die(msg, *args):\n>> + print >> sys.stderr, 'ERROR:', msg % args\n>> + sys.exit(1)\n>\n> While we don't have to code for py3, avoiding '>>' will help us later.\n> (It got removed in py3.) sys.sdterr.write() should be most portable.\n\nAll right.\n\n>> +def gitmode(flags):\n>> + return 'l' in flags and '120000' or 'x' in flags and '100755' or '100644'\n>> +\n>> +def export_file(fc):\n>> + if fc.path() == '.hgtags':\n>\n> Is this always relative? Just wondering, dunno.\n\nAFAIK, why wouldn't it?\n\n>> +def get_repo(path, alias):\n>> + myui = ui.ui()\n>> + myui.setconfig('ui', 'interactive', 'off')\n>> + repo = hg.repository(myui, path)\n>> + return repo\n>\n> Is there a reason to spell this out, e.g.: Why not\n>\n> return hg.repository(myui, path)\n\nNo reason, but I already have patches on top of this.\n\n>> +def hg_branch(b):\n>> + if b == 'master':\n>> + return 'default'\n>> + return b\n>> +\n>> +def git_branch(b):\n>> + if b == 'default':\n>> + return 'master'\n>> + return b\n>> +\n>> +def export_tag(repo, tag):\n>> + global prefix\n>> + print \"reset %s/tags/%s\" % (prefix, tag)\n>> + print \"from :%s\" % (repo[tag].rev() + 1)\n>> + print\n>> +\n>> +def export_branch(repo, branch):\n>> + global prefix, marks, cache, branches\n>> +\n>> + heads = branches[hg_branch(branch)]\n>> +\n>> + # verify there's only one head\n>> + if (len(heads) > 1):\n>> + die(\"Branch '%s' has more than one head\" % hg_branch(branch))\n>\n> We have to deal with this at some point... Do you support \"hg\n> bookmarks\"? They'd be an option, or we implement better detached head\n> handling in git...\n\nI already updated this, I converted it to a warning and just picked a\nrandom head.\n\nAdding support for bookmarks should be easy, it's just a matter of\ndeciding how to differentiate branches from bookmarks. Perhaps\n'refs/heads/bookmarks/foo'.\n\n>> + revs = [rev for rev in revs if not cache.get(rev, False)]\n>> +\n>> + for rev in revs:\n>\n> Those lines set up the list just so that you iterate over it later.\n> Please don't do this (unless you know that revs is very small).\n>\n> for rev in revs:\n> if cache.get(rev, False):\n> continue\n>\n> is more performant. You can reduce this further to\n>\n> count=0\n> for rev in repo.revs('%u:%u' % (tip, head)):\n> if cache.get(rev, False):\n> continue\n>\n> which is even more performant generally, and especially if repo.revs()\n> returns an iterator rather than a list.\n>\n> [Yes, you could use lambda+filter, but let's not get religious. The\n> above is simple and pythonic.]\n\nAnd we wouldn't be able to calculate the progress: len(revs).\n\n>> +\n>> + c = repo[rev]\n>> + (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(c.node())\n>\n> Same here, you introduce c just for the next line (unless I'm mistaken).\n\nThe second line is already too big.\n\n>> + print \"from :%s\" % (parents[0] + 1)\n>\n> What is this +1 offset that is appearing here and there?\n\nMarks start from 1, no? If not, this can be removed.\n\n>> +def do_list(repo, args):\n>> + global branches\n>> +\n>> + head = repo.dirstate.branch()\n>> + for branch in repo.branchmap():\n>> + heads = repo.branchheads(branch)\n>> + if len(heads):\n>> + branches[branch] = heads\n>\n> I'm a bit confused here. repo.brancheads() is a list, no? Is this the\n> single head case only? I'd expect [0] of that, but you seem to be\n> getting branch names (strings).\n\nNo, heads is a list of heads, as nodes.\n\n> Also, if len(heads) == 0 then branches[branch] is undefined or stale. No?\n\nYes, but we don't care about those branches; they are closed.\n\n> Overall, this looks like plain scripting in python rather than anything\n> oo'ish, but that's okay and probably dictated by the remote helper\n> interface and/or the existing exporter.\n\nIf there was an advantage of creating new classes, I would be all for\nit, but I don't see any.\n\n> I'm all for an improvement in that area, but still feel we'd need a\n> combined effort rather than yet another start.\n\nMe too, but as far a sverre's remote-hg code goes, there isn't even a\nbranch to work from.\n\nCheers.\n\n-- \nFelipe Contreras\n"},{"id":"201641","messageId":"CAMP44s0jgxqb8EyjCcEdzaDR6y8gqydW7X-C268NahjaXMrh6A@mail.gmail.com","threadId":"31849","inReplyTo":"alpine.DEB.1.00.1210181031320.3049@bonsai2","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-21T18:03:27Z","receivedAt":"2012-10-21T18:03:27Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 10:47 AM, Johannes Schindelin\n wrote:\n\n> Without push support, remote-hg is useless to me. Without regression tests\n> proving that it is rock solid, I will not use remote-hg.\n\nDone and done. My remote-hg now has support for pushing, all in less\nthan 500 lines of code. It also manages to pass all 14 of the\n\"extensive tests\" of your remote-hg. Anything else?\n\n-- \nFelipe Contreras\n"},{"id":"201649","messageId":"alpine.DEB.1.00.1210212138290.2695@bonsai2","threadId":"31849","inReplyTo":"CAMP44s0jgxqb8EyjCcEdzaDR6y8gqydW7X-C268NahjaXMrh6A@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2012-10-21T20:03:31Z","receivedAt":"2012-10-21T20:03:31Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Felipe,\n\nOn Sun, 21 Oct 2012, Felipe Contreras wrote:\n\n> On Thu, Oct 18, 2012 at 10:47 AM, Johannes Schindelin\n> wrote:\n> \n> > Without push support, remote-hg is useless to me. Without regression\n> > tests proving that it is rock solid, I will not use remote-hg.\n> \n> Done and done. My remote-hg now has support for pushing, all in less\n> than 500 lines of code. It also manages to pass all 14 of the \"extensive\n> tests\" of your remote-hg. Anything else?\n\nWhile I think that a lot of effort was duplicated now, and while I am\nstill interested in less handwaving arguments than \"I find the code\nbloated\", I will compare the performance on both hg and openjdk and if I\ndo not find any issues, have a look at the code, too.\n\nThat will have to wait until I am home in a bit more than a week, though.\n\nCiao,\nJohannes\n\nP.S.: Sverre's remote-hg does not really handle octopus merges. It is\nincomplete. I had a good plan how to complete it (see the msysGit wiki\npage about remote-hg) but lacked the time to implement it (the problem is\nthat hg does not have octopus merges, and we want things to be\nbidirectional).\n"},{"id":"201652","messageId":"CAMP44s3xo6dnNa_-AkhDurvWqp5yC-idoZ_bkcPdPwYsmtWf-A@mail.gmail.com","threadId":"31849","inReplyTo":"alpine.DEB.1.00.1210212138290.2695@bonsai2","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-21T20:31:27Z","receivedAt":"2012-10-21T20:31:27Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Sun, Oct 21, 2012 at 10:03 PM, Johannes Schindelin\n wrote:\n> On Sun, 21 Oct 2012, Felipe Contreras wrote:\n>\n>> On Thu, Oct 18, 2012 at 10:47 AM, Johannes Schindelin\n>> wrote:\n>>\n>> > Without push support, remote-hg is useless to me. Without regression\n>> > tests proving that it is rock solid, I will not use remote-hg.\n>>\n>> Done and done. My remote-hg now has support for pushing, all in less\n>> than 500 lines of code. It also manages to pass all 14 of the \"extensive\n>> tests\" of your remote-hg. Anything else?\n>\n> While I think that a lot of effort was duplicated now, and while I am\n> still interested in less handwaving arguments than \"I find the code\n> bloated\",\n\nThe only way to avoid duplicated effort is to work together, and I've\nyet to see where the remote-hg branch is supposed to be (without any\nmsysgit stuff), so that other people can give it a try, and propose\nchanges.\n\n> P.S.: Sverre's remote-hg does not really handle octopus merges. It is\n> incomplete. I had a good plan how to complete it (see the msysGit wiki\n> page about remote-hg) but lacked the time to implement it (the problem is\n> that hg does not have octopus merges, and we want things to be\n> bidirectional).\n\nYeah, I'm aware mercurial doesn't have those, that's why I didn't\nimplement that, other tools do something similar as you mention in the\nwiki, but the code is rather convoluted.\n\nNote that this doesn't prevent things to be bidirectional, what it\nprevents is using this tool to export git repositories to mercurial,\nnot the other way around. If you do an octopus merge on a repository\nthat you know is going to end in mercurial, that's just asking for\ntrouble, and complaints from the other users of that repo.\n\nAnyway, I don't think that feature is that important, what is more\nimportant is to make sure renames and branches are stored properly. I\nhave tests that check that the output is the the same as hg-git, but\nI'm still not there.\n\nCheers.\n\n-- \nFelipe Contreras\n"},{"id":"201932","messageId":"CAMP44s2H=jTBJbDu-Fzc6Z9NLOmheKty5HFcv4hND=_rDV==Vg@mail.gmail.com","threadId":"31849","inReplyTo":"CAGdFq_hsCjL5uunwg=N-y+FkFt-ZxORtrEcxAAHt-rDfA04cZw@mail.gmail.com","subject":"Re: [PATCH] Add new git-remote-hd helper","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2012-10-26T09:02:18Z","receivedAt":"2012-10-26T09:02:18Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Thu, Oct 18, 2012 at 8:12 AM, Sverre Rabbelier wrote:\n> On Wed, Oct 17, 2012 at 10:18 PM, Felipe Contreras\n> wrote:\n>> Right now I've just added an error when using remote repositories. But\n>> it seems there's no way around it; if we want to have support for\n>> remote repos, we need to make a local clone.\n>\n> My git-remote-hg does the local clone into .git/ using a hash of the\n> url (although you could just as well use urlencode, basically any way\n> to safely use a url as a directory name). Have a look if you want.\n\nI can't find that code.\n\n-- \nFelipe Contreras\n"}]}