{"thread":{"id":"29423","subject":"[RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper \"import\" command","startedAt":"2012-01-22T05:46:57Z","lastAt":"2012-02-03T09:43:35Z","messageCount":4,"participants":["Jonathan Nieder","Jelmer Vernooij","Gabriel Filion"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"182919","messageId":"20120122054657.GA25103@burratino","threadId":"29423","inReplyTo":null,"subject":"[RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper \"import\" command","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-01-22T05:46:57Z","receivedAt":"2012-01-22T05:46:57Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Git 1.7.7 (commit 9504bc9d, \"transport-helper: change import\nsemantics\", 2011-07-16) incompatibly changed the interface of the\n\"import\" capability.\n\nBefore, git would always send a single import command, which the\nremote helper would respond to with a fast-import stream, terminated\nby end of file, meaning there was no way to fetch multiple refs in one\nconnection.  Nowadays, git instead sends a sequence of import lines:\n\n\timport refs/heads/foo\n\timport refs/heads/bar\n\nterminated by a blank line.  The helper is to respond with a\nfast-import stream terminated by the \"done\" command and process\nfurther commands until another blank line indicates the end of the\ncommand stream.\n---\nHi Simon and Gabriel,\n\nHere's a rough patch against git://github.com/lelutin/git-remote-bzr.git\nmaster.\n\nWithout this patch, whenever I try to use \"git clone bzr::<something>\",\nafter doing all the work it removes the resulting repo and exits with\nstatus 141 (SIGPIPE).  Maybe the transport-helper should mask SIGPIPE\nwhen writing the final newline to avoid that.\n\nI'd have prefered to write a patch for remote-bzr that works with\nolder versions of git fast-import, too, but it wasn't obvious how.\nHints welcome.\n\nBTW, would you mind if I sent a patch to include git-remote-bzr in\ngit.git under contrib/?\n\nThanks for git remote-bzr!  I'd be happy for any thoughts you have.\n\nCiao,\nJonathan\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/176002/focus=176606\n\n README.rst     |    2 +-\n git-remote-bzr |   33 ++++++++++++++++++++++++++++++---\n 2 files changed, 31 insertions(+), 4 deletions(-)\n\ndiff --git a/README.rst b/README.rst\nindex 3eb3e476..f4dbbeb2 100644\n--- a/README.rst\n+++ b/README.rst\n@@ -34,7 +34,7 @@ Relevant bug reports\n Requirements\n ------------\n \n-- git 1.6.6 or later\n+- git 1.7.7 or later\n - python 2.5 +\n - bzr 2.x\n - bzr-fastimport\ndiff --git a/git-remote-bzr b/git-remote-bzr\nindex 1e3a05f9..501fffe3 100755\n--- a/git-remote-bzr\n+++ b/git-remote-bzr\n@@ -49,7 +49,7 @@ def do_list(repo, args):\n     print  # end list\n \n \n-def do_import(repo, args):\n+def import_one_ref(repo, args):\n     \"\"\"Import a fast-import stream that is exported from Bazaar.\"\"\"\n     if len(args) != 1:\n         die(\"Import needs exactly one ref\")\n@@ -65,6 +65,23 @@ def do_import(repo, args):\n     if bzrp.wait():\n         die(\"'bzr fast-export' returned unexpectedly with code %d\",\n             bzrp.returncode)\n+    print \"done\"\n+\n+\n+def do_import(repo,args):\n+    import_one_ref(repo, args)\n+\n+    cmdline = True\n+    while cmdline:\n+        cmdline = next_command()\n+        if not cmdline:\n+            # Return to main processing loop\n+            return True\n+        cmd = cmdline.pop(0)\n+        if cmd != \"import\":\n+            warn(\"Unexpected command %s during import\" % cmd)\n+            return False\n+        import_one_ref(repo, cmdline)\n \n \n def do_push(repo, args):\n@@ -123,8 +140,8 @@ def sanitize(value):\n     return value\n \n \n-def read_one_line(repo):\n-    \"\"\"Read and process one command.\"\"\"\n+def next_command():\n+    \"\"\"Read one command.\"\"\"\n     line = sys.stdin.readline()\n \n     cmdline = line\n@@ -138,6 +155,16 @@ def read_one_line(repo):\n         # Blank line means we're about to quit\n         return False\n \n+    return cmdline\n+\n+\n+def read_one_line(repo):\n+    \"\"\"Read and process one command.\"\"\"\n+    cmdline = next_command()\n+\n+    if not cmdline:\n+        return False\n+\n     cmd = cmdline.pop(0)\n     debug(\"Got command '%s' with args '%s'\", cmd, ' '.join(cmdline))\n \n-- \n1.7.9.rc2\n"},{"id":"182948","messageId":"4F1C9D4C.6090603@samba.org","threadId":"29423","inReplyTo":"20120122054657.GA25103@burratino","subject":"Re: [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper \"import\" command","fromName":"Jelmer Vernooij","fromEmail":"jelmer@samba.org","sentAt":"2012-01-22T23:35:40Z","receivedAt":"2012-01-22T23:35:40Z","isPatch":true,"sender":{"key":"jelmer@samba.org","avatar":"https://avatars.githubusercontent.com/u/49032?v=4"},"body":"On 01/22/2012 06:46 AM, Jonathan Nieder wrote:\n> Git 1.7.7 (commit 9504bc9d, \"transport-helper: change import\n> semantics\", 2011-07-16) incompatibly changed the interface of the\n> \"import\" capability.\n>\n> Before, git would always send a single import command, which the\n> remote helper would respond to with a fast-import stream, terminated\n> by end of file, meaning there was no way to fetch multiple refs in one\n> connection.  Nowadays, git instead sends a sequence of import lines:\n>\n> \timport refs/heads/foo\n> \timport refs/heads/bar\n>\n> terminated by a blank line.  The helper is to respond with a\n> fast-import stream terminated by the \"done\" command and process\n> further commands until another blank line indicates the end of the\n> command stream.\n> ---\n> Hi Simon and Gabriel,\n>\n> Here's a rough patch against git://github.com/lelutin/git-remote-bzr.git\n> master.\n>\n> Without this patch, whenever I try to use \"git clone bzr::<something>\",\n> after doing all the work it removes the resulting repo and exits with\n> status 141 (SIGPIPE).  Maybe the transport-helper should mask SIGPIPE\n> when writing the final newline to avoid that.\n>\n> I'd have prefered to write a patch for remote-bzr that works with\n> older versions of git fast-import, too, but it wasn't obvious how.\n> Hints welcome.\n>\n> BTW, would you mind if I sent a patch to include git-remote-bzr in\n> git.git under contrib/?\nPlease note that the bzr-git package, which provides git integration for \nbzr and vice versa, also includes a 'git-remote-bzr' command. Apart from \nthe 'import' command, it includes experimental implementations of \n'fetch' and push as well.\n\nIt would be nice to consolidate the efforts, or at the very least \nprevent name clashes.\n\nCheers,\n\nJelmer\n"},{"id":"182949","messageId":"20120123001228.GA1209@burratino","threadId":"29423","inReplyTo":"4F1C9D4C.6090603@samba.org","subject":"Re: [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper \"import\" command","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-01-23T00:12:28Z","receivedAt":"2012-01-23T00:12:28Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jelmer Vernooij wrote:\n> On 01/22/2012 06:46 AM, Jonathan Nieder wrote:\n\n>> BTW, would you mind if I sent a patch to include git-remote-bzr in\n>> git.git under contrib/?\n>\n> Please note that the bzr-git package, which provides git integration\n> for bzr and vice versa, also includes a 'git-remote-bzr' command.\n\nThat's good to hear.  Then there should be no need for git.git to have\nits own helper.\n\nUnfortunately, when I try to clone any repo, at the last step I get\n\n Traceback (most recent call last):map 6/7\n  File \"/usr/lib/git-core/git-remote-bzr\", line 220, in <module>\n    commands[argv[0]](argv, shortname, remote_dir)\n  File \"/usr/lib/git-core/git-remote-bzr\", line 89, in cmd_list\n    for ref, git_sha1 in refs.as_dict().iteritems():\n  File \"/usr/lib/python2.7/dist-packages/dulwich/repo.py\", line 196, in as_dict\n    ret[key] = self[(\"%s/%s\" % (base, key)).strip(\"/\")]\n  File \"/usr/lib/python2.7/dist-packages/dulwich/repo.py\", line 267, in __getitem__\n    _, sha = self._follow(name)\n  File \"/usr/lib/python2.7/dist-packages/dulwich/repo.py\", line 249, in _follow\n    contents = self.read_ref(refname)\n  File \"/usr/lib/python2.7/dist-packages/dulwich/repo.py\", line 225, in read_ref\n    contents = self.read_loose_ref(refname)\n  File \"/usr/lib/python2.7/dist-packages/bzrlib/plugins/git/refs.py\", line 129, in read_loose_ref\n    tag_name = ref_to_tag_name(ref)\n  File \"/usr/lib/python2.7/dist-packages/bzrlib/plugins/git/refs.py\", line 89, in ref_to_tag_name\n    raise ValueError(\"unable to map ref %s back to tag name\" % ref)\n ValueError: unable to map ref refs/heads back to tag name\n\nWill file a bug.\n"},{"id":"183702","messageId":"4F2BAC47.6090502@gmail.com","threadId":"29423","inReplyTo":"20120122054657.GA25103@burratino","subject":"Re: [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper \"import\" command","fromName":"Gabriel Filion","fromEmail":"lelutin@gmail.com","sentAt":"2012-02-03T09:43:35Z","receivedAt":"2012-02-03T09:43:35Z","isPatch":true,"sender":{"key":"lelutin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/108728?v=4"},"body":"Hello,\n\nOn 12-01-22 12:46 AM, Jonathan Nieder wrote:\n> Hi Simon and Gabriel,\n> \n> Here's a rough patch against git://github.com/lelutin/git-remote-bzr.git\n> master.\n\ngreat! thanks for your help.\n\nI must admit that this project never got to completion and is now\ngetting quite the low priority for my part. I'm no longer working for\nthe company that was using mainly Bazaar as their VCS, and I've been a\nhappy git-only user for some time now.\nSo, I don't have the same incentive to complete the project as I had before.\n\nBut I'd be happy to see this get to a point where it's working fine.\n\n> Without this patch, whenever I try to use \"git clone bzr::<something>\",\n> after doing all the work it removes the resulting repo and exits with\n> status 141 (SIGPIPE).  Maybe the transport-helper should mask SIGPIPE\n> when writing the final newline to avoid that.\n> \n> I'd have prefered to write a patch for remote-bzr that works with\n> older versions of git fast-import, too, but it wasn't obvious how.\n> Hints welcome.\n\nhmm.. I can wait some time to see if some ideas come out around this,\nand commit your patch as-is if there are no comments/reworks.\n\n> BTW, would you mind if I sent a patch to include git-remote-bzr in\n> git.git under contrib/?\n\nabsolutely not, that'd be great actually :)\n\nI didn't do that up to now, though, since I bumped into so much bugs\nthat I couldn't work out -- some very bad performance issues, and\nproblems with handling mark files with bzr-fastimport.\n\n> Thanks for git remote-bzr!  I'd be happy for any thoughts you have.\n\nThe idea behind git-remote-bzr was to be able to interact with Bazaar\nfrom within your git repository, i.e. to expose remote branches that you\ncan pull from and push to using the default git commands, without having\nto learn to use yet another tool since the remote-helper would be\ninterfacing with the tool for you.\n\nI have dived for a short period of time into bzrlib, the python library\nbehind Bazaar, to see how much work it would take to put together a\nsimplified fast-import client.. and .... wechrk!\nIt was a huge maze of version-dependant code (the API undergoes big\nchanges frequently, and backwards compatibility is maintained pretty\nfar) mixed with a 40-foot-deep class hierarchy. So I quickly gave up on\nthat idea..\n\nIIRC, at the time I started work on this, Simon and I used\nbzr-fastimport[1] because we were able to get farther with this tool. I\nhaven't used bzr-git[2] that much, though, so I can't comment too\nextensively on it. But I would guess that it is maintained more\nfrequently than bzr-fastimport is, so it could be a better choice for\nthe backend fast-import client.\n\nI would guess that Jelmer would be happy to help out with interfacing\nwith bzr-git.\n\n-- \nGabriel Filion\n"}]}