{"thread":{"id":"34521","subject":"[PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","startedAt":"2013-07-23T21:40:16Z","lastAt":"2013-07-25T19:30:27Z","messageCount":9,"participants":["Joern Hees","Antoine Pelisse","Jörn Hees","Junio C Hamano","Felipe Contreras"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"224007","messageId":"1374615616-4730-1-git-send-email-dev@joernhees.de","threadId":"34521","inReplyTo":null,"subject":"[PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Joern Hees","fromEmail":"dev@joernhees.de","sentAt":"2013-07-23T21:40:16Z","receivedAt":"2013-07-23T21:40:16Z","isPatch":true,"sender":{"key":"dev@joernhees.de","avatar":"https://gravatar.com/avatar/590cc6f9e7423070747b155451ff7227c749cc3d3621359f2f3eddce0099a8f4?d=mp&s=160"},"body":"6796d49 introduced a bug by making shared_path == \".git/hg' which\nwill most likely exist already, causing a new remote never to be\ncloned and subsequently causing hg.share to fail with error msg:\n\"mercurial.error.RepoError: repository .git/hg not found\"\n\nChanging gitdir to dirname causes shared_path ==\n.git/hg//hg. The call to hg.share with local_path ==\n.git/hg//clone works again.\n\nSigned-off-by: Joern Hees \n---\n contrib/remote-helpers/git-remote-hg | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg\nindex 0194c67..89dd4cc 100755\n--- a/contrib/remote-helpers/git-remote-hg\n+++ b/contrib/remote-helpers/git-remote-hg\n@@ -390,7 +390,7 @@ def get_repo(url, alias):\n if not os.path.exists(dirname):\n os.makedirs(dirname)\n else:\n- shared_path = os.path.join(gitdir, 'hg')\n+ shared_path = os.path.join(dirname, 'hg')\n if not os.path.exists(shared_path):\n try:\n hg.clone(myui, {}, url, shared_path, update=False, pull=True)\n-- \n1.8.3.3\n"},{"id":"224031","messageId":"CALWbr2zRsCk1N5xUUDQeWX6CbvLHYWnxiYpea+etoWvXHNhPEA@mail.gmail.com","threadId":"34521","inReplyTo":"1374615616-4730-1-git-send-email-dev@joernhees.de","subject":"Re: [PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Antoine Pelisse","fromEmail":"apelisse@gmail.com","sentAt":"2013-07-24T08:52:12Z","receivedAt":"2013-07-24T08:52:12Z","isPatch":true,"sender":{"key":"apelisse@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1929644?v=4"},"body":"On Tue, Jul 23, 2013 at 11:40 PM, Joern Hees wrote:\n> 6796d49 introduced a bug by making shared_path == \".git/hg' which\n> will most likely exist already, causing a new remote never to be\n> cloned and subsequently causing hg.share to fail with error msg:\n> \"mercurial.error.RepoError: repository .git/hg not found\"\n\nIndeed, no clone is performed if the .git/hg dir already exists.\nI think it assumes that it's already done.\nThat will certainly lead to the failure you are reporting.\n\nAlso, the directory can be created to store marks for a local repository.\nremote-hg won't require nor do a local clone in .git/hg for local repositories.\n\nIt should also be noted that once .git/hg is not empty, it will no\nlonger be possible to create a mercurial repository in there (it will\ndie with \"destination '.git/hg' is not empty\")\n\nI think the best way would be to create the shared repository in\n.git/hg/$share, with $share being a path that can't be a remote name\n(so that it doesn't conflict with remote directories),\nand then apply the following patch (copied in gmail)\n\ndiff --git a/contrib/remote-helpers/git-remote-hg\nb/contrib/remote-helpers/git-remote-hg\nindex 0194c67..21c8091 100755\n--- a/contrib/remote-helpers/git-remote-hg\n+++ b/contrib/remote-helpers/git-remote-hg\n@@ -390,7 +390,7 @@ def get_repo(url, alias):\n if not os.path.exists(dirname):\n os.makedirs(dirname)\n else:\n- shared_path = os.path.join(gitdir, 'hg')\n+ shared_path = os.path.join(gitdir, 'hg', $share)\n if not os.path.exists(shared_path):\n try:\n hg.clone(myui, {}, url, shared_path, update=False, pull=True)\n\nThat way, the share can be created even if .git/hg already exists\n(because of a previous import, before the shared machinery existed, or\nbecause you already have a local remote).\n\n> Changing gitdir to dirname causes shared_path ==\n> .git/hg//hg. The call to hg.share with local_path ==\n> .git/hg//clone works again.\n\nI think that will be a problem, because then the shared_path will no\nlonger be shared, will it ?\n"},{"id":"224032","messageId":"F0461ED2-7B5F-4657-B0D4-3CBBE15FDD48@joernhees.de","threadId":"34521","inReplyTo":"CALWbr2zRsCk1N5xUUDQeWX6CbvLHYWnxiYpea+etoWvXHNhPEA@mail.gmail.com","subject":"Re: [PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Jörn Hees","fromEmail":"dev@joernhees.de","sentAt":"2013-07-24T09:59:01Z","receivedAt":"2013-07-24T09:59:01Z","isPatch":true,"sender":{"key":"dev@joernhees.de","avatar":"https://gravatar.com/avatar/590cc6f9e7423070747b155451ff7227c749cc3d3621359f2f3eddce0099a8f4?d=mp&s=160"},"body":"Hi,\n\nOn 24.07.2013, at 10:52, Antoine Pelisse wrote:\n> I think the best way would be to create the shared repository in\n> .git/hg/$share, with $share being a path that can't be a remote name\n> (so that it doesn't conflict with remote directories),\n> and then apply the following patch (copied in gmail)\n\nMaybe \".git/hg/.share\"?\n\n\n> diff --git a/contrib/remote-helpers/git-remote-hg\n> b/contrib/remote-helpers/git-remote-hg\n> index 0194c67..21c8091 100755\n> --- a/contrib/remote-helpers/git-remote-hg\n> +++ b/contrib/remote-helpers/git-remote-hg\n> @@ -390,7 +390,7 @@ def get_repo(url, alias):\n> if not os.path.exists(dirname):\n> os.makedirs(dirname)\n> else:\n> - shared_path = os.path.join(gitdir, 'hg')\n> + shared_path = os.path.join(gitdir, 'hg', $share)\n> if not os.path.exists(shared_path):\n> try:\n> hg.clone(myui, {}, url, shared_path, update=False, pull=True)\n> \n> That way, the share can be created even if .git/hg already exists\n> (because of a previous import, before the shared machinery existed, or\n> because you already have a local remote).\n\nI like the idea of having independent remotes (fetching one, doesn't update another). http://mercurial.selenic.com/wiki/ShareExtension warns about this, and i wasn't sure it wouldn't cause intricate bugs. This is why I opted for the explicit cloning, no shared history for several remotes.\n\nI'd really like some feedback on this one as he probably knows the hg internals well enough that he can make a more educated guess on this than I can: when you import several hg remotes and fetch them / push to one, wouldn't such a shared repo cause problems?\nIf unsure i still opt for my version as it keeps things isolated at the cost of some optimization.\n\n\n>> Changing gitdir to dirname causes shared_path ==\n>> .git/hg//hg. The call to hg.share with local_path ==\n>> .git/hg//clone works again.\n> \n> I think that will be a problem, because then the shared_path will no\n> longer be shared, will it ?\n\nYupp, the shared_paths won't be shared, so it's not as optimal as possible, but it will work at least ;)\n\nCheers,\nJörn\n"},{"id":"224039","messageId":"CALWbr2wDqo29kRJ2eHsozRCN_fT3tumYz23pQa5P-9dm27OL6A@mail.gmail.com","threadId":"34521","inReplyTo":"F0461ED2-7B5F-4657-B0D4-3CBBE15FDD48@joernhees.de","subject":"Re: [PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Antoine Pelisse","fromEmail":"apelisse@gmail.com","sentAt":"2013-07-24T13:14:03Z","receivedAt":"2013-07-24T13:14:03Z","isPatch":true,"sender":{"key":"apelisse@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1929644?v=4"},"body":"On Wed, Jul 24, 2013 at 11:59 AM, Jörn Hees wrote:\n> On 24.07.2013, at 10:52, Antoine Pelisse wrote:\n>> I think the best way would be to create the shared repository in\n>> .git/hg/$share, with $share being a path that can't be a remote name\n>> (so that it doesn't conflict with remote directories),\n>\n> Maybe \".git/hg/.share\"?\n\nAccording to Documentation/git-check-ref-format.txt, I'm not sure if\nwe should start with a dot, or end with it.\n\n>> That way, the share can be created even if .git/hg already exists\n>> (because of a previous import, before the shared machinery existed, or\n>> because you already have a local remote).\n>\n> I like the idea of having independent remotes (fetching one, doesn't update another). http://mercurial.selenic.com/wiki/ShareExtension warns about this, and i wasn't sure it wouldn't cause intricate bugs. > This is why I opted for the explicit cloning, no shared history for several remotes.\n\nI think the goal of using sharing here is that Mercurial and Git may\nuse different schemes to handle branches. Mercurial may lead you to\nhave separate repositories for each branch (They seem to do it for its\nown development [1]). All these branches actually share most of the\nsame history, and are fully related, and we usually handle this\nsituation in Git with one repository with multiple branches.\nUsing \"hg share\", we allow a smooth transition from Mercurial model to\nGit model by merging all Mercurial repositories into one, and then map\nthis single repository to the Git repository.\nIOW, the goal is to have only one copy of each \"hg object\" that are\nshared amongst many \"remotes\" (and potentially import them only once,\nthough I don't think it currently works for me).\n\n>>> Changing gitdir to dirname causes shared_path ==\n>>> .git/hg//hg. The call to hg.share with local_path ==\n>>> .git/hg//clone works again.\n>>\n>> I think that will be a problem, because then the shared_path will no\n>> longer be shared, will it ?\n>\n> Yupp, the shared_paths won't be shared, so it's not as optimal as possible, but it will work at least ;)\n\nIf we decided to remove the sharing idea, I think we should revert\nFelipe's commit rather than leave the shared_path variable, and call\nhg.share() on repository we don't even mean to share. That would be\nvery confusing.\n\n[1]: http://mercurial.selenic.com/wiki/DeveloperRepos\n"},{"id":"224040","messageId":"11280A0C-C7C7-4B26-95FE-5C91EF191276@joernhees.de","threadId":"34521","inReplyTo":"CALWbr2wDqo29kRJ2eHsozRCN_fT3tumYz23pQa5P-9dm27OL6A@mail.gmail.com","subject":"Re: [PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Jörn Hees","fromEmail":"dev@joernhees.de","sentAt":"2013-07-24T14:21:33Z","receivedAt":"2013-07-24T14:21:33Z","isPatch":true,"sender":{"key":"dev@joernhees.de","avatar":"https://gravatar.com/avatar/590cc6f9e7423070747b155451ff7227c749cc3d3621359f2f3eddce0099a8f4?d=mp&s=160"},"body":"\nOn 24.07.2013, at 15:14, Antoine Pelisse wrote:\n\n> On Wed, Jul 24, 2013 at 11:59 AM, Jörn Hees wrote:\n>> On 24.07.2013, at 10:52, Antoine Pelisse wrote:\n>>> I think the best way would be to create the shared repository in\n>>> .git/hg/$share, with $share being a path that can't be a remote name\n>>> (so that it doesn't conflict with remote directories),\n>> \n>> Maybe \".git/hg/.share\"?\n> \n> According to Documentation/git-check-ref-format.txt, I'm not sure if\n> we should start with a dot, or end with it.\n\nI favor starting with a dot as it's nothing the user should fiddle with ;)\n\n>>> That way, the share can be created even if .git/hg already exists\n>>> (because of a previous import, before the shared machinery existed, or\n>>> because you already have a local remote).\n>> \n>> I like the idea of having independent remotes (fetching one, doesn't update another). http://mercurial.selenic.com/wiki/ShareExtension warns about this, and i wasn't sure it wouldn't cause intricate bugs. > This is why I opted for the explicit cloning, no shared history for several remotes.\n> \n> I think the goal of using sharing here is that Mercurial and Git may\n> use different schemes to handle branches. Mercurial may lead you to\n> have separate repositories for each branch (They seem to do it for its\n> own development [1]). All these branches actually share most of the\n> same history, and are fully related, and we usually handle this\n> situation in Git with one repository with multiple branches.\n> Using \"hg share\", we allow a smooth transition from Mercurial model to\n> Git model by merging all Mercurial repositories into one, and then map\n> this single repository to the Git repository.\n> IOW, the goal is to have only one copy of each \"hg object\" that are\n> shared amongst many \"remotes\" (and potentially import them only once,\n> though I don't think it currently works for me).\n\nAlright, i just tested it out by sharing several repos and pushing to one of them, then fetching all again. Behavior seems as expected, so the remotes and their branches shown are isolated correctly.\nPlus the initial fetching is quite a lot faster, less disk space used, etc…\nSo i think this is the way to go, thanks for the nudge.\n\n\n>>>> Changing gitdir to dirname causes shared_path ==\n>>>> .git/hg//hg. The call to hg.share with local_path ==\n>>>> .git/hg//clone works again.\n>>> \n>>> I think that will be a problem, because then the shared_path will no\n>>> longer be shared, will it ?\n>> \n>> Yupp, the shared_paths won't be shared, so it's not as optimal as possible, but it will work at least ;)\n> \n> If we decided to remove the sharing idea, I think we should revert\n> Felipe's commit rather than leave the shared_path variable, and call\n> hg.share() on repository we don't even mean to share. That would be\n> very confusing.\n\n+1\n\nI'll prepare a v2 of the patch.\n\nCheers,\nJörn\n"},{"id":"224045","messageId":"7v7ggg6l2o.fsf@alter.siamese.dyndns.org","threadId":"34521","inReplyTo":"CALWbr2wDqo29kRJ2eHsozRCN_fT3tumYz23pQa5P-9dm27OL6A@mail.gmail.com","subject":"Re: [PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-07-24T15:20:47Z","receivedAt":"2013-07-24T15:20:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Antoine Pelisse writes:\n\n> On Wed, Jul 24, 2013 at 11:59 AM, Jörn Hees wrote:\n>> On 24.07.2013, at 10:52, Antoine Pelisse wrote:\n>>> I think the best way would be to create the shared repository in\n>>> .git/hg/$share, with $share being a path that can't be a remote name\n>>> (so that it doesn't conflict with remote directories),\n>>\n>> Maybe \".git/hg/.share\"?\n>\n> According to Documentation/git-check-ref-format.txt, I'm not sure if\n> we should start with a dot, or end with it.\n\nWhat are in these directories under .git/hg? Surely they cannot be\nrefs in Git's sense, as that hierarchy is not known to anything and\nwill not be protected from \"git gc\".\n\nPuzzled...\n\n\tGoes and looks...\n\nOK, the tracking branches for these are created under refs/hg/*\nusing the same name.\n\nA refname shouldn't begin or end with a dot, because the range\n\n\tmaster...share\n\nwill become ambiguous if you allowed \".share\" as a refname\nshorthand. It could mean either one of these:\n\n\tmaster..refs/heads/.share\n master...refs/heads/share\n\nThe same for the trailing dot \"share.\"; the range \"share...master\"\nbecomes ambiguous.\n"},{"id":"224046","messageId":"665576E7-6083-4535-9CE0-773236A0E9ED@joernhees.de","threadId":"34521","inReplyTo":"7v7ggg6l2o.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Jörn Hees","fromEmail":"dev@joernhees.de","sentAt":"2013-07-24T15:28:35Z","receivedAt":"2013-07-24T15:28:35Z","isPatch":true,"sender":{"key":"dev@joernhees.de","avatar":"https://gravatar.com/avatar/590cc6f9e7423070747b155451ff7227c749cc3d3621359f2f3eddce0099a8f4?d=mp&s=160"},"body":"\nOn 24 Jul 2013, at 17:20, Junio C Hamano wrote:\n\n> Antoine Pelisse writes:\n> \n>> On Wed, Jul 24, 2013 at 11:59 AM, Jörn Hees wrote:\n>>> On 24.07.2013, at 10:52, Antoine Pelisse wrote:\n>>>> I think the best way would be to create the shared repository in\n>>>> .git/hg/$share, with $share being a path that can't be a remote name\n>>>> (so that it doesn't conflict with remote directories),\n>>> \n>>> Maybe \".git/hg/.share\"?\n>> \n>> According to Documentation/git-check-ref-format.txt, I'm not sure if\n>> we should start with a dot, or end with it.\n> \n> What are in these directories under .git/hg? Surely they cannot be\n> refs in Git's sense, as that hierarchy is not known to anything and\n> will not be protected from \"git gc\".\n> \n> Puzzled...\n> \n> \tGoes and looks...\n> \n> OK, the tracking branches for these are created under refs/hg/*\n> using the same name.\n> \n> A refname shouldn't begin or end with a dot, because the range\n> \n> \tmaster...share\n> \n> will become ambiguous if you allowed \".share\" as a refname\n> shorthand. It could mean either one of these:\n> \n> \tmaster..refs/heads/.share\n> master...refs/heads/share\n> \n> The same for the trailing dot \"share.\"; the range \"share...master\"\n> becomes ambiguous.\n\n\nI think there is a slight misunderstanding here:\n.git/hg/ will be the actual directory for a hg:: remote, which will then use mercurial internal magic to refer to the shared repo .git/hg/.shared in case the remote is not somewhere on the local filesystem, otherwise that path is used.\n\nWhat will appear in the refs is something like: hg//{branches,bookmarks}/{master,default,…}.\nSo the .shared will correctly never appear in a git ref, which is what we want. It can also not clash with a remote as \".shared\" is not a valid name… also what we want ;)\n\nCheers,\nJörn\n"},{"id":"224062","messageId":"7vhafj606p.fsf@alter.siamese.dyndns.org","threadId":"34521","inReplyTo":"665576E7-6083-4535-9CE0-773236A0E9ED@joernhees.de","subject":"Re: [PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-07-24T22:51:58Z","receivedAt":"2013-07-24T22:51:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jörn Hees writes:\n\n> On 24 Jul 2013, at 17:20, Junio C Hamano wrote:\n>\n>> \tGoes and looks...\n>> \n>> OK, the tracking branches for these are created under refs/hg/*\n>> using the same name.\n>> ...\n>> A refname shouldn't begin or end with a dot, because the range\n>> ... ellided a correct description of the reason behind rule, which is\n>> ... irrelevant to the topic\n>\n> I think there is a slight misunderstanding here:\n>\n> .git/hg/ will be the actual directory for a hg::\n> remote, which will then use mercurial internal magic to refer to\n> the shared repo .git/hg/.shared in case the remote is not\n> somewhere on the local filesystem, otherwise that path is used.\n\nYes, it is not a \"slight\" but a \"huge\" misunderstanding.\n\nI saw the caller of get_repo(url, alias) doing a\n\n 'refs/hg/%s' % alias\n\nimmediately after the call, and somehow assumed that the proposal\nwould result in stuffing .shared to the \"alias\", which is not the\ncase. The hg/$name directories may be used to store Hg repositories\nfor 'alias' that is passed by the caller, but hg/.shared thing you\nguys were discussing is internal to get_repo() implementation and\nthere won't be refs/hg/.shared created because of this change.\n"},{"id":"224087","messageId":"CAMP44s1H1G-G0SvvegLsTfFizprzRkw8TfqcJi8hCr-6+8WnKQ@mail.gmail.com","threadId":"34521","inReplyTo":"CALWbr2wDqo29kRJ2eHsozRCN_fT3tumYz23pQa5P-9dm27OL6A@mail.gmail.com","subject":"Re: [PATCH] [SIGNED-OFF] remotes-hg: bugfix for fetching non local remotes","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2013-07-25T19:30:27Z","receivedAt":"2013-07-25T19:30:27Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Wed, Jul 24, 2013 at 8:14 AM, Antoine Pelisse wrote:\n\n> IOW, the goal is to have only one copy of each \"hg object\" that are\n> shared amongst many \"remotes\" (and potentially import them only once,\n> though I don't think it currently works for me).\n\nThat's right. I had code to import only once, but it didn't work\ncorrectly; we would need a way to have shared fast-import/export\nmarks, and I don't think it's even possible from Mercurial's API to\nfigure out which objects are shared and which specific, so I gave up\non that. Sharing the repository is the only thing we can do safely and\nsanely.\n\n-- \nFelipe Contreras\n"}]}