threads / patch / 36537

patch, 8 partsremote-helper: fixes

Subject: [PATCH 0/8] remote-helper: fixes

## tl;dr

10 messages between Apr 29, 2014 and Apr 29, 2014. Diffs are folded; open one to read it.

replies: 9people: 2as markdown or json

Felipe Contreras· Apr 29, 2014, 09:04 UTC · lore
A few trivial updates, a few important fixes.
Daniel Liew (1):
  git-remote-hg: use internal clone's hgrc
Felipe Contreras (7):
  remote-hg: fix parsing of custom committer
  remote-hg: update to 'public' phase when pushing
  remote-{hg,bzr}: store marks only on success
  remote-hg: properly detect missing contexts
  t: remote-hg: split into setup test
  remote-hg: make sure we omit multiple heads
  remote-hg: trivial cleanups
 git-remote-bzr.py    |  8 +++---
 git-remote-hg.py     | 72 ++++++++++++++++++++++++++++++++++++++++++----------
 t/t5810-remote-hg.sh | 65 +++++++----------------------------------------
 3 files changed, 70 insertions(+), 75 deletions(-)
-- 
1.9.2+fc1.3.gade8541
Felipe Contreras· Apr 29, 2014, 09:04 UTC · re: Felipe Contreras · lore

[PATCH 1/8] remote-hg: fix parsing of custom committer

Other tools use the 'committer' extra field differently, so let's make the parsing more reliable and don't assume it's in a certain format.

Reported-by: Kevin Cox <kevincox@kevincox.ca>
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Show changes to git-remote-hg.py +6 −2
diff --git a/git-remote-hg.py b/git-remote-hg.py
index 34cda02..c849abb 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -461,8 +461,12 @@ def export_ref(repo, name, kind, head):
 
         author = "%s %d %s" % (fixup_user(user), time, gittz(tz))
         if 'committer' in extra:
-            user, time, tz = extra['committer'].rsplit(' ', 2)
-            committer = "%s %s %s" % (user, time, gittz(int(tz)))
+            try:
+                cuser, ctime, ctz = extra['committer'].rsplit(' ', 2)
+                committer = "%s %s %s" % (cuser, ctime, gittz(int(ctz)))
+            except ValueError:
+                cuser = extra['committer']
+                committer = "%s %d %s" % (fixup_user(cuser), time, gittz(tz))
         else:
             committer = author
 
-- 
1.9.2+fc1.3.gade8541
Felipe Contreras· Apr 29, 2014, 09:04 UTC · re: Felipe Contreras · lore

[PATCH 2/8] remote-hg: update to 'public' phase when pushing

This is what Mercurial does.
Reported-by: Nathan Palmer
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
Show changes to git-remote-hg.py +10 −2
diff --git a/git-remote-hg.py b/git-remote-hg.py
index c849abb..204ceeb 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -995,9 +995,17 @@ def push_unsafe(repo, remote, parsed_refs, p_revs):
     if unbundle:
         if force:
             remoteheads = ['force']
-        return remote.unbundle(cg, remoteheads, 'push')
+        ret = remote.unbundle(cg, remoteheads, 'push')
     else:
-        return remote.addchangegroup(cg, 'push', repo.url())
+        ret = remote.addchangegroup(cg, 'push', repo.url())
+
+    phases = remote.listkeys('phases')
+    if phases:
+        for head in p_revs:
+            # update to public
+            remote.pushkey('phases', hghex(head), '1', '0')
+
+    return ret
 
 def push(repo, remote, parsed_refs, p_revs):
     if hasattr(remote, 'canpush') and not remote.canpush():
-- 
1.9.2+fc1.3.gade8541
Felipe Contreras· Apr 29, 2014, 09:04 UTC · re: Felipe Contreras · lore

[PATCH 3/8] remote-{hg,bzr}: store marks only on success

Commit 2594a79 (remote-hg: fix bad state issue) originally introduced this code in order to avoid synchronization issues while pushing, because `git fast-export` might end up writing the marks before a crash in the remote helper occurs.

However, the problem is in `git fast-export`; the marks should only be written after both have finished successfully.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-bzr.py | 8 +++-----
 git-remote-hg.py  | 8 +++-----
 2 files changed, 6 insertions(+), 10 deletions(-)
Show changes to 2 files +6 −10

git-remote-bzr.py, git-remote-hg.py

diff --git a/git-remote-bzr.py b/git-remote-bzr.py
index 9abb58e..71b138e 100755
--- a/git-remote-bzr.py
+++ b/git-remote-bzr.py
@@ -971,12 +971,10 @@ def main(args):
             die('unhandled command: %s' % line)
         sys.stdout.flush()
 
+    marks.store()
+
 def bye():
-    if not marks:
-        return
-    if not is_tmp:
-        marks.store()
-    else:
+    if is_tmp:
         shutil.rmtree(dirname)
 
 atexit.register(bye)
diff --git a/git-remote-hg.py b/git-remote-hg.py
index 204ceeb..8aca6dd 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -1258,12 +1258,10 @@ def main(args):
             die('unhandled command: %s' % line)
         sys.stdout.flush()
 
+    marks.store()
+
 def bye():
-    if not marks:
-        return
-    if not is_tmp:
-        marks.store()
-    else:
+    if is_tmp:
         shutil.rmtree(dirname)
 
 atexit.register(bye)
-- 
1.9.2+fc1.3.gade8541
Felipe Contreras· Apr 29, 2014, 09:04 UTC · re: Felipe Contreras · lore

[PATCH 4/8] remote-hg: properly detect missing contexts

This can happen when there's a synchronization issue between marks-git and marks-hg; a key is missing in marks-hg, and when we receive a reset command the value of ctx basically comes from None.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 5 +++++
 1 file changed, 5 insertions(+)
Show changes to git-remote-hg.py +5 −0
diff --git a/git-remote-hg.py b/git-remote-hg.py
index 8aca6dd..1972f7f 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -917,6 +917,11 @@ def checkheads_bmark(repo, ref, ctx):
 
     ctx_old = bmarks[bmark]
     ctx_new = ctx
+
+    if not ctx.rev():
+        print "error %s unknown" % ref
+        return False
+
     if not repo.changelog.descendant(ctx_old.rev(), ctx_new.rev()):
         if force_push:
             print "ok %s forced update" % ref
-- 
1.9.2+fc1.3.gade8541
Felipe Contreras· Apr 29, 2014, 09:04 UTC · re: Felipe Contreras · lore

[PATCH 5/8] t: remote-hg: split into setup test

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 t/t5810-remote-hg.sh | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
Show changes to t/t5810-remote-hg.sh +5 −4
diff --git a/t/t5810-remote-hg.sh b/t/t5810-remote-hg.sh
index 594a0a1..ba8b2d8 100755
--- a/t/t5810-remote-hg.sh
+++ b/t/t5810-remote-hg.sh
@@ -105,17 +105,18 @@ setup () {
 
 setup
 
-test_expect_success 'cloning' '
-	test_when_finished "rm -rf gitrepo*" &&
-
+test_expect_success 'setup' '
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo zero >content &&
 	hg add content &&
 	hg commit -m zero
-	) &&
+	)
+'
 
+test_expect_success 'cloning' '
+	test_when_finished "rm -rf gitrepo*" &&
 	git clone "hg::hgrepo" gitrepo &&
 	check gitrepo HEAD zero
 '
-- 
1.9.2+fc1.3.gade8541
Felipe Contreras· Apr 29, 2014, 09:04 UTC · re: Felipe Contreras · lore

[PATCH 6/8] git-remote-hg: use internal clone's hgrc

From: Daniel Liew <delcypher@gmail.com>

Use the hgrc configuration file in the internal mercurial repository in addition to the other system wide hgrc files. This is done by using the 'ui' object from the 'repository' object which will have loaded the repository hgrc file if it exists.

Signed-off-by: Dan Liew <delcypher@gmail.com>
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to git-remote-hg.py +1 −1
diff --git a/git-remote-hg.py b/git-remote-hg.py
index 1972f7f..cd3d79e 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -422,7 +422,7 @@ def get_repo(url, alias):
 
         repo = hg.repository(myui, local_path)
         try:
-            peer = hg.peer(myui, {}, url)
+            peer = hg.peer(repo.ui, {}, url)
         except:
             die('Repository error')
         repo.pull(peer, heads=None, force=True)
-- 
1.9.2+fc1.3.gade8541
Felipe Contreras· Apr 29, 2014, 09:04 UTC · re: Felipe Contreras · lore

[PATCH 7/8] remote-hg: make sure we omit multiple heads

We want to ignore secondary heads, otherwise we will import revisions that won't have any ref pointing to them and might eventually be pruned, which would cause problems with the synchronization of marks.

This can only be expressed properly as '::b - ::a', but that's not efficient, specially in older versions of Mercurial. 'ancestor(a,b)::b - ancestor(a,b)::a' might be better, but it's not particularly pretty. Mercurial v3.0 will have a new 'only(b, a)' that does the same, but that hasn't even been released yet. Either way all of these require repo.revs() which is only available after Mercurial v2.1.

Also, we would need special considerations for when there's no base revision (importing from root).

It's much better to implement our own function to get a range of revisions.

The new gitrange() is inspired by Mercurial's revset.missingancestors().
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)
Show changes to git-remote-hg.py +32 −3
diff --git a/git-remote-hg.py b/git-remote-hg.py
index cd3d79e..402b92f 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -437,16 +437,45 @@ def rev_to_mark(rev):
 def mark_to_rev(mark):
     return marks.to_rev(mark)
 
+# Get a range of revisions in the form of a..b (git committish)
+def gitrange(repo, a, b):
+    positive = []
+    pending = set([int(b)])
+    negative = set([int(a)])
+    for cur in xrange(b, -1, -1):
+        if not pending:
+            break
+
+        parents = [p for p in repo.changelog.parentrevs(cur) if p >= 0]
+
+        if cur in pending:
+            positive.append(cur)
+            pending.remove(cur)
+            for p in parents:
+                if not p in negative:
+                    pending.add(p)
+        elif cur in negative:
+            negative.remove(cur)
+            for p in parents:
+                if not p in pending:
+                    negative.add(p)
+                else:
+                    pending.discard(p)
+
+    positive.reverse()
+    return positive
+
 def export_ref(repo, name, kind, head):
     ename = '%s/%s' % (kind, name)
     try:
         tip = marks.get_tip(ename)
-        tip = repo[tip].rev()
+        tip = repo[tip]
     except:
-        tip = 0
+        tip = repo[-1]
 
-    revs = xrange(tip, head.rev() + 1)
+    revs = gitrange(repo, tip, head)
     total = len(revs)
+    tip = tip.rev()
 
     for rev in revs:
 
-- 
1.9.2+fc1.3.gade8541
Felipe Contreras· Apr 29, 2014, 09:04 UTC · re: Felipe Contreras · lore

[PATCH 8/8] remote-hg: trivial cleanups

Cleanup 51be46e (remote-hg: do not fail on invalid bookmarks).

Having a 40-characters string is not ideal, and having three tests for basically the same relatively rare situation is overkill.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py     |  2 +-
 t/t5810-remote-hg.sh | 56 ++++------------------------------------------------
 2 files changed, 5 insertions(+), 53 deletions(-)
Show changes to 2 files +5 −53

git-remote-hg.py, t/t5810-remote-hg.sh

diff --git a/git-remote-hg.py b/git-remote-hg.py
index 402b92f..74f2a2e 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -677,7 +677,7 @@ def do_list(parser):
             print "? refs/heads/branches/%s" % gitref(branch)
 
     for bmark in bmarks:
-        if  bmarks[bmark].hex() == '0000000000000000000000000000000000000000':
+        if  bmarks[bmark].hex() == '0' * 40:
             warn("Ignoring invalid bookmark '%s'", bmark)
         else:
             print "? refs/heads/%s" % gitref(bmark)
diff --git a/t/t5810-remote-hg.sh b/t/t5810-remote-hg.sh
index ba8b2d8..9946f57 100755
--- a/t/t5810-remote-hg.sh
+++ b/t/t5810-remote-hg.sh
@@ -772,7 +772,7 @@ test_expect_success 'remote double failed push' '
 	)
 '
 
-test_expect_success 'clone remote with master null bookmark, then push to the bookmark' '
+test_expect_success 'clone remote with null bookmark, then push' '
 	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	hg init hgrepo &&
@@ -781,67 +781,19 @@ test_expect_success 'clone remote with master null bookmark, then push to the bo
 		echo a >a &&
 		hg add a &&
 		hg commit -m a &&
-		hg bookmark -r null master
+		hg bookmark -r null bookmark
 	) &&
 
 	git clone "hg::hgrepo" gitrepo &&
 	check gitrepo HEAD a &&
 	(
 		cd gitrepo &&
-		git checkout --quiet -b master &&
-		echo b >b &&
-		git add b &&
-		git commit -m b &&
-		git push origin master
-	)
-'
-
-test_expect_success 'clone remote with default null bookmark, then push to the bookmark' '
-	test_when_finished "rm -rf gitrepo* hgrepo*" &&
-
-	hg init hgrepo &&
-	(
-		cd hgrepo &&
-		echo a >a &&
-		hg add a &&
-		hg commit -m a &&
-		hg bookmark -r null -f default
-	) &&
-
-	git clone "hg::hgrepo" gitrepo &&
-	check gitrepo HEAD a &&
-	(
-		cd gitrepo &&
-		git checkout --quiet -b default &&
-		echo b >b &&
-		git add b &&
-		git commit -m b &&
-		git push origin default
-	)
-'
-
-test_expect_success 'clone remote with generic null bookmark, then push to the bookmark' '
-	test_when_finished "rm -rf gitrepo* hgrepo*" &&
-
-	hg init hgrepo &&
-	(
-		cd hgrepo &&
-		echo a >a &&
-		hg add a &&
-		hg commit -m a &&
-		hg bookmark -r null bmark
-	) &&
-
-	git clone "hg::hgrepo" gitrepo &&
-	check gitrepo HEAD a &&
-	(
-		cd gitrepo &&
-		git checkout --quiet -b bmark &&
+		git checkout --quiet -b bookmark &&
 		git remote -v &&
 		echo b >b &&
 		git add b &&
 		git commit -m b &&
-		git push origin bmark
+		git push origin bookmark
 	)
 '
 
-- 
1.9.2+fc1.3.gade8541
Philip Oakley· Apr 29, 2014, 22:06 UTC · re: Felipe Contreras · lore

Re: [PATCH 8/8] remote-hg: trivial cleanups

From: "Felipe Contreras" <felipe.contreras@gmail.com>
Subject: [PATCH 8/8] remote-hg: trivial cleanups

It's useful, as a reader of the shortlog and email message subject lines, to know what sort of triviality is being tidied. Usually they are 'spelling' or 'variable naming' or some such that can easily be squeezed on the end without breaking the ~50 char guide. In this case there were two types, so I ended up with this suggestion for the message heading

    remote-hg: cleanup 40*{0} string, and de-dup tests

which adds just enough (for me) to get a feel for the style of what's inside

regards
Philip
Show 122 quoted lines
> Cleanup 51be46e (remote-hg: do not fail on invalid bookmarks).
>
> Having a 40-characters string is not ideal, and having three tests for
> basically the same relatively rare situation is overkill.
>
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---
> git-remote-hg.py     |  2 +-
> t/t5810-remote-hg.sh | 56 
> ++++------------------------------------------------
> 2 files changed, 5 insertions(+), 53 deletions(-)
>
> diff --git a/git-remote-hg.py b/git-remote-hg.py
> index 402b92f..74f2a2e 100755
> --- a/git-remote-hg.py
> +++ b/git-remote-hg.py
> @@ -677,7 +677,7 @@ def do_list(parser):
>             print "? refs/heads/branches/%s" % gitref(branch)
>
>     for bmark in bmarks:
> -        if  bmarks[bmark].hex() == 
> '0000000000000000000000000000000000000000':
> +        if  bmarks[bmark].hex() == '0' * 40:
>             warn("Ignoring invalid bookmark '%s'", bmark)
>         else:
>             print "? refs/heads/%s" % gitref(bmark)
> diff --git a/t/t5810-remote-hg.sh b/t/t5810-remote-hg.sh
> index ba8b2d8..9946f57 100755
> --- a/t/t5810-remote-hg.sh
> +++ b/t/t5810-remote-hg.sh
> @@ -772,7 +772,7 @@ test_expect_success 'remote double failed push' '
>  )
> '
>
> -test_expect_success 'clone remote with master null bookmark, then 
> push to the bookmark' '
> +test_expect_success 'clone remote with null bookmark, then push' '
>  test_when_finished "rm -rf gitrepo* hgrepo*" &&
>
>  hg init hgrepo &&
> @@ -781,67 +781,19 @@ test_expect_success 'clone remote with master 
> null bookmark, then push to the bo
>  echo a >a &&
>  hg add a &&
>  hg commit -m a &&
> - hg bookmark -r null master
> + hg bookmark -r null bookmark
>  ) &&
>
>  git clone "hg::hgrepo" gitrepo &&
>  check gitrepo HEAD a &&
>  (
>  cd gitrepo &&
> - git checkout --quiet -b master &&
> - echo b >b &&
> - git add b &&
> - git commit -m b &&
> - git push origin master
> - )
> -'
> -
> -test_expect_success 'clone remote with default null bookmark, then 
> push to the bookmark' '
> - test_when_finished "rm -rf gitrepo* hgrepo*" &&
> -
> - hg init hgrepo &&
> - (
> - cd hgrepo &&
> - echo a >a &&
> - hg add a &&
> - hg commit -m a &&
> - hg bookmark -r null -f default
> - ) &&
> -
> - git clone "hg::hgrepo" gitrepo &&
> - check gitrepo HEAD a &&
> - (
> - cd gitrepo &&
> - git checkout --quiet -b default &&
> - echo b >b &&
> - git add b &&
> - git commit -m b &&
> - git push origin default
> - )
> -'
> -
> -test_expect_success 'clone remote with generic null bookmark, then 
> push to the bookmark' '
> - test_when_finished "rm -rf gitrepo* hgrepo*" &&
> -
> - hg init hgrepo &&
> - (
> - cd hgrepo &&
> - echo a >a &&
> - hg add a &&
> - hg commit -m a &&
> - hg bookmark -r null bmark
> - ) &&
> -
> - git clone "hg::hgrepo" gitrepo &&
> - check gitrepo HEAD a &&
> - (
> - cd gitrepo &&
> - git checkout --quiet -b bmark &&
> + git checkout --quiet -b bookmark &&
>  git remote -v &&
>  echo b >b &&
>  git add b &&
>  git commit -m b &&
> - git push origin bmark
> + git push origin bookmark
>  )
> '
>
> -- 
> 1.9.2+fc1.3.gade8541
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

← back to recent threads