threads / patch / 33836

patch, 8 partsremote-bzr: patches for next

Subject: [PATCH 0/8] remote-bzr: patches for next

## tl;dr

12 messages between May 16, 2013 and May 16, 2013. Diffs are folded; open one to read it.

replies: 11people: 2as markdown or json

Felipe Contreras· May 16, 2013, 08:36 UTC · lore
Hi,

These patches have been cooking in my github repository, and improve the situation when bzr servers don't support repositories properly.

Felipe Contreras (8):
  remote-bzr: recover from failed clones
  remote-bzr: fix for files with spaces
  remote-bzr: simplify get_remote_branch()
  remote-bzr: delay cloning/pulling
  remote-bzr: change global repo
  remote-bzr: trivial cleanups
  remote-bzr: reorganize the way 'wanted' works
  remote-bzr: add fallback check for a partial clone
 contrib/remote-helpers/git-remote-bzr | 102 +++++++++++++++++-----------------
 1 file changed, 50 insertions(+), 52 deletions(-)
-- 
1.8.3.rc2.542.g24820ba
Felipe Contreras· May 16, 2013, 08:36 UTC · re: Felipe Contreras · lore

[PATCH 1/8] remote-bzr: recover from failed clones

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
Show changes to contrib/remote-helpers/git-remote-bzr +10 −8
diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index b295dd4..7cd9ed8 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -768,22 +768,24 @@ def get_remote_branch(origin, remote_branch, name):
     global dirname, peers
 
     branch_path = os.path.join(dirname, 'clone', name)
-    if os.path.exists(branch_path):
-        # pull
+
+    try:
         d = bzrlib.bzrdir.BzrDir.open(branch_path)
         branch = d.open_branch()
-        try:
-            branch.pull(remote_branch, [], None, False)
-        except bzrlib.errors.DivergedBranches:
-            # use remote branch for now
-            return remote_branch
-    else:
+    except bzrlib.errors.NotBranchError:
         # clone
         d = origin.sprout(branch_path, None,
                 hardlink=True, create_tree_if_local=False,
                 force_new_repo=False,
                 source_branch=remote_branch)
         branch = d.open_branch()
+    else:
+        # pull
+        try:
+            branch.pull(remote_branch, [], None, False)
+        except bzrlib.errors.DivergedBranches:
+            # use remote branch for now
+            return remote_branch
 
     return branch
 

-- 
1.8.3.rc2.542.g24820ba
Felipe Contreras· May 16, 2013, 08:36 UTC · re: Felipe Contreras · lore

[PATCH 2/8] remote-bzr: fix for files with spaces

Set the maximum number of splits to make when dividing the diff stat lines based on space characters.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to contrib/remote-helpers/git-remote-bzr +1 −1
diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 7cd9ed8..b849336 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -620,7 +620,7 @@ def parse_commit(parser):
             mark = int(mark_ref[1:])
             f = { 'mode' : m, 'mark' : mark }
         elif parser.check('D'):
-            t, path = line.split(' ')
+            t, path = line.split(' ', 1)
             f = { 'deleted' : True }
         else:
             die('Unknown file command: %s' % line)

-- 
1.8.3.rc2.542.g24820ba
Felipe Contreras· May 16, 2013, 08:36 UTC · re: Felipe Contreras · lore

[PATCH 3/8] remote-bzr: simplify get_remote_branch()

No need for 'origin', it's only needed for the bzrdir 'sprout' method, which can be greatly simplified.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)
Show changes to contrib/remote-helpers/git-remote-bzr +12 −11
diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index b849336..b7656df 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -764,25 +764,26 @@ def do_list(parser):
     print "@refs/heads/%s HEAD" % master_branch
     print
 
-def get_remote_branch(origin, remote_branch, name):
+def clone(path, remote_branch):
+    bdir = bzrlib.bzrdir.BzrDir.create(path)
+    repo = bdir.find_repository()
+    repo.fetch(remote_branch.repository)
+    return remote_branch.sprout(bdir, repository=repo)
+
+def get_remote_branch(remote_branch, name):
     global dirname, peers
 
     branch_path = os.path.join(dirname, 'clone', name)
 
     try:
-        d = bzrlib.bzrdir.BzrDir.open(branch_path)
-        branch = d.open_branch()
+        branch = bzrlib.branch.Branch.open(branch_path)
     except bzrlib.errors.NotBranchError:
         # clone
-        d = origin.sprout(branch_path, None,
-                hardlink=True, create_tree_if_local=False,
-                force_new_repo=False,
-                source_branch=remote_branch)
-        branch = d.open_branch()
+        branch = clone(branch_path, remote_branch)
     else:
         # pull
         try:
-            branch.pull(remote_branch, [], None, False)
+            branch.pull(remote_branch, overwrite=True)
         except bzrlib.errors.DivergedBranches:
             # use remote branch for now
             return remote_branch
@@ -850,7 +851,7 @@ def get_repo(url, alias):
 
         if not is_local:
             peers[name] = remote_branch.base
-            branch = get_remote_branch(origin, remote_branch, name)
+            branch = get_remote_branch(remote_branch, name)
         else:
             branch = remote_branch
 
@@ -868,7 +869,7 @@ def get_repo(url, alias):
 
             if not is_local:
                 peers[name] = remote_branch.base
-                branch = get_remote_branch(origin, remote_branch, name)
+                branch = get_remote_branch(remote_branch, name)
             else:
                 branch = remote_branch
 

-- 
1.8.3.rc2.542.g24820ba
Felipe Contreras· May 16, 2013, 08:36 UTC · re: Felipe Contreras · lore

[PATCH 4/8] remote-bzr: delay cloning/pulling

Until the branch is actually going to be used.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 28 +++++++++++++---------------
 1 file changed, 13 insertions(+), 15 deletions(-)
Show changes to contrib/remote-helpers/git-remote-bzr +13 −15
diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index b7656df..2ba49ff 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -277,7 +277,7 @@ def export_branch(repo, name):
     ref = '%s/heads/%s' % (prefix, name)
     tip = marks.get_tip(name)
 
-    branch = bzrlib.branch.Branch.open(branches[name])
+    branch = get_remote_branch(name)
     repo = branch.repository
 
     branch.lock_read()
@@ -589,7 +589,7 @@ def parse_commit(parser):
 
     if ref.startswith('refs/heads/'):
         name = ref[len('refs/heads/'):]
-        branch = bzrlib.branch.Branch.open(branches[name])
+        branch = get_remote_branch(name)
     else:
         die('unknown ref')
 
@@ -691,7 +691,7 @@ def do_export(parser):
     for ref, revid in parsed_refs.iteritems():
         if ref.startswith('refs/heads/'):
             name = ref[len('refs/heads/'):]
-            branch = bzrlib.branch.Branch.open(branches[name])
+            branch = get_remote_branch(name)
             branch.generate_revision_history(revid, marks.get_tip(name))
 
             if name in peers:
@@ -748,7 +748,7 @@ def do_list(parser):
             master_branch = name
         print "? refs/heads/%s" % name
 
-    branch = bzrlib.branch.Branch.open(branches[master_branch])
+    branch = get_remote_branch(master_branch)
     branch.lock_read()
     for tag, revid in branch.tags.get_tag_dict().items():
         try:
@@ -770,8 +770,12 @@ def clone(path, remote_branch):
     repo.fetch(remote_branch.repository)
     return remote_branch.sprout(bdir, repository=repo)
 
-def get_remote_branch(remote_branch, name):
-    global dirname, peers
+def get_remote_branch(name):
+    global dirname, branches
+
+    remote_branch = bzrlib.branch.Branch.open(branches[name])
+    if isinstance(remote_branch.user_transport, bzrlib.transport.local.LocalTransport):
+        return remote_branch
 
     branch_path = os.path.join(dirname, 'clone', name)
 
@@ -851,13 +855,10 @@ def get_repo(url, alias):
 
         if not is_local:
             peers[name] = remote_branch.base
-            branch = get_remote_branch(remote_branch, name)
-        else:
-            branch = remote_branch
 
-        branches[name] = branch.base
+        branches[name] = remote_branch.base
 
-        return branch.repository
+        return remote_branch.repository
     else:
         # repository
 
@@ -869,11 +870,8 @@ def get_repo(url, alias):
 
             if not is_local:
                 peers[name] = remote_branch.base
-                branch = get_remote_branch(remote_branch, name)
-            else:
-                branch = remote_branch
 
-            branches[name] = branch.base
+            branches[name] = remote_branch.base
 
         return repo
 

-- 
1.8.3.rc2.542.g24820ba
Felipe Contreras· May 16, 2013, 08:36 UTC · re: Felipe Contreras · lore

[PATCH 5/8] remote-bzr: change global repo

It's not used anyway.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to contrib/remote-helpers/git-remote-bzr +2 −2
diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 2ba49ff..fdc2e69 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -858,7 +858,7 @@ def get_repo(url, alias):
 
         branches[name] = remote_branch.base
 
-        return remote_branch.repository
+        return origin
     else:
         # repository
 
@@ -873,7 +873,7 @@ def get_repo(url, alias):
 
             branches[name] = remote_branch.base
 
-        return repo
+        return origin
 
 def fix_path(alias, orig_url):
     url = urlparse.urlparse(orig_url, 'file')

-- 
1.8.3.rc2.542.g24820ba
Felipe Contreras· May 16, 2013, 08:36 UTC · re: Felipe Contreras · lore

[PATCH 6/8] remote-bzr: trivial cleanups

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
Show changes to contrib/remote-helpers/git-remote-bzr +7 −7
diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index fdc2e69..dd3d71c 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -814,7 +814,7 @@ def find_branches(repo, wanted):
         except bzrlib.errors.NotBranchError:
             continue
         else:
-            yield name, branch
+            yield name, branch.base
 
 def get_repo(url, alias):
     global dirname, peer, branches
@@ -851,12 +851,12 @@ def get_repo(url, alias):
         # branch
 
         name = 'master'
-        remote_branch = origin.open_branch()
+        branch = origin.open_branch().base
 
         if not is_local:
-            peers[name] = remote_branch.base
+            peers[name] = branch
 
-        branches[name] = remote_branch.base
+        branches[name] = branch
 
         return origin
     else:
@@ -866,12 +866,12 @@ def get_repo(url, alias):
         # stupid python
         wanted = [e for e in wanted if e]
 
-        for name, remote_branch in find_branches(repo, wanted):
+        for name, branch in find_branches(repo, wanted):
 
             if not is_local:
-                peers[name] = remote_branch.base
+                peers[name] = branch
 
-            branches[name] = remote_branch.base
+            branches[name] = branch
 
         return origin
 

-- 
1.8.3.rc2.542.g24820ba
Felipe Contreras· May 16, 2013, 08:36 UTC · re: Felipe Contreras · lore

[PATCH 7/8] remote-bzr: reorganize the way 'wanted' works

If the user specified a list of branches, we ignore what the remote repository lists, and simply use the branches directly. Since some remotes don't report the branches correctly, this is useful.

Otherwise either fetch the repo, or the branch.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 48 +++++++++++++++--------------------
 1 file changed, 21 insertions(+), 27 deletions(-)
Show changes to contrib/remote-helpers/git-remote-bzr +21 −27
diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index dd3d71c..434e613 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -794,7 +794,7 @@ def get_remote_branch(name):
 
     return branch
 
-def find_branches(repo, wanted):
+def find_branches(repo):
     transport = repo.user_transport
 
     for fn in transport.iter_files_recursive():
@@ -805,9 +805,6 @@ def find_branches(repo, wanted):
         name = name if name != '' else 'master'
         name = name.replace('/', '+')
 
-        if wanted and not name in wanted:
-            continue
-
         try:
             cur = transport.clone(subdir)
             branch = bzrlib.branch.Branch.open_from_transport(cur)
@@ -845,35 +842,32 @@ def get_repo(url, alias):
             except bzrlib.errors.NotBranchError:
                 pass
 
-    try:
-        repo = origin.open_repository()
-    except bzrlib.errors.NoRepositoryPresent:
-        # branch
-
-        name = 'master'
-        branch = origin.open_branch().base
+    wanted = get_config('remote-bzr.branches').rstrip().split(', ')
+    # stupid python
+    wanted = [e for e in wanted if e]
 
-        if not is_local:
-            peers[name] = branch
+    if not wanted:
+        try:
+            repo = origin.open_repository()
+        except bzrlib.errors.NoRepositoryPresent:
+            wanted = ['master']
 
-        branches[name] = branch
+    if wanted:
+        def list_wanted(url, wanted):
+            for name in wanted:
+                subdir = name if name != 'master' else ''
+                yield name, bzrlib.urlutils.join(url, subdir)
 
-        return origin
+        branch_list = list_wanted(url, wanted)
     else:
-        # repository
-
-        wanted = get_config('remote-bzr.branches').rstrip().split(', ')
-        # stupid python
-        wanted = [e for e in wanted if e]
-
-        for name, branch in find_branches(repo, wanted):
+        branch_list = find_branches(repo)
 
-            if not is_local:
-                peers[name] = branch
-
-            branches[name] = branch
+    for name, url in branch_list:
+        if not is_local:
+            peers[name] = url
+        branches[name] = url
 
-        return origin
+    return origin
 
 def fix_path(alias, orig_url):
     url = urlparse.urlparse(orig_url, 'file')

-- 
1.8.3.rc2.542.g24820ba
Junio C Hamano· May 16, 2013, 18:35 UTC · re: Felipe Contreras · lore

Re: [PATCH 7/8] remote-bzr: reorganize the way 'wanted' works

The series fails to apply (on top of the hotfix for 1.8.3 you sent earlier today) at this point. It wants to remove the code added by the hotfix but I do not see a corresponding check for listable, which may make sense, as the log message says, when the user specified a list of branches:

    If the user specified a list of branches, we ignore what the remote
    repository lists, and simply use the branches directly. Since some
    remotes don't report the branches correctly, this is useful.

but it is unclear if the listable code needs to go somewhere else when the user didn't.

I'll stop at 6/8 for now, as this is not urgent before the 1.8.3 release.

Felipe Contreras· May 16, 2013, 23:09 UTC · re: Junio C Hamano · lore

Re: [PATCH 7/8] remote-bzr: reorganize the way 'wanted' works

On Thu, May 16, 2013 at 1:35 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 15 quoted lines
> The series fails to apply (on top of the hotfix for 1.8.3 you sent
> earlier today) at this point.  It wants to remove the code added by
> the hotfix but I do not see a corresponding check for listable,
> which may make sense, as the log message says, when the user
> specified a list of branches:
>
>     If the user specified a list of branches, we ignore what the remote
>     repository lists, and simply use the branches directly. Since some
>     remotes don't report the branches correctly, this is useful.
>
> but it is unclear if the listable code needs to go somewhere else
> when the user didn't.
>
> I'll stop at 6/8 for now, as this is not urgent before the 1.8.3
> release.

I can send the conflict fix, or I can send a v2 of the 47-something patches. I already have a few fixes on top that might make sense to squash. Up to you.

-- 
Felipe Contreras
Junio C Hamano· May 16, 2013, 23:38 UTC · re: Felipe Contreras · lore

Re: [PATCH 7/8] remote-bzr: reorganize the way 'wanted' works

Felipe Contreras <felipe.contreras@gmail.com> writes:
Show 6 quoted lines
>> I'll stop at 6/8 for now, as this is not urgent before the 1.8.3
>> release.
>
> I can send the conflict fix, or I can send a v2 of the 47-something
> patches. I already have a few fixes on top that might make sense to
> squash. Up to you.

Probably neither at this point in the pre-1.8.3 cycle; a reroll that includes these 6 or 8 patches as its early part after the next cycle begins would be a good way to go.

Felipe Contreras· May 16, 2013, 08:36 UTC · re: Felipe Contreras · lore

[PATCH 8/8] remote-bzr: add fallback check for a partial clone

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-bzr | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Show changes to contrib/remote-helpers/git-remote-bzr +4 −1
diff --git a/contrib/remote-helpers/git-remote-bzr b/contrib/remote-helpers/git-remote-bzr
index 434e613..acc0dc9 100755
--- a/contrib/remote-helpers/git-remote-bzr
+++ b/contrib/remote-helpers/git-remote-bzr
@@ -765,7 +765,10 @@ def do_list(parser):
     print
 
 def clone(path, remote_branch):
-    bdir = bzrlib.bzrdir.BzrDir.create(path)
+    try:
+        bdir = bzrlib.bzrdir.BzrDir.create(path)
+    except bzrlib.errors.AlreadyControlDirError:
+        bdir = bzrlib.bzrdir.BzrDir.open(path)
     repo = bdir.find_repository()
     repo.fetch(remote_branch.repository)
     return remote_branch.sprout(bdir, repository=repo)

-- 
1.8.3.rc2.542.g24820ba

← back to recent threads