threads / patch / 2348

patchCogito: Support for implicit remote branches in cloned repositories

Subject: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

## tl;dr

10 messages between Nov 4, 2005 and Nov 8, 2005. Diffs are folded; open one to read it.

replies: 9people: 4as markdown or json

Josef Weidendorfer· Nov 4, 2005, 16:01 UTC · lore

This allows to specify remote branch names from the repository this one was cloned from (= remote repository of "origin" branch), without explicitly adding a branch for them. Therefore, these remote branches are called implicit.

Use them together with cg-fetch or cg-update.
Example: Your current repository was cloned with
  cg-clone git+ssh://remotehost:/path myrep

If this remote repository contains a branch "foo", you can say inside of myrep:

  cg-fetch foo

This fetches "git+ssh://remotehost:/path#foo" into a local branch "foo" (created if not existing). Similarily, a "cg-update foo" will fetch the remote branch and merge it into your current local branch.

After fetching from an implicit remote branch, cg-status will show the corresponding local branch, but still without an "R", because that is only shown for explicit branches. The implicit remote branch in the example above can be made explicit with

  cg-branch-add foo git+ssh://remotehost:/path#foo

Note that cg-update now always tries to fetch from a remote repository, as every non-explicit branch name is supposed to be an implicit remote branch.

Signed-off-by: Josef Weidendorfer <Josef.Weidendorfer@gmx.de>
---

This patch is RFC. IMHO, it simplifies the usage of Cogito quite a lot. One difference of cloning with GIT vs. with Cogito is that Git always clones all remote branches. This can be limiting if you want to work with multiple repositories, but allows you to immediatly work with all the branches.

Cogito instead clones only one remote branch, and requires you to explicitly add further branches with "cg-branch-add". This is not necessary with implicit remote branches, and they should cover the standard use case which works with branches from the repository you clone from.

The nice thing here is that the patch is really small for the added functionality. If you accept it, I will provide updates for the documentation and tutorial.

Josef
 cg-fetch  |   10 +++++++++-
 cg-update |    6 +-----
 2 files changed, 10 insertions(+), 6 deletions(-)

applies-to: 22948869bcf0ec216ed9aa14e1c5ecc22114d66b 769aa2148fc2b583b071354e8eef3dae74e09c14

Show changes to 2 files +10 −6

cg-fetch, cg-update

diff --git a/cg-fetch b/cg-fetch
index 759488a..5a5aeb1 100755
--- a/cg-fetch
+++ b/cg-fetch
@@ -262,7 +262,15 @@ name=${ARGS[0]}
 
 [ "$name" ] || { [ -s "$_git/branches/origin" ] && name=origin; }
 [ "$name" ] || die "where to fetch from?"
-uri=$(cat "$_git/branches/$name" 2>/dev/null) || die "unknown branch: $name"
+uri=$(cat "$_git/branches/$name" 2>/dev/null) || \
+    { if [ -s "$_git/branches/origin" ]; then
+	uri=$(cat "$_git/branches/origin" 2>/dev/null)
+	uri="$(echo "$uri" | cut -d '#' -f 1)#$name"
+	echo Fetching implicit remote branch "$uri".
+      else
+        die "unknown branch: $name"
+      fi
+    }
 
 rembranch=
 if echo "$uri" | grep -q '#'; then
diff --git a/cg-update b/cg-update
index 96035c5..534bf4a 100755
--- a/cg-update
+++ b/cg-update
@@ -39,11 +39,7 @@ name=${ARGS[0]}
 [ "$name" ] || { [ -s $_git/branches/origin ] && name=origin; }
 [ "$name" ] || die "where to update from?"
 
-if [ -s "$_git/branches/$name" ]; then
-	cg-fetch $force $name || exit 1
-else
-	echo "Updating from a local branch."
-fi
+cg-fetch $force $name || exit 1
 echo
 echo "Applying changes..."
 cg-merge $name
Junio C Hamano· Nov 4, 2005, 17:43 UTC · re: Josef Weidendorfer · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

Josef Weidendorfer <Josef.Weidendorfer@gmx.de> writes:
> ... One difference of cloning with GIT vs. with
> Cogito is that Git always clones all remote branches. This can
> be limiting if you want to work with multiple repositories,
> but allows you to immediatly work with all the branches.
Three points, and two footnotes.
. After a git clone, you can still fetch from non-origin
  repository; you can also set up .git/remotes/somewhere-else if
  you pull from the non-origin repository regularly, so it is
  not really "limiting".  It is just being slightly less
  convenient, in that setting up for the origin is done for you
  by clone [*1*] while you have to arrange for non-origin
  repository yourself afterwards.
. The namespace under refs/heads is and always will be an issue.
  It is a local matter and how remote branches are named should
  not dictate what local branch names you can use in your
  repository, but that essentially is what happens after
  git-clone to users who do not rename those branches from the
  initial cloning.
  I once considered to give an option to clone to map the origin
  heads to .git/refs/heads/origin/{master,maint,pu,...}.  In
  hindsight that might have been cleaner.  Instead I just followed
  what Cogito already established, and mapped remote "master" to
  "origin".
. The namespace under refs/tags theoretically also has the same
  issue, but I suspect it would not matter too much in practice.
  The tags people fetches from remote tend to be release-point
  tags (e.g. v2.6.14) whose names implicitly follow an obvious
  (to humans) naming convention; when you name your temporary
  anchor points using lightweight tags, you can easily avoid
  name clashes with those "for other people" tags [*2*].
[Footnotes]

*1* Actually, even the setting up for the origin is done only halfway -- it only arranges 'git fetch/pull' to fetch from the master branch, and other branches are not tracked unless you explicitly arrange them to be. This is somewhat deliberate; the refs/ namespace management is a local matter and you do not necessarily want to keep tracking all the branches from origin.

*2* This becomes somewhat problematic when the tool automatically follows/fetches tags, and that is why git-core barebone Porcelainish requires an explicit 'git fetch --tags'.

Josef Weidendorfer· Nov 4, 2005, 18:38 UTC · re: Junio C Hamano · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

On Friday 04 November 2005 18:43, Junio C Hamano wrote:
Show 13 quoted lines
> Josef Weidendorfer <Josef.Weidendorfer@gmx.de> writes:
> 
> > ... One difference of cloning with GIT vs. with
> > Cogito is that Git always clones all remote branches. This can
> > be limiting if you want to work with multiple repositories,
> > but allows you to immediatly work with all the branches.
> 
> Three points, and two footnotes.
> 
> . After a git clone, you can still fetch from non-origin
>   repository; you can also set up .git/remotes/somewhere-else if
>   you pull from the non-origin repository regularly, so it is
>   not really "limiting".

With "limiting" I only meant the fact that all the refs/heads names are copied over, thus polluting your local namespace. Additionally, it can be some transfer burden to copy all the objects instead only the objects needed for one head.

Cogito does not have an issue with pollution of refs/heads namespace as it only clones one head to the special "origin" head [*1*]. But as I said, the Git way can be more convenient. I think that the proposed "implicit remote branches" are a way between: make it convenient for Cogito users to fast access branches from the origin repository, but pollute the name space only "a little".

[*1*] Still, it would be nice to specify the name of the branch tracking the origin repository via on option in cg-clone (defaulting to "origin"), and to also be able to specify the head which is branched off and you are put on after cloning (defaulting to "master"); but I do not think there is much use to such flexibility.

>   It is just being slightly less 
>   convenient, in that setting up for the origin is done for you
>   by clone [*1*] while you have to arrange for non-origin
>   repository yourself afterwards.

I do not think there is much difference in editing files under .git/remotes, or using cg-branch-add.

Show 6 quoted lines
> . The namespace under refs/heads is and always will be an issue.
>   It is a local matter and how remote branches are named should
>   not dictate what local branch names you can use in your
>   repository, but that essentially is what happens after
>   git-clone to users who do not rename those branches from the
>   initial cloning.
Yes, and that is nice about Cogito.
Show 5 quoted lines
>   I once considered to give an option to clone to map the origin
>   heads to .git/refs/heads/origin/{master,maint,pu,...}.  In
>   hindsight that might have been cleaner.  Instead I just followed
>   what Cogito already established, and mapped remote "master" to
>   "origin".

That is a nice idea. What is the exact compatiblity problem with Cogito here? Either a repository is cloned with Git or with Cogito, and afterwards, both should be able to cope with any naming scheme.

Both Git and Cogito do setup local "master" for local work after cloning. To still be able to track a remote "master", this is mapped to local "origin". Am I right here? What happens if the remote side itself has an "origin" head?

Show 8 quoted lines
> [Footnotes]
> 
> *1* Actually, even the setting up for the origin is done only
> halfway -- it only arranges 'git fetch/pull' to fetch from the
> master branch, and other branches are not tracked unless you
> explicitly arrange them to be.  This is somewhat deliberate; the
> refs/ namespace management is a local matter and you do not
> necessarily want to keep tracking all the branches from origin.

Yes. I do not see any problem here with explicitly specifying a head you want to track in "git fetch origin <head>". Still, I would not call this syntax 'porcelainish'.

But we have Cogito for this.
> *2* This becomes somewhat problematic when the tool
> automatically follows/fetches tags, and that is why git-core
> barebone Porcelainish requires an explicit 'git fetch --tags'.
Ah, ok.
Josef
Petr Baudis· Nov 4, 2005, 21:08 UTC · re: Junio C Hamano · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

Dear diary, on Fri, Nov 04, 2005 at 06:43:04PM CET, I got a letter where Junio C Hamano <junkio@cox.net> told me that...

Show 5 quoted lines
>   I once considered to give an option to clone to map the origin
>   heads to .git/refs/heads/origin/{master,maint,pu,...}.  In
>   hindsight that might have been cleaner.  Instead I just followed
>   what Cogito already established, and mapped remote "master" to
>   "origin".

Well, that obviously works only when you resign on the workflow where pull means fetch + merge, since your local master would conflict with the remote master, therefore having nowhere to store the remote master.

Show 7 quoted lines
> . The namespace under refs/tags theoretically also has the same
>   issue, but I suspect it would not matter too much in practice.
>   The tags people fetches from remote tend to be release-point
>   tags (e.g. v2.6.14) whose names implicitly follow an obvious
>   (to humans) naming convention; when you name your temporary
>   anchor points using lightweight tags, you can easily avoid
>   name clashes with those "for other people" tags [*2*].

I still believe we need the notion of private tags which shouldn't be cloned.

Hmm. Wait.

All right. git-update-server-info ignores hidden refs, but referencing a hidden ref works all right (unsurprisingly). So let's just codify that private tags which shan't be fetched (unless requested explicitly) start with a dot (/^\./) and we are all set...?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
Junio C Hamano· Nov 4, 2005, 21:50 UTC · re: Petr Baudis · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

Petr Baudis <pasky@suse.cz> writes:
> I still believe we need the notion of private tags which shouldn't be
> cloned.

I agree with you violently. I just do not know what is the right convention to tell which is private and which is not.

Fetching used to be a different matter, because we did not do 'git-fetch --tags' for a reason: you do not have any business with my tags unless I tell you about them. But now we tell others about what tags we have, so...

> All right. git-update-server-info ignores hidden refs, but referencing a
> hidden ref works all right (unsurprisingly). So let's just codify that
> private tags which shan't be fetched (unless requested explicitly) start
> with a dot (/^\./) and we are all set...?

Except that I suspect refs.c::check_ref_format() will barf on it instead of ignoring it, and obviously you would want it to do two different things depending on what kind of operation you are doing. You for example would want to change git-branch or git-tag not to refuse creating such "private" tag. You would still want git-upload-pack to show it to the other end, for better common commit computation purposes, especially if the tag is of lightweight kind, but would want git-clone to ignore them. It should be doable but we first need a plan.

Linus Torvalds· Nov 4, 2005, 22:07 UTC · re: Junio C Hamano · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

On Fri, 4 Nov 2005, Junio C Hamano wrote:
> 
> I agree with you violently.  I just do not know what is the
> right convention to tell which is private and which is not.

I don't like the dot idea, but it certainly _would_ make sense to have a local prefix.

We could even make it a _totally_ different namespace:
	.git/refs/local/heads
	.git/refs/local/tags

which means that even if you call a local head the same thing as a global one, both can still exist.

Then, make "origin" always be a local head.
That avoids the issue of the remote repo _also_ having an "origin" branch.

Locally, the "local" heads would always take precedence, with this trivial patch making sure of that..

If you want to access a global tag or head, you can always do so by using the full path ("refs/tags/tagname" is the global one, but "tags/tagname" or "tagname" would be the local one if one exists).

		Linus
---
Show changes to sha1_name.c +2 −0
diff --git a/sha1_name.c b/sha1_name.c
index be1755a..5daaa11 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -231,6 +231,8 @@ static int get_sha1_basic(const char *st
 	static const char *prefix[] = {
 		"",
 		"refs",
+		"refs/local/tags",
+		"refs/local/heads",
 		"refs/tags",
 		"refs/heads",
 		NULL
Junio C Hamano· Nov 6, 2005, 09:11 UTC · re: Linus Torvalds · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

Linus Torvalds <torvalds@osdl.org> writes:
Show 12 quoted lines
> I don't like the dot idea, but it certainly _would_ make sense to have a 
> local prefix.
>
> We could even make it a _totally_ different namespace:
>
> 	.git/refs/local/heads
> 	.git/refs/local/tags
>
> which means that even if you call a local head the same thing as a global 
> one, both can still exist.
>
> Then, make "origin" always be a local head.

While that is logically the right thing to do, it is somewhat confusing to see all those branch heads we obtained from the remote side stored as our "local" branches -- meaning they are private to us.

Just for discussion, I'd suggest a less structured approach as an alternative, and making it just a convention.

 - clone by default [*1*] creates .git/refs/heads/origin/*
   heads; your "master" becomes my "origin/master".
   Interestingly but logically, if your repository was cloned
   from somewhere else, you would have "origin/master", which
   would become my "origin/origin/master".
 - clone creates remotes/origin that knows all branches from the
   origin as copied by the above process, as suggested by Daniel
   in the other thread.

Then we would have a recommended workflow example that uses origin/* as reference branches after they are set up. After becoming familiar with this setup, when people fetch/pull from more than one repository, they can configure things correctly by emulating what clone did to the "origin" remote repository for them.

[Footnote]

*1* I say "by default" here; I think there should be an operating mode for git clone that acts as a straight copy, not even the current "HEAD" -> "origin" renaming. In fact that is what you do with low-level git-clone-pack.

Petr Baudis· Nov 7, 2005, 23:21 UTC · re: Junio C Hamano · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

Dear diary, on Sun, Nov 06, 2005 at 10:11:10AM CET, I got a letter where Junio C Hamano <junkio@cox.net> told me that...

> Just for discussion, I'd suggest a less structured approach as
> an alternative, and making it just a convention.

The problem is that at least Cogito defaults to the 'origin' branch when pulling/merging, so if you'll have 'origin/master' instead of 'origin', I will have to hack that around. So it's not _just_ a convention but something reflected in some code as well.

Show 5 quoted lines
>  - clone by default [*1*] creates .git/refs/heads/origin/*
>    heads; your "master" becomes my "origin/master".
>    Interestingly but logically, if your repository was cloned
>    from somewhere else, you would have "origin/master", which
>    would become my "origin/origin/master".

This loses the pretty nice property that 'origin' refers to the HEAD branch of the remote repository, no matter how is it called (well, it is nice as long as the remote repository doesn't _switch_ the HEAD to some non-superset branch; but repositories you're cloning from shouldn't do that). So you should either have origin/HEAD locally, or make the 'master' branch more "socially canonical".

> *1* I say "by default" here; I think there should be an
> operating mode for git clone that acts as a straight copy, not
> even the current "HEAD" -> "origin" renaming.  In fact that is
> what you do with low-level git-clone-pack.
What would it be useful for?
-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
Junio C Hamano· Nov 7, 2005, 23:45 UTC · re: Petr Baudis · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

Petr Baudis <pasky@suse.cz> writes:
> This loses the pretty nice property that 'origin' refers to the HEAD
> branch of the remote repository,...

I think the code actually honors what your HEAD points at; I did not go into that detail in the above description.

Show 6 quoted lines
>> *1* I say "by default" here; I think there should be an
>> operating mode for git clone that acts as a straight copy, not
>> even the current "HEAD" -> "origin" renaming.  In fact that is
>> what you do with low-level git-clone-pack.
>
> What would it be useful for?
Cheap back-up, Linus style.
Josef Weidendorfer· Nov 8, 2005, 16:46 UTC · re: Petr Baudis · lore

Re: [PATCH] Cogito: Support for implicit remote branches in cloned repositories

On Tuesday 08 November 2005 00:21, Petr Baudis wrote:
> The problem is that at least Cogito defaults to the 'origin' branch when
> pulling/merging, so if you'll have 'origin/master' instead of 'origin',
> I will have to hack that around. So it's not _just_ a convention but
> something reflected in some code as well.

The new git-clone simply would not produce an "origin" branch at all, and I do not think Cogito expects that a repository always has an "origin" branch.

The following fact (Git's default clone setup until now, Cogito's clone setup)
	"origin" tracks remote "master", to be merged into local "master"

has to be stored into some configuration file, and not to be set up implicitly as it is done currently. Then you take this setup from the configuration, and there never would be something hardcoded, and Junios suggestion would be a convention.

Git already stores half of this into remotes/repo: the "Pull" lines specify which local head tracks which remote head; the same with Cogito's .git/branches. Missing in both cases is the information that "origin" maps to the local development head "master". More common name for this information: "origin" is the upstream for "master".

This information is useful at least for two cases: (1) for specifying the default action to be done on "git-pull" or "cg-update", and (2) try to prohibit accidental bogus merges.

The default action currently is either hardcoded (Cogito) or working somehow by luck (Git: you have to be on "master" for git-pull). AFAIK, Cogito prohibits bogus merges by enforcing to be either on "master" or to be in a not-mergeable state (cg-seeking).

Merge relationship is useful to be specified not only for upstream direction, but also for branches which branched off the current head and should be merged back (e.g. topical branches). For this, multiple branches to be merged with "git pull" should be specifyable.

Where to put the information about default merge relation?

Both .git/remotes and .git/branches are candidates, but merge relation also makes sense for local branches, i.e. git-branch <branch> <origbranch> should set up to merge from <origbranch> if you are on <branch> by default. Similarly, it should set up a relation the other way to state that it is useful to merge from <branch> into <origbranch> (this is for (2) above).

A possiblity would be to put the merge relationship into .git/config:

[master] merge-canditate: origin merge-default: origin

Problem here is that head names should not appear in key names, because key names are case insensitive :-( A possibility would be for config keys to become case sensitive after a ":" (of course only for toplevel keys):

[head:master] merge-default: origin

Show 12 quoted lines
> >  - clone by default [*1*] creates .git/refs/heads/origin/*
> >    heads; your "master" becomes my "origin/master".
> >    Interestingly but logically, if your repository was cloned
> >    from somewhere else, you would have "origin/master", which
> >    would become my "origin/origin/master".
> 
> This loses the pretty nice property that 'origin' refers to the HEAD
> branch of the remote repository, no matter how is it called (well, it is
> nice as long as the remote repository doesn't _switch_ the HEAD to some
> non-superset branch; but repositories you're cloning from shouldn't do
> that). So you should either have origin/HEAD locally, or make the
> 'master' branch more "socially canonical".

Ah, so git-clone sets up a local development head for the remote HEAD, which does not always have to be "master", right? This information would be available by the proposed storing of merge relationship: local "master" wants get merges from remote HEAD to be updated.

Josef

← back to recent threads