Document clone of clone loosing branches?

I wander whether man git-clone is correct when it says "creates remote-tracking branches for each branch in the cloned repository".

IMHO remote-tracking branches in the original repository _are_ branches and they are _not_ cloned (when using git-clone with no options) - maybe this is worth noting very explicitly?

When git newby like me converts a CVS repository, containing just few short old branches (stable release bug fixes) and then clones it around, with naive belief that clone is a 1:1 copy or something close, nasty surprise can happen:

 - converted repository has those branches, OK
 - clone of it also has them, but as a remote tracking branches
 - clone of clone has it as dangling objects only (can go away later)

Trying to play it safe, I used git-clone many times while starting with git, and I got really nervous when I first discovered that my old stable release bug fix branch is not visible in some repositories :-)

Is it just my failure to read those few hundred man pages carefully enough (I did my best :-) ), or something worth fixing in man git-clone and tutorials?

Regards,

Vaclav Hanzl

On Sat, Jun 14, 2008 at 03:05:48PM +0200, Vaclav Hanzl wrote:

The new repository has remote tracking branches of the _regular_ branches that are in the original repository. So the statement is correct; git-clone creates remote-tracking branches, and it makes one such branch for each branch in the cloned repository.

Unless you are complaining that it makes one for each non-remote branch in the cloned repository. But I think it is the general pattern to refer to things in refs/heads/ as simply unadorned "branch". If you want to say "all refs, including remote-tracking branches", you would typically say "refs" (which would also include tags).

In that sense, clone is a bit of a misnomer, because the two _are_ different. The cloned repo has its origin pointing to the original repo, and the origin of the original repo (and its associated tracking branches) are forgotten.

The clone of clone does not have dangling objects; either it sees a ref (because it is a branch in the clone) and it grabs the objects, or it does not see it, in which case it does not download those objects.

Yes, this is sometimes annoying if what you _really_ want is another clone of the clone's origin. For me, this happens because I want to clone Junio's git.git (to do some experimentation that might trash the repo), but:

  - I am too lazy to type Junio's git URL
  - I want the hard-link space-saving of cloning my local repo

So I do something like:

  ref=/path/to/my/git/repo
  git clone --reference $ref `cd $ref && git config remote.origin.url`

which you can easily put in an alias.

-Peff

Yes, there should not be a dangling object, but I actually got one. I was surprised, but I thought it is just an undocumented benign behavior (optimization overkill - clone rather gets those objects instead of thinking what it needs). Now I think it may be a bug.

Can someone with deeper knowledge and fresh git please try this:

 rm -rf A B C
 mkdir A B C
 (cd A; mkdir X; cd X; git init; echo x>x; git add x; git commit -m xxx;
  git checkout -b br; echo y>y; git add y; git commit -m yyy;
  git checkout master)
 (cd B; git clone ../A/X)
 (cd C; git clone ../B/X; cd X; git fsck --full)

With my 11 days old version of git, I get:

 dangling commit ...

and 'git show ...' reveals that this commit it that otherwise (rightfully) lost branch 'br'.

SHA1 is changing when I repeat this (due to dates in reflogs??).

When I omit 'git checkout master', no dangling commit appears. Strange.

Is it my faulty thinking, benign undocumented thing, or a bug?

Vaclav Hanzl

Vaclav Hanzl <hanzl@noel.feld.cvut.cz> writes:

[...]

_Local_ clone? This is result of optimization; in cloning over local filesystem case git-clone simply hardlinks object database (if possible) instead of transferring objects. This was only on request in earlier versions of git.

You can use filr:// protocol to force generating of pack-file and actual transfer of objects.

Still dangling even with 'file:', using (in my case) this:

 pwd
 rm -rf A B C
 mkdir A B C
 (cd A; mkdir X; cd X; git init; echo x>x; git add x; git commit -m xxx;
  git checkout -b y; echo y>y; git add y; git commit -m yyy;
  git checkout master)
 (cd B; git clone file:///home/kroupa/tmp/gittest/A/X; cd X; git branch -a)
 (cd C; git clone file:///home/kroupa/tmp/gittest/B/X; cd X; git branch -a; git fsck --full)

The actual result of pasting this to shell is:

~/tmp/gittest>pwd /home/kroupa/tmp/gittest ~/tmp/gittest>rm -rf A B C ~/tmp/gittest>mkdir A B C ~/tmp/gittest>(cd A; mkdir X; cd X; git init; echo x>x; git add x; git commit -m xxx;

Initialized empty Git repository in .git/
Created initial commit ec59a7e: xxx
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 x
Switched to a new branch "y"
Created commit 2be193d: yyy
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 y
Switched to branch "master"
~/tmp/gittest>(cd B; git clone file:///home/kroupa/tmp/gittest/A/X; cd X; git branch -a)
Initialized empty Git repository in /home/kroupa/tmp/gittest/B/X/.git/
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (6/6), done.
* master
  origin/HEAD
  origin/master
  origin/y
~/tmp/gittest>(cd C; git clone file:///home/kroupa/tmp/gittest/B/X; cd X; git branch -a; git fsck --full)
Initialized empty Git repository in /home/kroupa/tmp/gittest/C/X/.git/
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 6 (delta 0), reused 6 (delta 0)
Receiving objects: 100% (6/6), done.
* master
  origin/HEAD
  origin/master
dangling commit 2be193dc7dc3e1051dcf5af9d03f7587b61f9fc3

And the dangling object is:

~/tmp/gittest>cd C/X; git show 2be193d
commit 2be193dc7dc3e1051dcf5af9d03f7587b61f9fc3
Author: kroupa <kroupa@noli.localdomain>
Date:   Sat Jun 14 22:47:46 2008 +0200

    yyy

Vaclav Hanzl <hanzl@noel.feld.cvut.cz> writes:

[...]

The idea is for git-clone to clone (by default) _your_ work, not sb else work. Think about two repositories, fetching from each other: you don't want for branches to proliferate like mad, remote of remote, then remote of remote of remote, and ad infinitum.

Besides there is I think implicit assumption that public repositories one might want to clone are _bare_ repositories, 1:1 or mirror refspecs, which simply do not contain remote tracking branches.

Vaclav Hanzl wrote:

I got bitten by that, too.  Perhaps the intro paragraph of git-clone.txt 
(man git-clone) could mention that remote branches are not cloned or so. 
  (I don't know git-clone well enough to be able to phrase it 
accurately, but perhaps someone else wants to send a documentation patch...)

-- Lea

Lea Wiemann wrote:

This is exactly what I would like to happen. (If nobody seems interested, I might propose my own bad very first patch few days or weeks later...)

Jeff King wrote:

I am not sure that the term 'branch' can be reasonably expected to mean 'regular branch' unless specified otherwise. For example, 'man git' says:

       git-branch(1)
          List, create, or delete branches.

It can also list remote-tracking branches, cannot it? Or:

       git-show-branch(1)
          Show branches and their commits.

Can also show remote-tracking branches, cannot it?

Even if 'branch' were very well known to mean 'regular branch', I would still argue that 'man clone' is a very good place to define these vocabulary conventions.

So I still think that 'man clone' is wrong as it stands now. It is not true that all 'branches' are cloned. It would be better to say quite explicitly that regular branches are cloned and remote tracking branches are not.

Jakub Narebski wrote:

Yes. It would be no shame if an explanation like this made it to 'man clone'?

After all, how many other commands do distinguish regular branches and remote tracking branches? Even if there are any other (I do not know), git-clone is likely the most prominent of them and 'man git-clone' is quite good place to document this. Unless it is explained in 'man git' itself (I think it is not now).

(Thought I am quite happy with UNIX tradition of very exact and very condensed man pages, up to the point of being a hard puzzle, and I agree that man pages are no tutorial, in this case I would be happy to see 'regular branches' and 'remote tracking branches' clearly distinguished in 'man git-clone' itself, without an implicit reference to 'usual' meaning of words among geeks.)

Regards,

Vaclav Hanzl

P.S. Sorry for screwing up mail threads by this synthetic answer but I thought it is not worth 3 messages (?)

Vaclav Hanzl <hanzl@noel.feld.cvut.cz> writes:

These fall into "quibbling" category, but that judgement can only be made by people who know the history.

Updates to glossary and other introductory documents might be necessary. For example, the entry about "tracking branch" in the glossary still talks about the ancient convention of copying 'master' to 'origin' as a regular branch:

    [[def_tracking_branch]]tracking branch::
            A regular git <<def_branch,branch>> that is used to follow changes from
            another <<def_repository,repository>>. A tracking
            branch should not contain direct modifications or have local commits
            made to it. A tracking branch can usually be
            identified as the right-hand-side <<def_ref,ref>> in a Pull:
            <<def_refspec,refspec>>.

This does _not_ reflect post v1.3.0 reality at all. No git more recent than Apr 2006 has used a "regular git branch" for tracking.

It probably is enough to say:

        A ref that is used to follow changes from another repository.
	In modern git, they are found in `refs/remotes/` hierarchy.

because you cannot add "direct modifications or have local commits" to them anymore.

I allowed myself to quibble much more than usually as I think it serves a good technical purpose. I am a new git user but quite pervasive in getting all the info I can (I have bound copy of all git man pages and about ten tutorials, diligently devoting much of my time last month to reading it all; generally I am not scared by strange things like plain Tex, FORTH, Prolog etc.) so I might serve as a good example of user who tried hard and still did not get it with current docs.

But it is a thin border, so please do not hesitate to shut me up if my judgement of technical merit is wrong.

I would appreciate this update ;-)

Regards (and big thanks for git!),

Vaclav

Vaclav Hanzl wrote:

The question of course is _what_ to put into git-clone(1), what to generic documentation in git(1), and what in "Git User's Manual".

Errr... many of them? git-branch treats regular branches (refs/heads/*) and remote-tracking branches (refs/remotes/<remotename>/*) differently (compare "git branch" and "git branch -r", and "git branch -a"). git-checkout treats regular branches (can checkout) different from other refs, including remote-tracking branches (result in detached HEAD).

I'm not sure how it is put in documentation, but I wouldn't wonder
if it is not dicumented, because most of gitters who can write this
documentation do know the difference between regular branches and
remote-tracking branches, and know recoomended workflows and best
practices.
 

So, what should be mentioned ar two facts, I think. (a) that git-clone copies only regular branches and tags; it does not copy remote-tracking branches, it does not copy stash, it does not copy reflogs, it does not copy StGIT stacks... (b) that recommended workflow (best practice) is to have public published repository which is bare clone (1:1 correspondnce) of interesting subset of refs.

Vaclav Hanzl <hanzl@noel.feld.cvut.cz> writes:

It probably should read "for each _regular_ branch in the cloned repository".

And of course if you are creating bare clone it does mirror regular branches (1:1 mapping) instead of remote-tracking branches (mappping from refs/heads/* into refs/remotes/origin/* namespace).

[...]

Even if it is just your failure it would be worh correcting (enhancing) documentation to make it more clear.

Clarify that a clone is not an exact copy.

Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
---
 Documentation/git-clone.txt |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

Robin Rosenberg wrote:

One nitpick, one comment, one historical comment.

Nitpick: Actually biological clone is usually not identical in
phenotype, and we know now that due to different gene expression
it can have different genotype as well. ;-)

COMMENT: when you do "git clone --bare" it mirrors local (regular)
branches in 1:1 correspondence; but it still doesn't clone anything
outside refs/heads/ and refs/remotes (and HEAD).

Historical comment: git-clone used to do 1:1 mapping even for non-bare clone, with exception of <current branch> -> origin mapping, before behavior provided by --use-separate-remotes was made default (thanks Carl!).

El 15/6/2008, a las 15:05, Robin Rosenberg escribió:

Grammar: "become", not "becomes"

Typo: "security reasons", not "security reasone"

You might also want to mention that the clone is not necessarily a redundant _copy_ of the original repo, because at least for local clones on the same file system clone will use hard links rather than actually making a copy, unless you explicitly tell it not to.

W

At the moment this is covered in description of the --local option (I personally found it very easy to get this knowledge, dislike other differences from 1:1)

Vaclav

In order to better specify what gets cloned and what does not, one might also replace "active branch" (under DESCRIPTION) with "current branch".

According to grep in Documentation/*.txt, "current branch" is used much more often than "active branch". Most frequent use of term "active branch" is found in git-fast-import.txt and it seems to mean something else there.

(The fact that tiny volatile detail - which branch happens to be checked out at the moment of clone - has profound influence on the result of clone was quite a surprise for me, so I would like to see this described as exactly as possible.)

Regards

Vaclav

On Sun, Jun 15, 2008 at 03:05:27PM +0200, Robin Rosenberg <robin.rosenberg.lists@dewire.com> wrote:

s/triggers/hooks/?

Robin Rosenberg <robin.rosenberg.lists@dewire.com> writes:

Thanks.

But the above will not lose any information content if "Although... clone," is dropped. It in fact, dropping that would make it even clearer.

Saying only "beware that neither X nor Y nor Z is done" leaves readers wondering "why not". Saying "you may assume W but that is not the case" is the same. How about expressing it a bit differently and in a more positive way? Documentation is not a place to express your frustration you felt immediately after you found out that your initial assumption did not match reality.

And there is no triggers in git. If you are improving documentation, please get your terminology straight.

I think the first three paragraphs of this manual page need major overhaul. There were not much difference between bare and non-bare clone when historical layout was used, but exactly because a clone with a working tree always use separate remotes layout, what they do is vastly different these days.

Points to stress, and the order to present them, in the rewritten first paragraphs would be:

 - There are two different kind of clones.  A bare one and non bare one.

 - A non-bare one is a way to prepare where you work in.  It is assumed
   that you will want to keep updating the resulting repository from the
   source of the cloning operation from time to time, so 'origin' is set
   up for you automatically to track its branches in remotes/origin.

 - A bare one is a way to prepare where you push into so that others can
   fetch from it.  IOW, it is a place people meet to exchange history, not
   a place where somebody actually works in to build history.  It is
   expected that people will push into the resulting repository from
   elsewhere to update it, rather than somebody will fetch into it further
   from any 'origin'.  No 'origin' is made, and branches are copied 1:1.

 - In either case, only the information from the originating repository
   that are designed to be shared are propagated.  That includes branches
   and tags, but not things that are specific for working in the
   originating repository such as config, stash, hooks and remote tracking
   branches.

Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
---
 Documentation/git-cvsimport.txt |   10 +++++++---
 1 files changed, 7 insertions(+), 3 deletions(-)