Pushing to GitHub doesn't push all branches

Hi,

What am I doing wrong here. I 'git svn clone' a SubVersion repository and when it was done, I was placed in the "master" branch, which seems to track the "trunk" branch of the remote.

I then added GitHub as a remote repository:

$ git remote add origin git@github.com:graemeg/freepascal.git

I then tried to push all the SubVersion branches to GitHub, but only the "master" branch went through. I thought the --mirror will push everything.

Any ideas?

You can view the repository on GitHub with your Web Browser at this URL:
   http://github.com/graemeg/freepascal/tree
As you will notice, under the "all branches" option, only "master" is 
available.

=======================
$ git branch -a
* master
   remotes/aspect
   remotes/avr
   remotes/cleanroom
   remotes/cpstr
   remotes/ctypes
   remotes/fixes_2_0
   remotes/fixes_2_2
   remotes/fixes_2_2_0_dos
   remotes/florian
   remotes/fpc_2_3
   remotes/generics
   remotes/genfunc
   remotes/inline
   remotes/janbruns
   remotes/linker
   remotes/linker@2665
   remotes/llvm
   remotes/merged
   remotes/newthreading
   remotes/objc
   remotes/origin/master
   remotes/rc_2_2_2
   remotes/release_2_1_2x
   remotes/release_2_2_4_rc1
   remotes/resources
   remotes/ssa
   remotes/tags/release_2_0_0
   remotes/tags/release_2_0_2
   remotes/tags/release_2_0_4
   remotes/tags/release_2_0_4_macos
   remotes/tags/release_2_0_4_rc1
   remotes/tags/release_2_0_4_rc2
   remotes/tags/release_2_0_4_rc3
   remotes/tags/release_2_1_2
   remotes/tags/release_2_1_4
   remotes/tags/release_2_2_0
   remotes/tags/release_2_2_2
   remotes/tags/release_2_2_2_rc1
   remotes/tags/release_2_2_2_rc2
   remotes/tags/release_2_2_4
   remotes/tags/test_2_0_4
   remotes/tags/test_2_0_4_1
   remotes/trunk
   remotes/tue
   remotes/unicodertl
   remotes/unicodestring
   remotes/unitrw
   remotes/wpo
=======================

$ git push -v --mirror origin
Pushing to git@github.com:graemeg/freepascal.git
Counting objects: 130317, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (38881/38881), done.
Writing objects: 100% (130317/130317), 54.53 MiB | 51 KiB/s, done.
Total 130317 (delta 100296), reused 120757 (delta 90739)
To git@github.com:graemeg/freepascal.git
  * [new branch]      master -> master
  * [new branch]      aspect -> aspect
  * [new branch]      avr -> avr
  * [new branch]      cleanroom -> cleanroom
  * [new branch]      cpstr -> cpstr
  * [new branch]      ctypes -> ctypes
  * [new branch]      fixes_2_0 -> fixes_2_0
  * [new branch]      fixes_2_2 -> fixes_2_2
  * [new branch]      fixes_2_2_0_dos -> fixes_2_2_0_dos
  * [new branch]      florian -> florian
  * [new branch]      fpc_2_3 -> fpc_2_3
  * [new branch]      generics -> generics
  * [new branch]      genfunc -> genfunc
  * [new branch]      inline -> inline
  * [new branch]      janbruns -> janbruns
  * [new branch]      linker -> linker
  * [new branch]      linker@2665 -> linker@2665
  * [new branch]      llvm -> llvm
  * [new branch]      merged -> merged
  * [new branch]      newthreading -> newthreading
  * [new branch]      objc -> objc
  * [new branch]      rc_2_2_2 -> rc_2_2_2
  * [new branch]      release_2_1_2x -> release_2_1_2x
  * [new branch]      release_2_2_4_rc1 -> release_2_2_4_rc1
  * [new branch]      resources -> resources
  * [new branch]      ssa -> ssa
  * [new branch]      tags/release_2_0_0 -> tags/release_2_0_0
  * [new branch]      tags/release_2_0_2 -> tags/release_2_0_2
  * [new branch]      tags/release_2_0_4 -> tags/release_2_0_4
  * [new branch]      tags/release_2_0_4_macos -> tags/release_2_0_4_macos
  * [new branch]      tags/release_2_0_4_rc1 -> tags/release_2_0_4_rc1
  * [new branch]      tags/release_2_0_4_rc2 -> tags/release_2_0_4_rc2
  * [new branch]      tags/release_2_0_4_rc3 -> tags/release_2_0_4_rc3
  * [new branch]      tags/release_2_1_2 -> tags/release_2_1_2
  * [new branch]      tags/release_2_1_4 -> tags/release_2_1_4
  * [new branch]      tags/release_2_2_0 -> tags/release_2_2_0
  * [new branch]      tags/release_2_2_2 -> tags/release_2_2_2
  * [new branch]      tags/release_2_2_2_rc1 -> tags/release_2_2_2_rc1
  * [new branch]      tags/release_2_2_2_rc2 -> tags/release_2_2_2_rc2
  * [new branch]      tags/release_2_2_4 -> tags/release_2_2_4
  * [new branch]      tags/test_2_0_4 -> tags/test_2_0_4
  * [new branch]      tags/test_2_0_4_1 -> tags/test_2_0_4_1
  * [new branch]      trunk -> trunk
  * [new branch]      tue -> tue
  * [new branch]      unicodertl -> unicodertl
  * [new branch]      unicodestring -> unicodestring
  * [new branch]      unitrw -> unitrw
  * [new branch]      wpo -> wpo
updating local tracking ref 'refs/remotes/origin/master'

Regards,
   - Graeme -

Graeme Geldenhuys venit, vidit, dixit 10.07.2009 15:24:

git-svn's standard ref structure makes it easy to shoot yourself into the foot, and so does "--mirror". Combining the two is really a powerful weapon :)

$git ls-remote git://github.com/graemeg/freepascal.git 840169d189370c3b08760654800990404b2218df HEAD 840169d189370c3b08760654800990404b2218df refs/heads/master 3a5d5a06bbe62fada139c86438348e30ff4fcbe7 refs/remotes/aspect f8a21acf439e698e7cb57a2a68401cfca2486c0c refs/remotes/avr 78d1c138d4755f26ae1515842fcf36f13bf6b030 refs/remotes/cleanroom b2701a7710bc343257b47738a06cb1290cec902f refs/remotes/cpstr 92c78d3c8efe0bce64fda3f949596f37465bb6bb refs/remotes/ctypes

etc.

You really mirrored your repo: All your "lost" branches are remotes on the github side as well. That has two consequences:

1) They're not shown in the shiny interface.
2) They don't get cloned when someone clones from there.

At least 2) is completely unrelated to github, btw.

I think what you really want is to push everything under remote (assuming there are only svn branches) into proper heads on github, i.e. a refspec like '+refs/remotes/*:refs/*' for your pushes.

I assume you want to push master only if it differs from trunk, i.e. if that repos is going to host not only the svn mirror but also your additions. (I don't recommended that, a fork is better for that.)

Michael

Michael J Gruber venit, vidit, dixit 10.07.2009 16:45: ...

Uhm, make that '+refs/remotes/*:refs/heads/*' ;)

Michael J Gruber wrote:

The two branches that are of most importance to me, is "trunk" and "fixes_2_2" as found in the SubVersion repository.

refs/remotes/trunk refs/remotes/fixes_2_2

So should I have only pushed the above mentioned branches, but as "true" heads in GitHub. Geesh, I hope I am understanding what I am typing, because I feel a bit lost now. :-)

Is there any way to clean up the mess available on GitGub? So that 'git ls-remote ...' will only show the real remotes.... Or should there be no remotes on GitHub?

Sorry, I'm fairly new to Git and it feels like I jumped into the deap end here. ;-)

I'll read the man pages on what that refspec means... If I manage to only push 'trunk' which is master under git and 'fixes_2_2' which will be some other name under git, how to I keep both those in sync with the SubVersion repository.

At the moment I have a cronjob that executes the following every 30 minutes. ==================== cd /mnt/samba/git/fpc.git/ $GIT checkout master $GIT svn rebase $GIT gc --auto $GIT push origin master ====================

Does 'git svn rebase' get all branch or does it just update "master" (Trunk from SubVersion)?

I apologise for all the questions...

Regards,
   - Graeme -

Graeme Geldenhuys venit, vidit, dixit 10.07.2009 17:07:

Please don't! That's what we're here for ;)

A while ago I suggested that by default, no clone (whether git or git-svn) should have a master branch. I guess your example shows why. If you use that repo (/mnt/samba/git/fpc.git) only for converting from svn to git, not for doing local work, then you don't need any master branch. But git-svn will stubbornly recreate one.

Similarly (under the same assumption), you don't need to rebase at all. All you need is "git svn fetch", which populates the branches under remotes/.

You can safely delete the bogus remote branches on github using

git push origin :refs/remotes/trunk

etc., i.e.

git for-each-ref --shell --format="git push -f origin :%(refname)" refs/remotes/|while read line; do eval $line;done

(all on one line)

Then it's probably beneficial to do something like

git config remote.origin.push '+refs/remotes/*:refs/heads/*'

so that you have a nice default refspec for origin. [You may think about renaming origin to destination, though ;) ]

Cheers, Michael

Michael J Gruber wrote:

:) Thanks.

OK, done this. 'git ls-remote github' doesn't show any remotes/* references anymore.

Oh by the way, I followed your advice and renamed 'origin' to 'github' so it makes a bit more sense. I used the following command:

   $ git remote rename origin github

OK, I've done this, but I'm not 100% sure what this means. This is what I think (from reading various git help and users guide). Any branches I have which track remote references (from svn repository) will be pushed to github as various head references (normal branches). Is this correct?

On our server, which contain the svn cloned repository and the one that is only used for syncing svn -> our server -> github, I had to add a new commit which represents the svn-ignore meta data to a .gitignore file. I noticed I had to manually to a 'git checkout master' & 'git merge remotes/trunk' to pull in new updates. I thin pushed that to github.

So what is my twice hourly cron script supposed to look like? Is the following still ok?

========[ script executed by cron every 30 minutes ]============ #!/bin/sh GIT="/usr/local/bin/git"

# FPC repository cd /mnt/samba/git/fpc.git/ $GIT checkout master $GIT svn fetch $GIT gc --auto $GIT push github master ==============================

Or do I need to add a new line after '$GIT svn fetch' that does a merge.

============================== $GIT svn fetch $GIT merge remotes/trunk ... push to github ==============================

Also what do I do with the other branch I want to track. It's called remotes/fixes_2_2.

I believe I need to first create a local branch.

   git branch --track fixes_2_2 remotes/fixes_2_2

What do I need to modify in my cron script to keep both the master & fixes_2_2 branches in sync with SubVersion and push both to GitHub. I want Github to show two branches: master & fixes_2_2

Regards,
   - Graeme -

Graeme Geldenhuys venit, vidit, dixit 13.07.2009 10:12:

Yes, any ref you have that matches the pattern refs/remotes/* (all your remote branches) is pushed to a ref named refs/heads/* in the repo (i.e. a branch). The + means that non-fast-forward pushes are OK.

You didn't tell us *that* before... Exactly this is why I asked whether you want to do own work on those branches or just push a git-svn mirror.

I'm not sure adding .gitignore in a commit is a good idea, if that is the only change you will make to the original branches. It keeps you from doing a simple fetch and forces you to set up merging.

Also, svn-metadata may change over time. Do you intend to keep .gitignore up to date with those changes?

Well, the way you used push you completely ignored the suggested refspec. You pushed master into a matching ref (master). master may have happened to be trunk before you added .gitignore (git-svn does not guarantee that).

So, there are two options:

- Push the git-svn converted branches as is, using the suggested refspec
(or having it in config and doing "git push github"). "git svn fetch"
will have fetched all svn refs you need.

- Set up local branches and do your merge thing. In this case you
probably also want to keep up with svn metadata changes (mind that
ignore properties may depend on the branch as well). Done right this is
nice, but of course it may be a bit more fragile.

In the first case you would need to distribute .gitignore in a different way. You could set up a separate branch (say svnhelper) for that which consists of .gitignore (and possibly other things) only. "git show svnhelper:.gitignore > .git/info/excludes" will set up each user. You can also use a tag for that (i.e. instead of a branch).

Cheers, Michael

Michael J Gruber wrote:

When we last spoke, I did not have a .gitignore file. :) That is the only change I plan to make in that repository (on our server).

I have cloned that repository (on our server) to my local PC. The one on my PC is a true Git repository and that is the one I'll be making changes to and emailing patches back to the FPC mailing list.

OK, so seeing that I already done that and pushed it to GitHub, I now need to always do the following on our server git repository.

  $ git svn fetch
  $ git checkout master
  $ git merge origin/trunk
  $ git push github            <-- updates remotes/* to heads/*
  $ git push github master     <-- updates remote master branch

So 'git push github' as I have show above should do the trick for all refts/remotes/* branches. I gather that excludes the "master" branch, so I still need to push that one manually as well. Correct?

In the newer Git v1.6.x versions there is a new default setting that can be set for push.

   $ git config push.default <type>

If I set that to "matching", then I should be able to only need one 'git push github' command, and that should push remotes/* and the "master" branch. Correct?

In my local repository on my work PC (not the server), I normally set the push.default to "current" so I don't accidentally push something that has an incomplete feature.

No, just the initial setup.

OK, I haven't added a .gitignore to the fixes_2_2 local branch, so that one is still exactly as it was in SubVersion. And my local "fixes_2_2" has not been pushed to Github yet. So I should be able to just delete the local "fixes_2_2" branch. The normal refspec will push changes to Github as it should. So if anybody clones the Github repository, they can manually create their local "fixes_2_2" from the refs/remotes/fixes_2_2 branch. Correct?

I think I'm starting to see a little white light in the end of the tunnel. :-)

I think I will leave the local "master" branch (mapped to SubVersion Trunk) as-is. I'll then modify my cron script to do a manual merge after the 'git svn fetch'.

I'm not going to bother updating the .gitignore again. So from now onwards the merge should be a simple fast-forward merge I take it and should never get conflicts again.

Very clever indeed - pity I did not think of it before. I'll make a note of this and if I ever clone another SubVersion repository, I'll do it with a tag or separate branch.

Regards,
   - Graeme -