threads / discuss / 31415

Clone to an SSH destination

Subject: Clone to an SSH destination

## tl;dr

10 messages between Sep 3, 2012 and Sep 3, 2012.

replies: 9people: 5as markdown or json

Mark Hills· Sep 3, 2012, 10:21 UTC · lore

How do I clone a repo _to_ a new repo over SSH? I tried:

  cd xx
  git clone --bare . gitserver:/scm/xx.git
  git clone --bare . ssh://gitserver/scm/xx.git

This does not have the expected result, and instead a local path of the given name is created (eg. a 'gitserver:' directory)

This seems to be a FAQ, but the only answer I can find (Google) is to 
login to the server and create the repo, setup a remote and push to it.
 
This is quite cumbersome; we have a large team of devs who use a simple 
'git clone' to an NFS directory, but we wish to retire NFS access.

Is there a technical limiation preventing clone-to-ssh, or just something waiting to be implemented?

(Please keep me CC'd; I am currently reading the web archives)

Many thanks

-- 
Mark

Ævar Arnfjörð Bjarmason· Sep 3, 2012, 11:26 UTC · re: Mark Hills · lore

Re: Clone to an SSH destination

On Mon, Sep 3, 2012 at 12:21 PM, Mark Hills <Mark.Hills@framestore.com> wrote:

Show 11 quoted lines

> How do I clone a repo _to_ a new repo over SSH? I tried:
>
>   cd xx
>   git clone --bare . gitserver:/scm/xx.git
>   git clone --bare . ssh://gitserver/scm/xx.git
>
> This does not have the expected result, and instead a local path of the
> given name is created (eg. a 'gitserver:' directory)
>
> This seems to be a FAQ, but the only answer I can find (Google) is to
> login to the server and create the repo, setup a remote and push to it.

Basically Git doesn't support this yet, mainly because it could only be supported with the ssh or local transports.

With anything else it would break, so push can only assume that something on the other end can receive data, can update branch pointers etc. Not create a brand new repository.

You could of course supply your devs with an alias that ssh's to that server, does an init if needed, and then does a push.

Show 5 quoted lines

> This is quite cumbersome; we have a large team of devs who use a simple
> 'git clone' to an NFS directory, but we wish to retire NFS access.
>
> Is there a technical limiation preventing clone-to-ssh, or just something
> waiting to be implemented?

But I'm actually more curious about why you need this in the first place, there's a bunch of devs where I work as well, but they never have the need to create new repos on some NFS drive in this manner.

What are your devs doing when they do clone their current working directory to some NFS location, maybe there's a better way to do it.

Mark Hills· Sep 3, 2012, 13:07 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: Clone to an SSH destination

On Mon, 3 Sep 2012, Ævar Arnfjörð Bjarmason wrote:

Show 22 quoted lines

> On Mon, Sep 3, 2012 at 12:21 PM, Mark Hills <Mark.Hills@framestore.com> wrote:
> > How do I clone a repo _to_ a new repo over SSH? I tried:
> >
> >   cd xx
> >   git clone --bare . gitserver:/scm/xx.git
> >   git clone --bare . ssh://gitserver/scm/xx.git
> >
> > This does not have the expected result, and instead a local path of the
> > given name is created (eg. a 'gitserver:' directory)
> >
> > This seems to be a FAQ, but the only answer I can find (Google) is to
> > login to the server and create the repo, setup a remote and push to it.
> 
> Basically Git doesn't support this yet, mainly because it could only
> be supported with the ssh or local transports.
> 
> With anything else it would break, so push can only assume that
> something on the other end can receive data, can update branch
> pointers etc. Not create a brand new repository.
> 
> You could of course supply your devs with an alias that ssh's to that
> server, does an init if needed, and then does a push.

Yes, a last resort, but do-able of course.

Show 9 quoted lines

> > This is quite cumbersome; we have a large team of devs who use a simple
> > 'git clone' to an NFS directory, but we wish to retire NFS access.
> >
> > Is there a technical limiation preventing clone-to-ssh, or just something
> > waiting to be implemented?
> 
> But I'm actually more curious about why you need this in the first
> place, there's a bunch of devs where I work as well, but they never
> have the need to create new repos on some NFS drive in this manner.

Without a command-line onto the filesystem (either local or NFS), how do you create a new repository for a new project?

We have a fairly large team on a diverse set of projects. Projects come and go, so it's a burden if the administrator is needed just to create repos.

Likewise, it's a step backwards for the developer to need to login themselves over SSH -- whereas 'git clone' is so easy to NFS.

> What are your devs doing when they do clone their current working 
> directory to some NFS location, maybe there's a better way to do it.

Most projects start as a small test at some point; eg.

  mkdir xx
  cd xx
  git init
  <write some code>
  git commit
  ...

When a project becomes more official, the developer clones to a central location; eg.

  git clone --bare . /net/git/xx.git

This is the step that is inconvenient if only SSH access is available.

Thanks for your reply,

-- 
Mark

Konstantin Khomoutov· Sep 3, 2012, 14:08 UTC · re: Mark Hills · lore

Re: Clone to an SSH destination

On Mon, 3 Sep 2012 14:07:48 +0100 (BST) Mark Hills <Mark.Hills@framestore.com> wrote:

[...]

Show 32 quoted lines

> > But I'm actually more curious about why you need this in the first
> > place, there's a bunch of devs where I work as well, but they never
> > have the need to create new repos on some NFS drive in this manner.
> 
> Without a command-line onto the filesystem (either local or NFS), how
> do you create a new repository for a new project?
> 
> We have a fairly large team on a diverse set of projects. Projects
> come and go, so it's a burden if the administrator is needed just to
> create repos.
> 
> Likewise, it's a step backwards for the developer to need to login 
> themselves over SSH -- whereas 'git clone' is so easy to NFS.
> 
> > What are your devs doing when they do clone their current working 
> > directory to some NFS location, maybe there's a better way to do it.
> 
> Most projects start as a small test at some point; eg.
> 
>   mkdir xx
>   cd xx
>   git init
>   <write some code>
>   git commit
>   ...
> 
> When a project becomes more official, the developer clones to a
> central location; eg.
> 
>   git clone --bare . /net/git/xx.git
> 
> This is the step that is inconvenient if only SSH access is available.

Well, then it looks you want something like github. In this case look at some more integrated solution such as Gitlab [1] -- I did not try it, but it looks like you import your users there and then they can log in, add their SSH keys and create their projects.

I also think gitolite has some way to actually use regular SSH users (or even users coming from a web server which is a front-end for Smart HTTP Git transport, doing its own authentication). This is explained in [2], and I hope Sitaram could provide more insight on setting things up this way, if needed (I did not use this feature).

1. http://gitlabhq.com/
2. http://sitaramc.github.com/gitolite/g2/auth.html

Sitaram Chamarty· Sep 3, 2012, 14:26 UTC · re: Konstantin Khomoutov · lore

Re: Clone to an SSH destination

On Mon, Sep 3, 2012 at 7:38 PM, Konstantin Khomoutov <flatworm@users.sourceforge.net> wrote:

Show 41 quoted lines

> On Mon, 3 Sep 2012 14:07:48 +0100 (BST)
> Mark Hills <Mark.Hills@framestore.com> wrote:
>
> [...]
>> > But I'm actually more curious about why you need this in the first
>> > place, there's a bunch of devs where I work as well, but they never
>> > have the need to create new repos on some NFS drive in this manner.
>>
>> Without a command-line onto the filesystem (either local or NFS), how
>> do you create a new repository for a new project?
>>
>> We have a fairly large team on a diverse set of projects. Projects
>> come and go, so it's a burden if the administrator is needed just to
>> create repos.
>>
>> Likewise, it's a step backwards for the developer to need to login
>> themselves over SSH -- whereas 'git clone' is so easy to NFS.
>>
>> > What are your devs doing when they do clone their current working
>> > directory to some NFS location, maybe there's a better way to do it.
>>
>> Most projects start as a small test at some point; eg.
>>
>>   mkdir xx
>>   cd xx
>>   git init
>>   <write some code>
>>   git commit
>>   ...
>>
>> When a project becomes more official, the developer clones to a
>> central location; eg.
>>
>>   git clone --bare . /net/git/xx.git
>>
>> This is the step that is inconvenient if only SSH access is available.
>
> Well, then it looks you want something like github.
> In this case look at some more integrated solution such as Gitlab [1]
> -- I did not try it, but it looks like you import your users there and
> then they can log in, add their SSH keys and create their projects.

Anything web based would be even more overhead than a simple:

    ssh server git init --bare foo/bar.git && git push --mirror
ssh://git/~/foo/bar.git

Gitolite of course is even closer, as we discussed earlier.

Show 5 quoted lines

> I also think gitolite has some way to actually use regular SSH users
> (or even users coming from a web server which is a front-end for Smart
> HTTP Git transport, doing its own authentication).  This is explained
> in [2], and I hope Sitaram could provide more insight on setting things
> up this way, if needed  (I did not use this feature).

As I said earlier, regardless of how he does it, authentication will change, since he is no longer using a local (well, locally mounted) file system as the "server". That may be "get everyone to send us a pub key" or "give everyone an http password and use smart http".

In addition, if they choose smart http, they *have to* use gitolite. Unlike ssh, where that two command sequence above would do it all for them, there is no eqvt if your git server is behind http.

Show 7 quoted lines

>
> 1. http://gitlabhq.com/
> 2. http://sitaramc.github.com/gitolite/g2/auth.html
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Sitaram

Enrico Weigelt· Sep 3, 2012, 14:27 UTC · re: Mark Hills · lore

Re: Clone to an SSH destination

Hi,

> Without a command-line onto the filesystem (either local or NFS), how
> do you create a new repository for a new project?

These things are out of the scope of git itself, they belong to another layer - you're probably looking for some repo hosting solution.

One already was mentioned: gitolite. Another one is gitosis.

> We have a fairly large team on a diverse set of projects. Projects
> come and go, so it's a burden if the administrator is needed just to
> create repos.

Maybe you'd also like to have some project management / ticketing tool. We've got an redmine and gitolite based solution for that, that's automatically creating repos and managing access control on per-project basis. Will be publically released in a few days, just drop a note if you're interested in it.

Show 14 quoted lines

> Most projects start as a small test at some point; eg.
> 
>   mkdir xx
>   cd xx
>   git init
>   <write some code>
>   git commit
>   ...
> 
> When a project becomes more official, the developer clones to a
> central
> location; eg.
> 
>   git clone --bare . /net/git/xx.git

Do you have any kind of access control in place, or can everybody write just everywhere ?

cu

-- 
Mit freundlichen Grüßen / Kind regards 

Enrico Weigelt 
VNC - Virtual Network Consult GmbH 
Head Of Development 

Pariser Platz 4a, D-10117 Berlin
Tel.: +49 (30) 3464615-20
Fax: +49 (30) 3464615-59

enrico.weigelt@vnc.biz; www.vnc.de

Konstantin Khomoutov· Sep 3, 2012, 11:47 UTC · re: Mark Hills · lore

Re: Clone to an SSH destination

On Mon, 3 Sep 2012 11:21:43 +0100 (BST) Mark Hills <Mark.Hills@framestore.com> wrote:

Show 8 quoted lines

> How do I clone a repo _to_ a new repo over SSH? I tried:
> 
>   cd xx
>   git clone --bare . gitserver:/scm/xx.git
>   git clone --bare . ssh://gitserver/scm/xx.git
>
> This does not have the expected result, and instead a local path of
> the given name is created (eg. a 'gitserver:' directory)

No, `git clone` is intended to work with your local machine. And I can't really imagine how it would work in a way you want.

Show 9 quoted lines

> This seems to be a FAQ, but the only answer I can find (Google) is to 
> login to the server and create the repo, setup a remote and push to
> it. 
> This is quite cumbersome; we have a large team of devs who use a
> simple 'git clone' to an NFS directory, but we wish to retire NFS
> access.
> 
> Is there a technical limiation preventing clone-to-ssh, or just
> something waiting to be implemented?

I think this is more a conceptual issue: such a "remote cloning" would mean *creation* of a repository as part of the process. I'm not aware of any [D]VCS system which would routinely allow such a thing: usually the creation of a repository is an "administrative" act, which is distinct from regular (push/fetch) access to that repository.

gitolite kind of implements this ("wild repos") [1], you could look if it suits your needs.

Now back to your problem.

If you have NFS access, you can probably do proper cloning using NFS:
1) Export via NFS the directory in which you want your resulting
   bare repository located.
2) Do simple *local* clone (using paths accessed by NFS):
   git clone --bare /path/to/source/repo /path/to/destination/repo
3) Do certain fixups in the destination repo, if needed, then
   access it normally via SSH (or whatever protocol you prefer).

If you don't want to do this, then revert to a normal two-step
procedure:
1) Log into the server, create a repo by hand:
   git init --bare /path/to/repo
2) Push everything relevant from a source repo there:
   cd /path/to/local/repo
   git remote add origin ssh://gitserver/scm/xx.git
   git push origin refs/*:refs/*

Step (1) may depends on what server-side solution your need; for instance, in gitolite you would be configuring a new repo in the configuration file in your local clone of the gitolite admin repo and pushing your changes to the server.

1. http://sitaramc.github.com/gitolite/wild.html

Sitaram Chamarty· Sep 3, 2012, 12:54 UTC · re: Konstantin Khomoutov · lore

Re: Clone to an SSH destination

On Mon, Sep 3, 2012 at 5:17 PM, Konstantin Khomoutov <flatworm@users.sourceforge.net> wrote:

> On Mon, 3 Sep 2012 11:21:43 +0100 (BST)
> Mark Hills <Mark.Hills@framestore.com> wrote:

[snip]

>> This is quite cumbersome; we have a large team of devs who use a
>> simple 'git clone' to an NFS directory, but we wish to retire NFS
>> access.

[snip]

> gitolite kind of implements this ("wild repos") [1], you could look if
> it suits your needs.

The simplest conf to do what you want in gitolite is something like this:

    repo [a-zA-Z0-9]..*
        C   =   @all
        RW+ =   @all

But of course your *user* authentication will probably change quite a bit, since gitolite runs as one Unix user and merely simulates many "gitolite users", while in the NFS method each of your devs probably has a full login to the server.

Mark Hills· Sep 3, 2012, 13:15 UTC · re: Sitaram Chamarty · lore

Re: Clone to an SSH destination

On Mon, 3 Sep 2012, Sitaram Chamarty wrote:

Show 26 quoted lines

> On Mon, Sep 3, 2012 at 5:17 PM, Konstantin Khomoutov
> <flatworm@users.sourceforge.net> wrote:
> > On Mon, 3 Sep 2012 11:21:43 +0100 (BST)
> > Mark Hills <Mark.Hills@framestore.com> wrote:
> 
> [snip]
> 
> >> This is quite cumbersome; we have a large team of devs who use a
> >> simple 'git clone' to an NFS directory, but we wish to retire NFS
> >> access.
> 
> [snip]
> 
> > gitolite kind of implements this ("wild repos") [1], you could look if
> > it suits your needs.
> 
> The simplest conf to do what you want in gitolite is something like this:
> 
>     repo [a-zA-Z0-9]..*
>         C   =   @all
>         RW+ =   @all
> 
> But of course your *user* authentication will probably change quite a
> bit, since gitolite runs as one Unix user and merely simulates many
> "gitolite users", while in the NFS method each of your devs probably
> has a full login to the server.

I'll check out gitolite, thanks.

We use unix users extensively (groups, permissions etc.) with YP, and this works well; a separate permissions scheme is not very desireable.

The ssh method works very well right now, and nicely transparent. It's only the initial clone/creation that is harder than it was over NFS. And it prevents the use of git-shell too.

Thanks

-- 
Mark

Sitaram Chamarty· Sep 3, 2012, 13:48 UTC · re: Mark Hills · lore

Re: Clone to an SSH destination

On Mon, Sep 3, 2012 at 6:45 PM, Mark Hills <Mark.Hills@framestore.com> wrote:

Show 36 quoted lines

> On Mon, 3 Sep 2012, Sitaram Chamarty wrote:
>
>> On Mon, Sep 3, 2012 at 5:17 PM, Konstantin Khomoutov
>> <flatworm@users.sourceforge.net> wrote:
>> > On Mon, 3 Sep 2012 11:21:43 +0100 (BST)
>> > Mark Hills <Mark.Hills@framestore.com> wrote:
>>
>> [snip]
>>
>> >> This is quite cumbersome; we have a large team of devs who use a
>> >> simple 'git clone' to an NFS directory, but we wish to retire NFS
>> >> access.
>>
>> [snip]
>>
>> > gitolite kind of implements this ("wild repos") [1], you could look if
>> > it suits your needs.
>>
>> The simplest conf to do what you want in gitolite is something like this:
>>
>>     repo [a-zA-Z0-9]..*
>>         C   =   @all
>>         RW+ =   @all
>>
>> But of course your *user* authentication will probably change quite a
>> bit, since gitolite runs as one Unix user and merely simulates many
>> "gitolite users", while in the NFS method each of your devs probably
>> has a full login to the server.
>
> I'll check out gitolite, thanks.
>
> We use unix users extensively (groups, permissions etc.) with YP, and this
> works well; a separate permissions scheme is not very desireable.
> The ssh method works very well right now, and nicely transparent. It's
> only the initial clone/creation that is harder than it was over NFS. And
> it prevents the use of git-shell too.

If I had to do this, and didn't want to use gitolite or something like it, I'd just make a script that will create the repo using an ssh call then do a 'git push --mirror' to it.

Call it "git-new" or something and train people to use that instead of "clone" when the repo doesn't even exist yet.

Bound to be easier than the administrative hassle you spoke of in your other email...