{"thread":{"id":"32874","subject":"Pushing a git repository to a new server","startedAt":"2013-02-10T21:00:56Z","lastAt":"2013-02-13T08:08:36Z","messageCount":10,"participants":["Ethan Reesor","Konstantin Khomoutov","Jeff King","Michael J Gruber"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"209149","messageId":"CAE_TNin0Kb_38gnx9W36VZ8CTxYBZ9T1Dkhar1DUFHyQUq7ebg@mail.gmail.com","threadId":"32874","inReplyTo":null,"subject":"Pushing a git repository to a new server","fromName":"Ethan Reesor","fromEmail":"firelizzard@gmail.com","sentAt":"2013-02-10T21:00:56Z","receivedAt":"2013-02-10T21:00:56Z","isPatch":false,"sender":{"key":"firelizzard@gmail.com","avatar":"https://gravatar.com/avatar/65a178b01509a9c779e386b21651ed62eae41049adb69ed4ac31ea4a4dbdcd98?d=mp&s=160"},"body":"I'm looking to make a command to push a git repo to a new server. The\nway I just did it is as follows:\n\nlocalhost> git clone --bare /path/to/MyRepo /path/to/tmpdir/MyRepo.git\nlocalhost> tar xz /path/to/tmpdir/MyRepo.git | ssh myuser@remotehost\ntar cz \\~/      # If I don't escape '~', my local machine expands it\nlocalhost> ssh myuser@remotehost\nremotehost> sudo chown -R git:git MyRepo.git\n\nThe reason I had to use my user is the git user's shell is git-prompt\nand ~git/git-shell-commands is empty. I have repos set up using\n'git@remotehost:MyOtherRepo.git' as the remote and everything works.\n\nHow do I make a git command that can talk to the server using\ngit-prompt like the other commands do?\n\n--\nEthan Reesor\n"},{"id":"209234","messageId":"20130211075040.GJ5210@localhost.localdomain","threadId":"32874","inReplyTo":"CAE_TNin0Kb_38gnx9W36VZ8CTxYBZ9T1Dkhar1DUFHyQUq7ebg@mail.gmail.com","subject":"Re: Pushing a git repository to a new server","fromName":"Konstantin Khomoutov","fromEmail":"kostix+git@007spb.ru","sentAt":"2013-02-11T07:50:40Z","receivedAt":"2013-02-11T07:50:40Z","isPatch":false,"sender":{"key":"kostix+git@007spb.ru","avatar":null},"body":"On Sun, Feb 10, 2013 at 04:00:56PM -0500, Ethan Reesor wrote:\n\n> I'm looking to make a command to push a git repo to a new server. The\n> way I just did it is as follows:\n> \n> localhost> git clone --bare /path/to/MyRepo /path/to/tmpdir/MyRepo.git\n> localhost> tar xz /path/to/tmpdir/MyRepo.git | ssh myuser@remotehost\n> tar cz \\~/      # If I don't escape '~', my local machine expands it\n> localhost> ssh myuser@remotehost\n> remotehost> sudo chown -R git:git MyRepo.git\n\nWhat's wrong with\n$ ssh myuser@remotehost 'mkdir /path/to/MyRepo.git; cd $_; git init --bare'\n$ git push --all git@remotehost:MyOtherRepo.git\n?\n\n> The reason I had to use my user is the git user's shell is git-prompt\n\nThere's no such thing as git-prompt.  The restricted login shell for\nSSH-only access typically used for such a \"virtual\" Git user is\ngit-shell.\n\n> and ~git/git-shell-commands is empty. I have repos set up using\n> 'git@remotehost:MyOtherRepo.git' as the remote and everything works.\n> \n> How do I make a git command that can talk to the server using\n> git-prompt like the other commands do?\n\nIt's not really clear what do you want to achieve.\nThe reason the git-shell shell is *restricted* (read its manual page)\nis to shrink the surface of possible attacks in the case the shell\naccount used for accessing Git repos over SSH is compromized (the key or\npassword stolen, for instance).  This is achieved by only allowing\ncommands like git-upload-pack etc in the shell (no general file\nmanipulation commands etc).  So what creating \"git command that can\ntalk to the server using git-prompt ...\" would really buy you?\n\nI think the way to go is to start using gitolite [1] or implement by\nhand a subset of what it does (a custom login shell which is allowed to\ndo certain things in a special area of the filesystem designated to keep\nGit repositories) or just set up a special account on the server\n(\"git-admin\", for instance) which would have a regular login shell set\nfor it and would be in the same group as the user \"git\" (or even have\nthe same UID) so that they could share the files they create (subject to\nactive umasks of processes run as both users though).\n\n1. https://github.com/sitaramc/gitolite\n"},{"id":"209236","messageId":"CAE_TNin6-weutRDToZ7-BBGJTCcf0dwJn0ChUbFcACRU=SbjzA@mail.gmail.com","threadId":"32874","inReplyTo":"20130211075040.GJ5210@localhost.localdomain","subject":"Re: Pushing a git repository to a new server","fromName":"Ethan Reesor","fromEmail":"firelizzard@gmail.com","sentAt":"2013-02-11T07:57:51Z","receivedAt":"2013-02-11T07:57:51Z","isPatch":false,"sender":{"key":"firelizzard@gmail.com","avatar":"https://gravatar.com/avatar/65a178b01509a9c779e386b21651ed62eae41049adb69ed4ac31ea4a4dbdcd98?d=mp&s=160"},"body":"On Mon, Feb 11, 2013 at 2:50 AM, Konstantin Khomoutov\n<kostix+git@007spb.ru> wrote:\n> What's wrong with\n> $ ssh myuser@remotehost 'mkdir /path/to/MyRepo.git; cd $_; git init --bare'\n> $ git push --all git@remotehost:MyOtherRepo.git\n> ?\n\nNothing, I just wanted to make myself a command to do that for me.\n\n>> The reason I had to use my user is the git user's shell is git-prompt\n>\n> There's no such thing as git-prompt.  The restricted login shell for\n> SSH-only access typically used for such a \"virtual\" Git user is\n> git-shell.\n\nSorry, git-prompt is something I made for myself. I meant git-shell.\n\n> It's not really clear what do you want to achieve.\n> The reason the git-shell shell is *restricted* (read its manual page)\n> is to shrink the surface of possible attacks in the case the shell\n> account used for accessing Git repos over SSH is compromized (the key or\n> password stolen, for instance).  This is achieved by only allowing\n> commands like git-upload-pack etc in the shell (no general file\n> manipulation commands etc).  So what creating \"git command that can\n> talk to the server using git-prompt ...\" would really buy you?\n\nI want to create a git-command that 1) creates a bare version of the\ncurrent repo, 2) and uploads it to the specified path on my server\n(using tar, but that's not the point).\n\nMy problem is that I have no idea how things like git-push works via a\nuser with git-shell. Can you only run certain git commands, like\ngit-upload-pack? Because I tried running 'ssh git@server git status'\nand that failed.\n\n> I think the way to go is to start using gitolite [1] or implement by\n> hand a subset of what it does (a custom login shell which is allowed to\n> do certain things in a special area of the filesystem designated to keep\n> Git repositories) or just set up a special account on the server\n> (\"git-admin\", for instance) which would have a regular login shell set\n> for it and would be in the same group as the user \"git\" (or even have\n> the same UID) so that they could share the files they create (subject to\n> active umasks of processes run as both users though).\n\nI thought about the secondary user idea. I decided that trying to make\nmy own command would be more fun.\n\n\n-- \nEthan Reesor (Gmail)\n"},{"id":"209245","messageId":"20130211164518.ad0a21aff672ad0e4f03a6bb@domain007.com","threadId":"32874","inReplyTo":"CAE_TNin6-weutRDToZ7-BBGJTCcf0dwJn0ChUbFcACRU=SbjzA@mail.gmail.com","subject":"Re: Pushing a git repository to a new server","fromName":"Konstantin Khomoutov","fromEmail":"kostix+git@007spb.ru","sentAt":"2013-02-11T12:45:18Z","receivedAt":"2013-02-11T12:45:18Z","isPatch":false,"sender":{"key":"kostix+git@007spb.ru","avatar":null},"body":"On Mon, 11 Feb 2013 02:57:51 -0500\nEthan Reesor <firelizzard@gmail.com> wrote:\n\n[...]\n> I want to create a git-command that 1) creates a bare version of the\n> current repo, 2) and uploads it to the specified path on my server\n> (using tar, but that's not the point).\n\nThanks, it's now a bit more clear.\n\n> My problem is that I have no idea how things like git-push works via a\n> user with git-shell. Can you only run certain git commands, like\n> git-upload-pack?\n[...]\n\nPrecisely so.  With additional twist that you can create (or link)\nother commands under ~/git-shell-commands, and these will be available\nas well.\n\nOK, here's the sketch.\nOn the server, in the home directory of your \"git\" user, you create a\nwrapper around git-receive-pack, like this:\n\n# mkdir ~git/git-shell-commands\n# cat >~git/git-shell-commands/git-receive-new-repo\n#!/bin/sh\n\nset -e -u\n\nif [ $# -ne 1 ]; then\n        echo 'Missing required argument: <directory>' >&2\n        exit 1\nfi\n\nmkdir \"$1\" && git init --quiet --bare \"$1\" && git-receive-pack \"$1\"\n^D\n# chmod +x $_\n\nThen, on the client side, to push a new repo, you just do\n\n$ git push --receive-pack=git-receive-new-repo --all git@server:repo.git\n\nThis will make `git push` to spawn not just `git receive-pack <dir>` as\nit usually does but your wrapper, which would first create and\ninitialize a bare repository and then spawn `git receive-pack` on it\nwhich would then communicate with the client side and receive\neverything from it.\n\nYou could then create a client-side wrapper script or a Git alias for\nsuch \"creative pushing\", like this:\n\n$ git config --add --global alias.push-new-repo \\\n  'push --receive-pack=git-receive-new-repo --all'\n\nSo the whole client call is now reduced to\n\n$ git push-new-repo git@server:repo.git\n"},{"id":"209261","messageId":"20130211162714.GB16402@sigill.intra.peff.net","threadId":"32874","inReplyTo":"CAE_TNin6-weutRDToZ7-BBGJTCcf0dwJn0ChUbFcACRU=SbjzA@mail.gmail.com","subject":"Re: Pushing a git repository to a new server","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-02-11T16:27:14Z","receivedAt":"2013-02-11T16:27:14Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 11, 2013 at 02:57:51AM -0500, Ethan Reesor wrote:\n\n> On Mon, Feb 11, 2013 at 2:50 AM, Konstantin Khomoutov\n> <kostix+git@007spb.ru> wrote:\n> > What's wrong with\n> > $ ssh myuser@remotehost 'mkdir /path/to/MyRepo.git; cd $_; git init --bare'\n> > $ git push --all git@remotehost:MyOtherRepo.git\n> > ?\n> \n> Nothing, I just wanted to make myself a command to do that for me.\n\nWe talked about this a long time ago. One problem is that it's\ninherently unportable, as the procedure to make a repo is potentially\ndifferent on every server (and certainly that is the case between a\nregular user running stock git and something like GitHub or Google Code;\nI imagine even gitolite has some special procedures for creating repos,\ntoo).\n\nOne proposal made in the previous discussion was to define a microformat\nfor repository administration commands. So that you could connect and\nsay \"git admin-create-repo /path/to/MyRepo.git\", and the server-provided\nadmin-create-repo command would take care of the details. Then stock git\ncould forward it to \"git init --bare\", GitHub could do the same and\ncreate the necessary database records, etc.\n\nAnd once that standardized method was in place, it would be easy to add\na \"--create\" option to \"git push\" to request an \"admin-create-repo\"\nbefore pushing.\n\nI still think that's a reasonable way forward, but nobody was interested\nenough to start writing code for it.\n\n-Peff\n"},{"id":"209285","messageId":"CAE_TNi=ZN8L=Qy08=UPBi_1sOixJS6qwtKF6o9KFEfTSQnAs5g@mail.gmail.com","threadId":"32874","inReplyTo":"20130211162714.GB16402@sigill.intra.peff.net","subject":"Re: Pushing a git repository to a new server","fromName":"Ethan Reesor","fromEmail":"firelizzard@gmail.com","sentAt":"2013-02-11T18:17:29Z","receivedAt":"2013-02-11T18:17:29Z","isPatch":false,"sender":{"key":"firelizzard@gmail.com","avatar":"https://gravatar.com/avatar/65a178b01509a9c779e386b21651ed62eae41049adb69ed4ac31ea4a4dbdcd98?d=mp&s=160"},"body":"On Mon, Feb 11, 2013 at 11:27 AM, Jeff King <peff@peff.net> wrote:\n[...]\n> We talked about this a long time ago. One problem is that it's\n> inherently unportable, as the procedure to make a repo is potentially\n> different on every server (and certainly that is the case between a\n> regular user running stock git and something like GitHub or Google Code;\n> I imagine even gitolite has some special procedures for creating repos,\n> too).\n\nI was more interested in creating something for my self rather than\nmaking any changes to the mainstream git.\n"},{"id":"209286","messageId":"CAE_TNi=NW1F3ZjYD3CF6y5qRb=6eQwdmX2ZY4g0SExn-k7Ez-g@mail.gmail.com","threadId":"32874","inReplyTo":"20130211164518.ad0a21aff672ad0e4f03a6bb@domain007.com","subject":"Re: Pushing a git repository to a new server","fromName":"Ethan Reesor","fromEmail":"firelizzard@gmail.com","sentAt":"2013-02-11T18:18:38Z","receivedAt":"2013-02-11T18:18:38Z","isPatch":false,"sender":{"key":"firelizzard@gmail.com","avatar":"https://gravatar.com/avatar/65a178b01509a9c779e386b21651ed62eae41049adb69ed4ac31ea4a4dbdcd98?d=mp&s=160"},"body":"On Mon, Feb 11, 2013 at 7:45 AM, Konstantin Khomoutov\n<kostix+git@007spb.ru> wrote:\n[...]\n> OK, here's the sketch.\n> On the server, in the home directory of your \"git\" user, you create a\n> wrapper around git-receive-pack, like this:\n>\n> # mkdir ~git/git-shell-commands\n> # cat >~git/git-shell-commands/git-receive-new-repo\n> #!/bin/sh\n>\n> set -e -u\n>\n> if [ $# -ne 1 ]; then\n>         echo 'Missing required argument: <directory>' >&2\n>         exit 1\n> fi\n>\n> mkdir \"$1\" && git init --quiet --bare \"$1\" && git-receive-pack \"$1\"\n> ^D\n> # chmod +x $_\n>\n> Then, on the client side, to push a new repo, you just do\n>\n> $ git push --receive-pack=git-receive-new-repo --all git@server:repo.git\n>\n> This will make `git push` to spawn not just `git receive-pack <dir>` as\n> it usually does but your wrapper, which would first create and\n> initialize a bare repository and then spawn `git receive-pack` on it\n> which would then communicate with the client side and receive\n> everything from it.\n>\n> You could then create a client-side wrapper script or a Git alias for\n> such \"creative pushing\", like this:\n>\n> $ git config --add --global alias.push-new-repo \\\n>   'push --receive-pack=git-receive-new-repo --all'\n>\n> So the whole client call is now reduced to\n>\n> $ git push-new-repo git@server:repo.git\n\nThanks, that's what I was going for.\n"},{"id":"209361","messageId":"511A2775.9050209@drmicha.warpmail.net","threadId":"32874","inReplyTo":"20130211162714.GB16402@sigill.intra.peff.net","subject":"Re: Pushing a git repository to a new server","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2013-02-12T11:28:53Z","receivedAt":"2013-02-12T11:28:53Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Jeff King venit, vidit, dixit 11.02.2013 17:27:\n> On Mon, Feb 11, 2013 at 02:57:51AM -0500, Ethan Reesor wrote:\n> \n>> On Mon, Feb 11, 2013 at 2:50 AM, Konstantin Khomoutov\n>> <kostix+git@007spb.ru> wrote:\n>>> What's wrong with\n>>> $ ssh myuser@remotehost 'mkdir /path/to/MyRepo.git; cd $_; git init --bare'\n>>> $ git push --all git@remotehost:MyOtherRepo.git\n>>> ?\n>>\n>> Nothing, I just wanted to make myself a command to do that for me.\n> \n> We talked about this a long time ago. One problem is that it's\n> inherently unportable, as the procedure to make a repo is potentially\n> different on every server (and certainly that is the case between a\n> regular user running stock git and something like GitHub or Google Code;\n> I imagine even gitolite has some special procedures for creating repos,\n> too).\n> \n> One proposal made in the previous discussion was to define a microformat\n> for repository administration commands. So that you could connect and\n> say \"git admin-create-repo /path/to/MyRepo.git\", and the server-provided\n> admin-create-repo command would take care of the details. Then stock git\n> could forward it to \"git init --bare\", GitHub could do the same and\n> create the necessary database records, etc.\n>\n> And once that standardized method was in place, it would be easy to add\n> a \"--create\" option to \"git push\" to request an \"admin-create-repo\"\n> before pushing.\n> \n> I still think that's a reasonable way forward, but nobody was interested\n> enough to start writing code for it.\n> \n> -Peff\n> \n\nI'm not sure providers like GitHub would fancy an interface which allows\nthe programmatic creation of repos (giving a new meaning to \"fork\nbomb\"). But I bet you know better ;-)\n\nAn alternative would be to teach git (the client) about repo types and\nhow to create them. After all, a repo URL \"ssh://host/path\" gives a\nclear indication that \"ssh host git init path\" will create a repo. I'm\nwondering whether it's more likely to convince providers (the server\nside) or more is gained by covering the simpler cases client-side (our\nside).\n\nMichael\n"},{"id":"209406","messageId":"20130212204210.GA25330@sigill.intra.peff.net","threadId":"32874","inReplyTo":"511A2775.9050209@drmicha.warpmail.net","subject":"Re: Pushing a git repository to a new server","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-02-12T20:42:10Z","receivedAt":"2013-02-12T20:42:10Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Feb 12, 2013 at 12:28:53PM +0100, Michael J Gruber wrote:\n\n> I'm not sure providers like GitHub would fancy an interface which allows\n> the programmatic creation of repos (giving a new meaning to \"fork\n> bomb\"). But I bet you know better ;-)\n\nYou can already do that:\n\n  http://developer.github.com/v3/repos/#create\n\nWe rate-limit API requests, and I imagine we might do something similar\nwith create-over-git. But that is exactly the kind of implementation\ndetail that can go into a custom create-repo script.\n\n> An alternative would be to teach git (the client) about repo types and\n> how to create them. After all, a repo URL \"ssh://host/path\" gives a\n> clear indication that \"ssh host git init path\" will create a repo.\n\nBut that's the point of a microformat. It _doesn't_ always work, because\nthe server may not allow arbitrary commands, or may have special\nrequirements on top of the \"init\". You can make the microformat be \"git\ninit path\", and servers can intercept calls to \"git init\" and translate\nthem into custom magic. But I think the world is a little simpler if we\ndefine a new service type (alongside git-upload-pack, git-receive-pack,\netc), and let clients request it. Then it's clear what the client is\ntrying to do, it's easy for servers to hook into it, we can request it\nover http, etc. And it can be extended over time to take more fields\n(like repo description, etc).\n\nI'm really not suggesting anything drastic. The wrapper case for ssh\nwould be as simple as a 3-line shell script which calls \"git init\" under\nthe hood, but it provides one level of indirection that makes\nreplacing/hooking it much simpler for servers. So the parts that are in\nstock git would not be much work (most of the work would be on _calling_\nit, but that is the same for adding a call to \"git init\").\n\nI think the main reason the idea hasn't gone anywhere is that nobody\nreally cares _that_ much. People just don't create repositories that\noften. I feel like this is one of those topics that comes up once a\nyear, and then nothing happens on it, because people just make their\nrepo manually and then stop caring about it.\n\nJust my two cents, of course. :)\n\n-Peff\n"},{"id":"209451","messageId":"511B4A04.1000104@drmicha.warpmail.net","threadId":"32874","inReplyTo":"20130212204210.GA25330@sigill.intra.peff.net","subject":"Re: Pushing a git repository to a new server","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2013-02-13T08:08:36Z","receivedAt":"2013-02-13T08:08:36Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Jeff King venit, vidit, dixit 12.02.2013 21:42:\n> On Tue, Feb 12, 2013 at 12:28:53PM +0100, Michael J Gruber wrote:\n> \n>> I'm not sure providers like GitHub would fancy an interface which allows\n>> the programmatic creation of repos (giving a new meaning to \"fork\n>> bomb\"). But I bet you know better ;-)\n> \n> You can already do that:\n> \n>   http://developer.github.com/v3/repos/#create\n\nNice.\n\nI knew you knew ;)\n\n> We rate-limit API requests, and I imagine we might do something similar\n> with create-over-git. But that is exactly the kind of implementation\n> detail that can go into a custom create-repo script.\n> \n>> An alternative would be to teach git (the client) about repo types and\n>> how to create them. After all, a repo URL \"ssh://host/path\" gives a\n>> clear indication that \"ssh host git init path\" will create a repo.\n> \n> But that's the point of a microformat. It _doesn't_ always work, because\n> the server may not allow arbitrary commands, or may have special\n> requirements on top of the \"init\". You can make the microformat be \"git\n> init path\", and servers can intercept calls to \"git init\" and translate\n> them into custom magic. But I think the world is a little simpler if we\n> define a new service type (alongside git-upload-pack, git-receive-pack,\n> etc), and let clients request it. Then it's clear what the client is\n> trying to do, it's easy for servers to hook into it, we can request it\n> over http, etc. And it can be extended over time to take more fields\n> (like repo description, etc).\n> \n> I'm really not suggesting anything drastic. The wrapper case for ssh\n> would be as simple as a 3-line shell script which calls \"git init\" under\n> the hood, but it provides one level of indirection that makes\n> replacing/hooking it much simpler for servers. So the parts that are in\n> stock git would not be much work (most of the work would be on _calling_\n> it, but that is the same for adding a call to \"git init\").\n> \n> I think the main reason the idea hasn't gone anywhere is that nobody\n> really cares _that_ much. People just don't create repositories that\n> often. I feel like this is one of those topics that comes up once a\n> year, and then nothing happens on it, because people just make their\n> repo manually and then stop caring about it.\n> \n> Just my two cents, of course. :)\n\nMost repos are probably created by a local \"git init\" or \"git clone\", or\nby clicking a button on a provider's web interface. The need for\ngit-create-repo seems to be restricted to:\n\n- \"command line folks\" who use a provider for it's hosting service and\ndon't fancy a web interface for repo creation\n\n- noobs who need to get their head wrapped around local, remote,\npush/pull 'n' stuff...\n\nFor the server side git-create-repo to take off we would probably need\ntwo things (besides the client support):\n\n- Implement and ship a git-create-repo which makes this work for git\nover ssh seamlessly. (Will take some to trickle down to servers in the\nwild.)\n\n- Get a large provider to offer this.\n\nGitosis/Gitolite are probably to follow easily. I'm beginning to like\nidea ;)\n\nMichael\n"}]}