{"thread":{"id":"9391","subject":"GIT push to sftp (feature request)","startedAt":"2007-08-05T09:05:40Z","lastAt":"2007-08-07T21:50:34Z","messageCount":8,"participants":["pavlix","Johannes Schindelin","Martin Langhoff","Matthieu Moy","Jakub Narebski","Jan Hudec"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"49847","messageId":"200708051105.44376.pavlix@pavlix.net","threadId":"9391","inReplyTo":null,"subject":"GIT push to sftp (feature request)","fromName":"pavlix","fromEmail":"pavlix@pavlix.net","sentAt":"2007-08-05T09:05:40Z","receivedAt":"2007-08-05T09:05:40Z","isPatch":false,"sender":{"key":"pavlix@pavlix.net","avatar":null},"body":"Hello\n\nIt would be very useful if git supported sftp urls to push to remote \nrepositories.\n\nThe use cases are obvious... you would only need ssh run on the other side, \nwhich is usually available. One cannot or doesn't want to install git on \nevery machine where he wants his remote repository.\n\nGit would also have to be able to create a remote repository (maybe an option \nto push?).\n\nDid I miss something?\n\nPavel Šimerda\n\nP.S.: I am switching from bazaar-ng which can save so sftp and other protocols\n\n-- \n\nWeb: http://www.pavlix.net/\nJabber & E-mail: pavlix@pavlix.net\n\n"},{"id":"49886","messageId":"Pine.LNX.4.64.0708051436120.14781@racer.site","threadId":"9391","inReplyTo":"200708051105.44376.pavlix@pavlix.net","subject":"Re: GIT push to sftp (feature request)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-08-05T13:38:11Z","receivedAt":"2007-08-05T13:38:11Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sun, 5 Aug 2007, pavlix wrote:\n\n> Git would also have to be able to create a remote repository (maybe an option \n> to push?).\n> \n> Did I miss something?\n\nYes.\n\nFirst, we do not allow remote repository initialising yet.\n\nSecond, if you do not have git installed on the remote host, you probably \nwant to serve via http. This is a very suboptimal transport, as it cannot \nrepack the contents.\n\nAnd if you use such a suboptimal transport, people will blame _git_ for \nbeing slow, even if you made it slow deliberately.\n\nCiao,\nDscho\n"},{"id":"49943","messageId":"46a038f90708051412p722aa906v73e986a805f1558b@mail.gmail.com","threadId":"9391","inReplyTo":"200708051105.44376.pavlix@pavlix.net","subject":"Re: GIT push to sftp (feature request)","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2007-08-05T21:12:41Z","receivedAt":"2007-08-05T21:12:41Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On 8/5/07, pavlix wrote:\n> Did I miss something?\n\n\nUnfortunately, git does not \"push\" over protocols that cannot execute\ngit on the remote server. We call them \"dumb protocols\" and if you\nsearch this list for that name, you'll find lots.\n\nGit tries to be smart in at least 2 ways that don't work with dump\nprotocols: it works locklessly (yet it performs atomic updates) and it\nsends only the objects needed over the wire (saving a lot of\nbandwidth).\n\nUsing dumb protocols it's impossible to do either. And these days it's\nnot that hard to setup git (or any other binary) to execute at the\nremote end.\n\nBazaar-NG and others do support dumb protocols, and (I think) they do\nit by using one big lock over the repo. But the lock is not safe, and\nthings can (and do) go wrong with weak locking schemes.\n\ngit used to support rsync -- but I don't think that works anymore for\npushes. Other than git over ssh, perhaps you can try the apache module\nthat implements git over http?\n\nhope that helps,\n\n\n\nmartin\n"},{"id":"49948","messageId":"vpqir7t8vy0.fsf@bauges.imag.fr","threadId":"9391","inReplyTo":"46a038f90708051412p722aa906v73e986a805f1558b@mail.gmail.com","subject":"Re: GIT push to sftp (feature request)","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-08-05T22:20:55Z","receivedAt":"2007-08-05T22:20:55Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"\"Martin Langhoff\" writes:\n\n> Git tries to be smart in at least 2 ways that don't work with dump\n> protocols: it works locklessly (yet it performs atomic updates) and it\n> sends only the objects needed over the wire (saving a lot of\n> bandwidth).\n>\n> Using dumb protocols it's impossible to do either.\n\nThat's not exactly true. You can't be as efficient with dumb protocols\nthan you are with a dedicated protocol (something with some\nintelligence on both sides), but at least the second point you mention\ncan be achieved with a dumb protocol, and bzr is a proof of existance.\nTo read over HTTP, it uses ranges request, and to push over\nftp/sftp/webdav, it appends new data to existing files (its ancestor,\nGNU Arch, also had a way to be network-efficient on dumb protocols).\n\nRegarding atomic and lock-less updates, I believe this is\nimplementable too as soon as you have an atomit \"rename\" in the\nprotocol. But here, bzr isn't a proof of existance, it does locking.\n\n(BTW, about bzr, it also has a dedicated server now)\n\n-- \nMatthieu\n"},{"id":"49958","messageId":"46a038f90708051700t7a758f8fwdf7c63c8aeef9ee8@mail.gmail.com","threadId":"9391","inReplyTo":"vpqir7t8vy0.fsf@bauges.imag.fr","subject":"Re: GIT push to sftp (feature request)","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2007-08-06T00:00:03Z","receivedAt":"2007-08-06T00:00:03Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On 8/6/07, Matthieu Moy wrote:\n> \"Martin Langhoff\" writes:\n>\n> > Git tries to be smart in at least 2 ways that don't work with dump\n> > protocols: it works locklessly (yet it performs atomic updates) and it\n> > sends only the objects needed over the wire (saving a lot of\n> > bandwidth).\n> >\n> > Using dumb protocols it's impossible to do either.\n>\n> That's not exactly true. You can't be as efficient with dumb protocols\n\nYou are right -- I should have said: it's pretty hard, and we haven't\nput the effort ;-)\n\nthere's been discussion recently of having info in the pack index that\nwould support http range requests.\n\n> (its ancestor,\n> GNU Arch, also had a way to be network-efficient on dumb protocols).\n\nDo I remember your name from gnuarch-users? -- that Arch/tla was never\nparticularly efficient, and fetches of large updates were slow and\npainful. Surely it was efficient on paper though :-p\n\n> Regarding atomic and lock-less updates, I believe this is\n> implementable too as soon as you have an atomit \"rename\" in the\n> protocol. But here, bzr isn't a proof of existance, it does locking.\n\nAnd I should have said - minimal locking rather than no locking\n\nTo update it safely, you need to open with a lock, read to ensure the\nsha1 is what you think it is, write the new sha1, close. A rename is\nstill subject to race conditions.\n\nIMVHO it would be good to have a way to push over sftp even it it is\nslow, unsafe and full of big blinking warnings. git itself is sane\nenough that the client side won't get corrupted or lose data if there\nis a race condition on the server side.\n\nGiven a brief delay, the client can probably check - post push - that\nthe operation wasn't clobbered by a race condition. Of course, this\n*is* sticks-and-bubblegum approach on the server side. But a solid\nclient repo makes almost any server-side disaster recoverable.\n\ncheers,\n\n\n\nmartin\n"},{"id":"49961","messageId":"f95p47$607$1@sea.gmane.org","threadId":"9391","inReplyTo":"vpqir7t8vy0.fsf@bauges.imag.fr","subject":"Re: GIT push to sftp (feature request)","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2007-08-06T00:14:02Z","receivedAt":"2007-08-06T00:14:02Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Matthieu Moy wrote:\n\n> \"Martin Langhoff\" writes:\n> \n>> Git tries to be smart in at least 2 ways that don't work with dump\n>> protocols: it works locklessly (yet it performs atomic updates) and it\n>> sends only the objects needed over the wire (saving a lot of\n>> bandwidth).\n>>\n>> Using dumb protocols it's impossible to do either.\n\nBut git _can_ push over http protocol (with WebDAV), and http is a dumb\nprotocol, and over rsync (although it is deprecated).\n \n> That's not exactly true. You can't be as efficient with dumb protocols\n> than you are with a dedicated protocol (something with some\n> intelligence on both sides), but at least the second point you mention\n> can be achieved with a dumb protocol, and bzr is a proof of existance.\n> To read over HTTP, it uses ranges request, and to push over\n> ftp/sftp/webdav, it appends new data to existing files (its ancestor,\n> GNU Arch, also had a way to be network-efficient on dumb protocols).\n\nIf I understand correctly to read (fetch) over http and other dumb\nprotocols (like ftp), git uses two indices .git/info/refs\nand .git/objects/info/packs which must be present on the server serving\nhttp protocol (see git-update-server-info) to calculate which packs\nto get, and I think it always downloads whole packs, but I'm not sure...\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"49998","messageId":"vpq8x8pjawj.fsf@bauges.imag.fr","threadId":"9391","inReplyTo":"46a038f90708051700t7a758f8fwdf7c63c8aeef9ee8@mail.gmail.com","subject":"Re: GIT push to sftp (feature request)","fromName":"Matthieu Moy","fromEmail":"matthieu.moy@imag.fr","sentAt":"2007-08-06T08:59:56Z","receivedAt":"2007-08-06T08:59:56Z","isPatch":false,"sender":{"key":"git@matthieu-moy.fr","avatar":"https://avatars.githubusercontent.com/u/14709?v=4"},"body":"\"Martin Langhoff\" writes:\n\n> On 8/6/07, Matthieu Moy wrote:\n>> > Using dumb protocols it's impossible to do either.\n>>\n>> That's not exactly true. You can't be as efficient with dumb protocols\n>\n> You are right -- I should have said: it's pretty hard, and we haven't\n> put the effort ;-)\n\nYes, that seems more accurate.\n\n>> (its ancestor,\n>> GNU Arch, also had a way to be network-efficient on dumb protocols).\n>\n> Do I remember your name from gnuarch-users?\n\nPossibly so, yes. I also remembered yours from the old good time where\npeople started explaining why they unsubscribed the list and migrated\nto something better ;-).\n\n> -- that Arch/tla was never particularly efficient, and fetches of\n> large updates were slow and painful. Surely it was efficient on\n> paper though :-p\n\nIt was actually efficient in terms of bandwidth. You downloaded only\nthe needed pieces (this has to do with the fact that the original\nauthor wrote it at a time when he had only a slow modem connection).\nBut badly pipelined, and local operations were slow, so the result was\nobviously _very_ far from what git can do.\n\n-- \nMatthieu\n"},{"id":"50147","messageId":"20070807215034.GB3804@efreet.light.src","threadId":"9391","inReplyTo":"vpqir7t8vy0.fsf@bauges.imag.fr","subject":"Re: GIT push to sftp (feature request)","fromName":"Jan Hudec","fromEmail":"bulb@ucw.cz","sentAt":"2007-08-07T21:50:34Z","receivedAt":"2007-08-07T21:50:34Z","isPatch":false,"sender":{"key":"bulb@ucw.cz","avatar":null},"body":"On Mon, Aug 06, 2007 at 00:20:55 +0200, Matthieu Moy wrote:\n> \"Martin Langhoff\" writes:\n> \n> > Git tries to be smart in at least 2 ways that don't work with dump\n> > protocols: it works locklessly (yet it performs atomic updates) and it\n> > sends only the objects needed over the wire (saving a lot of\n> > bandwidth).\n> >\n> > Using dumb protocols it's impossible to do either.\n> \n> That's not exactly true. You can't be as efficient with dumb protocols\n> than you are with a dedicated protocol (something with some\n> intelligence on both sides), but at least the second point you mention\n> can be achieved with a dumb protocol, and bzr is a proof of existance.\n> To read over HTTP, it uses ranges request, and to push over\n> ftp/sftp/webdav, it appends new data to existing files (its ancestor,\n> GNU Arch, also had a way to be network-efficient on dumb protocols).\n\nI believe bzr locks are not completely safe in a sense that breaking a lock\ndoes not cause the operation to immediately abort. GNU Arch ones did, but\nit's specific data layout was part of a reason why it worked (it wrote the\ndata to a directory, so removing that would abort the operation).\n\n> Regarding atomic and lock-less updates, I believe this is\n> implementable too as soon as you have an atomit \"rename\" in the\n> protocol. But here, bzr isn't a proof of existance, it does locking.\n\nActually rename or link is necessary for atomic updates, lockless or lockful.\n\nSlight problem with it is, that unix (and similar) systems allow overwriting\nanother file on rename (and do so atomically in a sense the destination\nalways exists), while windooze fail if the target exists. Most network\nprotocols don't specify overwriting and simply do whatever the underlying\nsystem does. GNU Arch solved this by renaming directories, which are not\noverwriten under any system.\n\n-- \n\t\t\t\t\t\t Jan 'Bulb' Hudec \n"}]}