GIT push to sftp (feature request)

Hello

It would be very useful if git supported sftp urls to push to remote repositories.

The use cases are obvious... you would only need ssh run on the other side, which is usually available. One cannot or doesn't want to install git on every machine where he wants his remote repository.

Git would also have to be able to create a remote repository (maybe an option to push?).

Did I miss something?

Pavel Šimerda

P.S.: I am switching from bazaar-ng which can save so sftp and other protocols

Hi,

On Sun, 5 Aug 2007, pavlix wrote:

Yes.

First, we do not allow remote repository initialising yet.

Second, if you do not have git installed on the remote host, you probably want to serve via http. This is a very suboptimal transport, as it cannot repack the contents.

And if you use such a suboptimal transport, people will blame _git_ for being slow, even if you made it slow deliberately.

Ciao, Dscho

On 8/5/07, pavlix <pavlix@pavlix.net> wrote:

Unfortunately, git does not "push" over protocols that cannot execute git on the remote server. We call them "dumb protocols" and if you search this list for that name, you'll find lots.

Git tries to be smart in at least 2 ways that don't work with dump protocols: it works locklessly (yet it performs atomic updates) and it sends only the objects needed over the wire (saving a lot of bandwidth).

Using dumb protocols it's impossible to do either. And these days it's not that hard to setup git (or any other binary) to execute at the remote end.

Bazaar-NG and others do support dumb protocols, and (I think) they do it by using one big lock over the repo. But the lock is not safe, and things can (and do) go wrong with weak locking schemes.

git used to support rsync -- but I don't think that works anymore for pushes. Other than git over ssh, perhaps you can try the apache module that implements git over http?

hope that helps,

martin

"Martin Langhoff" <martin.langhoff@gmail.com> writes:

That's not exactly true. You can't be as efficient with dumb protocols than you are with a dedicated protocol (something with some intelligence on both sides), but at least the second point you mention can be achieved with a dumb protocol, and bzr is a proof of existance. To read over HTTP, it uses ranges request, and to push over ftp/sftp/webdav, it appends new data to existing files (its ancestor, GNU Arch, also had a way to be network-efficient on dumb protocols).

Regarding atomic and lock-less updates, I believe this is implementable too as soon as you have an atomit "rename" in the protocol. But here, bzr isn't a proof of existance, it does locking.

(BTW, about bzr, it also has a dedicated server now)

On 8/6/07, Matthieu Moy <Matthieu.Moy@imag.fr> wrote:

You are right -- I should have said: it's pretty hard, and we haven't put the effort ;-)

there's been discussion recently of having info in the pack index that would support http range requests.

Do I remember your name from gnuarch-users? -- that Arch/tla was never particularly efficient, and fetches of large updates were slow and painful. Surely it was efficient on paper though :-p

And I should have said - minimal locking rather than no locking

To update it safely, you need to open with a lock, read to ensure the sha1 is what you think it is, write the new sha1, close. A rename is still subject to race conditions.

IMVHO it would be good to have a way to push over sftp even it it is slow, unsafe and full of big blinking warnings. git itself is sane enough that the client side won't get corrupted or lose data if there is a race condition on the server side.

Given a brief delay, the client can probably check - post push - that the operation wasn't clobbered by a race condition. Of course, this *is* sticks-and-bubblegum approach on the server side. But a solid client repo makes almost any server-side disaster recoverable.

cheers,

martin

"Martin Langhoff" <martin.langhoff@gmail.com> writes:

Yes, that seems more accurate.

Possibly so, yes. I also remembered yours from the old good time where people started explaining why they unsubscribed the list and migrated to something better ;-).

It was actually efficient in terms of bandwidth. You downloaded only the needed pieces (this has to do with the fact that the original author wrote it at a time when he had only a slow modem connection). But badly pipelined, and local operations were slow, so the result was obviously _very_ far from what git can do.

Matthieu Moy wrote:

But git _can_ push over http protocol (with WebDAV), and http is a dumb
protocol, and over rsync (although it is deprecated).
 

If I understand correctly to read (fetch) over http and other dumb protocols (like ftp), git uses two indices .git/info/refs and .git/objects/info/packs which must be present on the server serving http protocol (see git-update-server-info) to calculate which packs to get, and I think it always downloads whole packs, but I'm not sure...

On Mon, Aug 06, 2007 at 00:20:55 +0200, Matthieu Moy wrote:

I believe bzr locks are not completely safe in a sense that breaking a lock does not cause the operation to immediately abort. GNU Arch ones did, but it's specific data layout was part of a reason why it worked (it wrote the data to a directory, so removing that would abort the operation).

Actually rename or link is necessary for atomic updates, lockless or lockful.

Slight problem with it is, that unix (and similar) systems allow overwriting another file on rename (and do so atomically in a sense the destination always exists), while windooze fail if the target exists. Most network protocols don't specify overwriting and simply do whatever the underlying system does. GNU Arch solved this by renaming directories, which are not overwriten under any system.