threads / discuss / 38180

remote helper example with push/fetch capabilities

Subject: remote helper example with push/fetch capabilities

## tl;dr

8 messages between Dec 15, 2014 and Jan 14, 2015.

replies: 7people: 4as markdown or json

Klein W· Dec 15, 2014, 20:17 UTC · lore
Is there any example of a remote helper [0] with push and fetch capabilities?
The git-remote-testgit.sh example [1] only has import/export capabilities.

Also, what are the advantages and disadvantages of a remote helper with push/fetch capabilities vs a remote helper with import/export capabilities?

Thanks.

[0] https://www.kernel.org/pub/software/scm/git/docs/git-remote-helpers.html [1] https://github.com/git/git/blob/master/git-remote-testgit.sh

Jonathan Nieder· Dec 15, 2014, 20:47 UTC · re: Klein W · lore

Re: remote helper example with push/fetch capabilities

Hi,
Klein W wrote:
> Is there any example of a remote helper [0] with push and fetch capabilities?
Sure --- see remote-curl.c.

There's also the "connect" capability. builtin/remote-ext.c and builtin/remote-fd.c are examples using that one.

[...]
> Also, what are the advantages and disadvantages of a remote helper
> with push/fetch capabilities vs a remote helper with import/export
> capabilities?

It mainly has to do with what it is convenient for your helper to produce. If the helper would find it more convenient to write native git objects (for example because the remote server speaks a git-specific protocol, as in the case of remote-curl.c) then the "fetch" capability will be more convenient. If the helper wants to make a batch of new objects then a fast-import stream can be a convenient way to do this and the "import" capability takes care of running fast-import to take care of that.

Thanks and hope that helps, Jonathan

Klein W· Dec 15, 2014, 21:42 UTC · re: Jonathan Nieder · lore

Re: remote helper example with push/fetch capabilities

On Mon, Dec 15, 2014 at 3:47 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Sure --- see remote-curl.c.
>
> There's also the "connect" capability.  builtin/remote-ext.c and
> builtin/remote-fd.c are examples using that one.
Thanks.
Show 12 quoted lines
>> Also, what are the advantages and disadvantages of a remote helper
>> with push/fetch capabilities vs a remote helper with import/export
>> capabilities?
>
> It mainly has to do with what it is convenient for your helper to
> produce.  If the helper would find it more convenient to write native
> git objects (for example because the remote server speaks a
> git-specific protocol, as in the case of remote-curl.c) then the
> "fetch" capability will be more convenient.  If the helper wants to
> make a batch of new objects then a fast-import stream can be a
> convenient way to do this and the "import" capability takes care of
> running fast-import to take care of that.

I'm trying to write a remote helper for hosting git remotes on Amazon S3. Do you have any intuition about which capabilities would work best for this case?

Thanks.
Jonathan Nieder· Dec 15, 2014, 21:44 UTC · re: Klein W · lore

Re: remote helper example with push/fetch capabilities

Klein W wrote:
> On Mon, Dec 15, 2014 at 3:47 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
Show 12 quoted lines
>> It mainly has to do with what it is convenient for your helper to
>> produce.  If the helper would find it more convenient to write native
>> git objects (for example because the remote server speaks a
>> git-specific protocol, as in the case of remote-curl.c) then the
>> "fetch" capability will be more convenient.  If the helper wants to
>> make a batch of new objects then a fast-import stream can be a
>> convenient way to do this and the "import" capability takes care of
>> running fast-import to take care of that.
>
> I'm trying to write a remote helper for hosting git remotes on Amazon
> S3.  Do you have any intuition about which capabilities would work
> best for this case?

fetch/push. I'd suggest looking at the "dumb" HTTP code (fetch_dumb, push_dav) in remote-curl.c to start.

Thanks, Jonathan

Klein W· Dec 15, 2014, 22:41 UTC · re: Jonathan Nieder · lore

Re: remote helper example with push/fetch capabilities

On Mon, Dec 15, 2014 at 4:44 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
Show 6 quoted lines
>> I'm trying to write a remote helper for hosting git remotes on Amazon
>> S3.  Do you have any intuition about which capabilities would work
>> best for this case?
>
> fetch/push.  I'd suggest looking at the "dumb" HTTP code (fetch_dumb,
> push_dav) in remote-curl.c to start.

This seems like a good starting point, but are there any simpler examples that are not clouded by WebDAV/Curl stuff? Perhaps a remote helper that reads/writes to the filesystem directly?

Klein W· Dec 18, 2014, 21:46 UTC · re: Klein W · lore

Re: remote helper example with push/fetch capabilities

Would someone be willing to extend the git-remote-testgit.sh example [1] with push and fetch capabilities? I am not familiar enough to do it myself.

Thanks
[1] https://github.com/git/git/blob/master/git-remote-testgit.sh
Junio C Hamano· Dec 15, 2014, 21:47 UTC · re: Klein W · lore

Re: remote helper example with push/fetch capabilities

Klein W <wineklein@gmail.com> writes:
Show 7 quoted lines
> Is there any example of a remote helper [0] with push and fetch capabilities?
>
> The git-remote-testgit.sh example [1] only has import/export capabilities.
>
> Also, what are the advantages and disadvantages of a remote helper
> with push/fetch capabilities vs a remote helper with import/export
> capabilities?

A helper with push/fetch capabilities is responsible for (and more importantly, "in control of") packdata creation, while helpers that use import/export interface rely on fast-import, which is quite dumb when it comes to storage efficiency of the resulting repository. The former might be more preferrable from the efficiency point of view.

BUT.

Unless your foreign SCM is Git itself, however, it is not practical to write your own pack data generator correctly and efficiently to support push/fetch capabilities anyway, so the choice is often made not because of "advantages vs disadvantages" but because of what you can write in practice.

Andrew Mackenzie· Jan 14, 2015, 21:57 UTC · re: Junio C Hamano · lore

Re: remote helper example with push/fetch capabilities

I'm interested in doing something very similar (not to S3, but to a customer CMS repository), but have found it very difficult to find any documentation or good example code on git-remote-helpers.

If you can share any learnings or references I would really appreciate it!
thanks, Andrew

← back to recent threads