# remote helper example with push/fetch capabilities 8 messages from 2014-12-15 to 2015-01-14. Participants: Klein W, Jonathan Nieder, Junio C Hamano, Andrew Mackenzie. Thread: https://gitlist.dev/t/38180 ## Klein W, 2014-12-15 20:17 Subject: remote helper example with push/fetch capabilities Message-ID: URL: https://gitlist.dev/e/CAPCWLt6kxoJJSWAcyH_kW071Md0vc4zeo41hCKBQHd-_pvUMXQ%40mail.gmail.com ``` Is there any example of a remote helper [0] with push and fetch capabilities? The git-remote-testgit.sh example [1] only has import/export capabilities. Also, what are the advantages and disadvantages of a remote helper with push/fetch capabilities vs a remote helper with import/export capabilities? Thanks. [0] https://www.kernel.org/pub/software/scm/git/docs/git-remote-helpers.html [1] https://github.com/git/git/blob/master/git-remote-testgit.sh ``` ## Jonathan Nieder, 2014-12-15 20:47 Subject: Re: remote helper example with push/fetch capabilities Message-ID: <20141215204740.GI29365@google.com> URL: https://gitlist.dev/e/20141215204740.GI29365%40google.com In-Reply-To: ``` Hi, Klein W wrote: > Is there any example of a remote helper [0] with push and fetch capabilities? Sure --- see remote-curl.c. There's also the "connect" capability. builtin/remote-ext.c and builtin/remote-fd.c are examples using that one. [...] > Also, what are the advantages and disadvantages of a remote helper > with push/fetch capabilities vs a remote helper with import/export > capabilities? It mainly has to do with what it is convenient for your helper to produce. If the helper would find it more convenient to write native git objects (for example because the remote server speaks a git-specific protocol, as in the case of remote-curl.c) then the "fetch" capability will be more convenient. If the helper wants to make a batch of new objects then a fast-import stream can be a convenient way to do this and the "import" capability takes care of running fast-import to take care of that. Thanks and hope that helps, Jonathan ``` ## Klein W, 2014-12-15 21:42 Subject: Re: remote helper example with push/fetch capabilities Message-ID: URL: https://gitlist.dev/e/CAPCWLt4%3DoYTPFXktCj8CgqNncaO2%3DsbwZcPOVa%2Ba5wgt7HPCUQ%40mail.gmail.com In-Reply-To: <20141215204740.GI29365@google.com> ``` On Mon, Dec 15, 2014 at 3:47 PM, Jonathan Nieder wrote: > Sure --- see remote-curl.c. > > There's also the "connect" capability. builtin/remote-ext.c and > builtin/remote-fd.c are examples using that one. Thanks. >> Also, what are the advantages and disadvantages of a remote helper >> with push/fetch capabilities vs a remote helper with import/export >> capabilities? > > It mainly has to do with what it is convenient for your helper to > produce. If the helper would find it more convenient to write native > git objects (for example because the remote server speaks a > git-specific protocol, as in the case of remote-curl.c) then the > "fetch" capability will be more convenient. If the helper wants to > make a batch of new objects then a fast-import stream can be a > convenient way to do this and the "import" capability takes care of > running fast-import to take care of that. I'm trying to write a remote helper for hosting git remotes on Amazon S3. Do you have any intuition about which capabilities would work best for this case? Thanks. ``` ## Jonathan Nieder, 2014-12-15 21:44 Subject: Re: remote helper example with push/fetch capabilities Message-ID: <20141215214459.GJ29365@google.com> URL: https://gitlist.dev/e/20141215214459.GJ29365%40google.com In-Reply-To: ``` Klein W wrote: > On Mon, Dec 15, 2014 at 3:47 PM, Jonathan Nieder wrote: >> It mainly has to do with what it is convenient for your helper to >> produce. If the helper would find it more convenient to write native >> git objects (for example because the remote server speaks a >> git-specific protocol, as in the case of remote-curl.c) then the >> "fetch" capability will be more convenient. If the helper wants to >> make a batch of new objects then a fast-import stream can be a >> convenient way to do this and the "import" capability takes care of >> running fast-import to take care of that. > > I'm trying to write a remote helper for hosting git remotes on Amazon > S3. Do you have any intuition about which capabilities would work > best for this case? fetch/push. I'd suggest looking at the "dumb" HTTP code (fetch_dumb, push_dav) in remote-curl.c to start. Thanks, Jonathan ``` ## Junio C Hamano, 2014-12-15 21:47 Subject: Re: remote helper example with push/fetch capabilities Message-ID: URL: https://gitlist.dev/e/xmqqegs0d1qv.fsf%40gitster.dls.corp.google.com In-Reply-To: ``` Klein W writes: > Is there any example of a remote helper [0] with push and fetch capabilities? > > The git-remote-testgit.sh example [1] only has import/export capabilities. > > Also, what are the advantages and disadvantages of a remote helper > with push/fetch capabilities vs a remote helper with import/export > capabilities? A helper with push/fetch capabilities is responsible for (and more importantly, "in control of") packdata creation, while helpers that use import/export interface rely on fast-import, which is quite dumb when it comes to storage efficiency of the resulting repository. The former might be more preferrable from the efficiency point of view. BUT. Unless your foreign SCM is Git itself, however, it is not practical to write your own pack data generator correctly and efficiently to support push/fetch capabilities anyway, so the choice is often made not because of "advantages vs disadvantages" but because of what you can write in practice. ``` ## Klein W, 2014-12-15 22:41 Subject: Re: remote helper example with push/fetch capabilities Message-ID: URL: https://gitlist.dev/e/CAPCWLt5pW6kwP4FCsJqG%2Bo5d%3Dp_WgKMSXqaANY7vSe6Bh7tMxQ%40mail.gmail.com In-Reply-To: <20141215214459.GJ29365@google.com> ``` On Mon, Dec 15, 2014 at 4:44 PM, Jonathan Nieder wrote: >> I'm trying to write a remote helper for hosting git remotes on Amazon >> S3. Do you have any intuition about which capabilities would work >> best for this case? > > fetch/push. I'd suggest looking at the "dumb" HTTP code (fetch_dumb, > push_dav) in remote-curl.c to start. This seems like a good starting point, but are there any simpler examples that are not clouded by WebDAV/Curl stuff? Perhaps a remote helper that reads/writes to the filesystem directly? ``` ## Klein W, 2014-12-18 21:46 Subject: Re: remote helper example with push/fetch capabilities Message-ID: URL: https://gitlist.dev/e/CAPCWLt7%2BdfcubjEaB%3DeacNK39CA7Q9K6Mg2FM%3D_3qQaGBOR2%2BA%40mail.gmail.com In-Reply-To: ``` Would someone be willing to extend the git-remote-testgit.sh example [1] with push and fetch capabilities? I am not familiar enough to do it myself. Thanks [1] https://github.com/git/git/blob/master/git-remote-testgit.sh ``` ## Andrew Mackenzie, 2015-01-14 21:57 Subject: Re: remote helper example with push/fetch capabilities Message-ID: URL: https://gitlist.dev/e/loom.20150114T225554-824%40post.gmane.org In-Reply-To: ``` I'm interested in doing something very similar (not to S3, but to a customer CMS repository), but have found it very difficult to find any documentation or good example code on git-remote-helpers. If you can share any learnings or references I would really appreciate it! thanks, Andrew ```