Re: [RFC 1/4 v2] Implement a basic remote helper for svn in C.
- From
Florian Achleitner <florian.achleitner.2.6.31@gmail.com>
- Date
- Aug 12, 2012, 10:06 UTC
- Message-ID
- <1636924.tANzCnKezB@flobuntu>
- In-Reply-To
- <20120801194247.GE24357@copier>
Hi,
back to the pipe-topic.
On Wednesday 01 August 2012 12:42:48 Jonathan Nieder wrote:
Show 22 quoted lines
> Hi again, > > Florian Achleitner wrote: > > When the first line arrives at the remote-helper, it starts importing one > > line at a time, leaving the remaining lines in the pipe. > > For importing it requires the data from fast-import, which would be mixed > > with import lines or queued at the end of them. > > Oh, good catch. > > The way it's supposed to work is that in a bidi-import, the remote > helper reads in the entire list of refs to be imported and only once > the newline indicating that that list is over arrives starts writing > its fast-import stream. We could make this more obvious by not > spawning fast-import until immediately before writing that newline. > > This needs to be clearly documented in the git-remote-helpers(1) page > if the bidi-import command is introduced. > > If a remote helper writes commands for fast-import before that newline > comes, that is a bug in the remote helper, plain and simple. It might > be fun to diagnose this problem:
This would require all existing remote helpers that use 'import' to be ported to the new concept, right? Probably there is no other..
Show 29 quoted lines
>
> static void pipe_drained_or_die(int fd, const char *msg)
> {
> char buf[1];
> int flags = fcntl(fd, F_GETFL);
> if (flags < 0)
> die_errno("cannot get pipe flags");
> if (fcntl(fd, F_SETFL, flags | O_NONBLOCK))
> die_errno("cannot set up non-blocking pipe read");
> if (read(fd, buf, 1) > 0)
> die("%s", msg);
> if (fcntl(fd, F_SETFL, flags))
> die_errno("cannot restore pipe flags");
> }
> ...
>
> for (i = 0; i < nr_heads; i++) {
> write "import %s\n", to_fetch[i]->name;
> }
>
> if (getenv("GIT_REMOTE_HELPERS_SLOW_SANITY_CHECK"))
> sleep(1);
>
> pipe_drained_or_die("unexpected output from remote helper before
> fast-import launch");
>
> if (get_importer(transport, &fastimport))
> die("couldn't run fast-import");
> write_constant(data->helper->in, "\n");I still don't believe that sharing the input pipe of the remote helper is worth the hazzle. It still requires an additional pipe to be setup, the one from fast-import to the remote-helper, sharing one FD at the remote helper. It still requires more than just stdin, stdout, stderr.
I would suggest to use a fifo. It can be openend independently, after forking and on windows they have named pipes with similar semantics, so I think this could be easily ported. I would suggest the following changes: - add a capability to the remote helper 'bidi-import', or 'bidi-pipe'. This signals that the remote helper requires data from fast-import.
- add a command 'bidi-import', or 'bidi-pipe' that is tells the remote helper which filename the fifo is at, so that it can open it and read it when it handles 'import' commands.
- transport-helper.c creates the fifo on demand, i.e. on seeing the capability, in the gitdir or in /tmp.
- fast-import gets the name of the fifo as a command-line argument. The alternative would be to add a command, but that's not allowed, because it changes the stream semantics. Another alternative would be to use the existing --cat-pipe-fd argument. But that requires to open the fifo before execing fast-import and makes us dependent on the posix model of forking and inheriting file descriptors, while opening a fifo in fast-import would not.