Re: [PATCH 10/19] fast-import: introduce 'done' command

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Jun 8, 2011, 20:03 UTC
Message-ID: <20110608200345.GJ27715@elie>
In-Reply-To: <1307558930-16074-11-git-send-email-srabbelier@gmail.com>

Sverre Rabbelier wrote:

Show 6 quoted lines

> Add a 'done' command that causes fast-import to stop reading from the
> stream and exit.
>
> If the new --done command line flag was passed on the command line
> (or a "feature done" declaration included at the start of the stream),
> make the 'done' command mandatory.

Hoorah!  Thanks for reviving this.

Show 6 quoted lines

> Another possible application of the 'done' command would to be allow a
> fast-import stream that is only a small part of a larger encapsulating
> stream to be easily parsed, leaving the file offset after the "done\n"
> so the other application can pick up from there.  This patch does not
> teach fast-import to do that --- fast-import still uses buffered input
> (stdio).

Possible approaches, two ugly and one sane:

 A) use setvbuf() to make input line-buffered.  Unfortunately, as
    POSIX tells us:

	Applications should note that many implementations only provide
	line buffering on input from terminal devices.

    So making it actually work involves pseudo-tty craziness.  Not a
    good idea.

 B) leave some room for buffering after the "done" command and make
    fast-import use setvbuf() to control the size of the stdio buffer.
    This is disgusting but it probably works.

 C) make sure the upstream of the pipe does not write anything after
    "done\n" until the "done\n" command has been read.  In other
    words:

	transport-helper>	done
	remote-helper>   	yep, I'm done
	transport-helper>	ah, now that you mention that, here are
	                 	a few more things you could do...

As long as there is nothing after the "done\n" in the pipe buffer when fast-import reads "done\n", it won't read too far.

This is all made complicated to debug by stdio's habit of closing streams at exit time, flushing streams at close time, and repositioning the file offset of seekable input (but not pipes) at flush time.

Hope that helps.