Re: [PATCH] git-rev-list: give better diagnostic for failed write

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Jun 26, 2007, 17:32 UTC
Message-ID: <alpine.LFD.0.98.0706261024210.8675@woody.linux-foundation.org>
In-Reply-To: <20070626171127.GA28810@thunk.org>

On Tue, 26 Jun 2007, Theodore Tso wrote:

Show 9 quoted lines

> 
> Is there something obviously wrong with doing something like this?
> 
> 	if ((fstat(fileno(stdout), &st) < 0) &&
> 	    !S_ISREG(st.st_mode))
> 		setbuf(stdout, NULL);
> 
> This would change stdout to use completely unbuffered I/O we're not
> sending the output to a file.

Well, we might as well keep it line-buffered, so I'd use setvbuf(_IOLBF) instead. Totally unbuffered is bad, since we often do printf's in smaller chunks.

But we actually _do_ want fully buffered from a performance angle. Especially for the big stuff, which is usually the *diffs*, not the commit messages. Not so much an issue with git-rev-list, but with "git log -p" you would normally not want it line-buffered, and it's actually much nicer to let it be fully buffered and then do a flush at the end.

Even pipes are often used for "throughput" stuff if you end up doing some post-processing (ie "git log -p | gather-statistics"), and yes, I actually do things like that - it's nice for things like looking at how many lines have been added during the last release cycle:

	git log -p v2.6.21.. | grep '^+' | wc -l

and I'd really like thigns like that to be close to optimal.

How much the system call overhead is, I don't know, though. So it might be worth testing out. Under Linux, you'll probably have a fairly hard time seeing any difference, but under other OS's you have both system call latency issues *and* possible scheduling issues (ie the above kind of pipeline can act very differently from a scheduling standpoint if you send lots of small things rather than buffer things a bit on the generating side)

		Linus