Re: [RFC \ WISH] Add -o option to git-rev-list

From: Linus Torvalds <torvalds@osdl.org>
Date: Dec 10, 2006, 18:16 UTC
Message-ID: <Pine.LNX.4.64.0612101009040.12500@woody.osdl.org>
In-Reply-To: <e5bfff550612100338ye2ca2a0u1c8f29bbc59c5431@mail.gmail.com>

On Sun, 10 Dec 2006, Marco Costalba wrote:

Show 13 quoted lines

> 
> The averaged results on linux tree (about 30MB of data) and CPU set at
> 1.2GHz are:
> 
> - QProcess 6734ms
> 
> - pipe and fread() with 64KB blocks 4832ms (38% faster then QProcess)
> 
> - temporary file and read() with 64KB blocks 4321ms (10% faster then pipe)
> 
> I have not enough knowledge to understand why temporary file is faster
> then pipe. My guess is, after reading some docs around, fread() uses a
> C standard I/O buffer, while read() is unbuffered.

Why don't you use the pipe and standard read()?

Even if you use "popen()" and get a "FILE *" back, you can still do

	int fd = fileno(file);

and use the raw IO capabilities.

The thing is, temporary files can actually be faster under Linux just because the Linux page-cache simply kicks ass. But it's not going to be _that_ big of a difference, and you need all that crazy "wait for rev-list to finish" and the "clean up temp-file on errors" etc crap, so there's no way it's a better solution.

If it really is stdio overhead (possibly locking), using "fileno()" and the raw unistd.h interfaces is going to avoid it.

(You still need to use "fclose()" to close the struct file afterwards, otherwise you'll leak memory, so you shouldn't _forget_ the original "struct FILE *", but you don't need to use it for anything else).

Using popen() and pipes also means that if the parent dies, the child will get a nice EPIPE on the writing side, which is what you want.