Re: [RFC \ WISH] Add -o option to git-rev-list

From: Linus Torvalds <torvalds@osdl.org>
Date: Dec 10, 2006, 22:16 UTC
Message-ID: <Pine.LNX.4.64.0612101410220.12500@woody.osdl.org>
In-Reply-To: <e5bfff550612101405h507cc877k87491264d11b0aa8@mail.gmail.com>

On Sun, 10 Dec 2006, Marco Costalba wrote:

> 
> Ok. Perhaps I'm doing something wrong but the following code it's
> always 10% slower then the temporary file one (4.7s against 4.3s for
> linux tree)

Why do you seem to be doing a "new" on every iteration inside the loop?

Also, why do you have that strange FILE_BLOCK_SIZE thing, and in particular the "if (len < FILE_BLOCK_SIZE)" check? One thing that pipes vs files do is the blocking factor.

Especially with older kernels, I _guarantee_ you that you'll only ever get 4kB at a time, so because of that "if (len < 64kB) break" thing, the only thing you're doing is to make sure things suck performance-wise, and you won't be reading the rest of the data until 100ms later.

IOW, your code is written for a file, and makes no sense there either (checking "feof(file)" is wrong, since you may well have hit the EOF *at*that*time*, but the file may GROW since you are doing it all in the background, so you can't rely on feof() anyway).

For a pipe, what you should do is to make sure it's in nonblocking mode, and just continue reading until it gets no more. And instead of using a timeout, you should use poll() or something to get notified when there is more data.

IOW, the reason it's slow is because you're doing the wrong thing.