Re: Huge win, compressing a window of delta runs as a unit

Mozilla CVS files have between 1 and 1700 deltas. The average is ten, but the deviation is high. I believe less than 10 files have over 1000 deltas, they are all in the root directory and related to the build process. Many files have no deltas, or because of CVS artifacts all of the revisions are identical.

I am still IO bound. Random access IO is the problem, not stream IO. I have to open and read 110,000 (5GB total) files. It takes about 2hrs to do all of the IO. I'm not CPU bound yet but as we make things more efficient, I am getting closer to being CPU bound.

Forking is not an option. It can takes days to fork 1M copies of an app. I have used oprofile on parsecvs. It spends 60% of the time in the kernel processing fork calls. Parsecvs runs for 6hrs on mozcvs and dies without finishing.

I am back to working on the branch code. I'm over the cold I got from my 2 yr old. It is slow going now. I am in the phase where the import process runs without error 5-10 minutes and then dies from some unusual branch case. I fix it up and try again. I am slowly identifying and removing all the code in cvs2svn that puts the branches and symbols into their own subdirectories.