From: Junio C Hamano <gitster@pobox.com>
Date: Sun, 10 May 2009 18:40:53 GMT
Subject: Re: [PATCH] Avoid unnecessary 'lstat()' calls in 'get_stat_data()'
Message-ID: <7v7i0oztqi.fsf@alter.siamese.dyndns.org>
In-Reply-To: <alpine.LFD.2.01.0905100943370.3586@localhost.localdomain>

Linus Torvalds <torvalds@linux-foundation.org> writes:

> I've found that "make -j64 test" does fairly well, bringing the cost down 
> to something reasonable. Some of the SVN tests seem to sometimes randomly 
> fail when done in parallel (I've not tried to debug it, I assume it's 
> either some SVN bug, or just the test infrastructure having some shared 
> SVN central repo thing), but it happens rarely enough that even if you 
> have to run it twice, it's still worth it. 
>
> [ Side note: the success output of "make test" makes it almost impossible 
>   to debug the error cases when you do that "make -j64" thing. Sad. It 
>   would be good to have the tests that fail clearly say exactly what test 
>   failed, because when you run 64 tests at the same time, having "case 9" 
>   fail is almost totally useless information - test 9 of _which_ 
>   testsuite? ]
>
> I don't generally build docs, but they should run in parallel too, and at 
> least your fedora build on kernel.org has a nice quad-core machine with 
> lots of memory, so "-j8" or something is reasonable.

Thanks; unfortunately I already do all the tricks known to me, including
running make in parallel (I happen to use -j4) and maintaining a separate
build farm for each of the branches to avoid recompilation of programs and
reformatting docs.  However, on a

    model name      : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
    stepping        : 1
    cpu MHz         : 2004.185
    cache size      : 512 KB

with slow IDE disks, optimizing and cheating at the software level goes
only so far...  For example, a typical cycle I just did looks like this:

$ /usr/bin/time Meta/Dothem --pedantic >:all.log 2>&1; tail -n 2 :all.log
2089.15user 1610.16system 45:56.91elapsed 134%CPU (0avgtext+0avgdata 0maxresident)k
220088inputs+3723296outputs (696major+305868883minor)pagefaults 0swaps

You have to remember that people use boxes that are a bit slower compared
to the boxes you are used to ;-)