Re: [PATCH v3] coccicheck: process every source file at once
- From
René Scharfe <l.s.r@web.de>
- Date
- Oct 6, 2018, 08:42 UTC
- Message-ID
- <8b51c4c0-df51-7639-07fe-74d6ca3b5944@web.de>
- In-Reply-To
- <20181005190045.GA17482@sigill.intra.peff.net>
Am 05.10.2018 um 21:00 schrieb Jeff King:
Show 34 quoted lines
> On Fri, Oct 05, 2018 at 08:50:50PM +0200, SZEDER Gábor wrote: > >> On Fri, Oct 05, 2018 at 12:59:01PM -0400, Jeff King wrote: >>> On Fri, Oct 05, 2018 at 04:53:35PM +0000, Keller, Jacob E wrote: >>> >>>>> Are we OK with saying 1.3-1.8GB is necessary to run coccicheck? That >>>>> doesn't feel like an exorbitant request for a developer-only tool these >>>>> days, but I have noticed some people on the list tend to have lousier >>>>> machines than I do. ;) >>>>> >>>>> -Peff >>>> >>>> It's probably not worth trying to make this more complicated and scale >>>> up how many files we do at once based on the amount of available >>>> memory on the system... >>> >>> Yeah, that sounds too complicated. At most I'd give a Makefile knob to >>> say "spatch in batches of $(N)". But I'd prefer to avoid even that >>> complexity if we can. >> >> But perhaps one more if-else, e.g.: >> >> if test -n "$(COCCICHECK_ALL_AT_ONCE)"; then \ >> <all at once from Jacob> >> else >> <old for loop> >> fi >> >> would be an acceptable compromise? Dunno. > > That's OK, too, assuming people would actually want to use it. I'm also > OK shipping this (with the "make -j" fix you suggested) and seeing if > anybody actually complains. I assume there are only a handful of people > running coccicheck in the first place.
FWIW, my development environment is a virtual machine with 1200MB RAM and 900MB swap space. coccicheck takes almost eight minutes sequentially, and four and a half minutes with -j4.
Unsurprisingly, it fails after almost three minutes with the patch, reporting that it ran out of memory. With 2900MB it fails after almost two minutes, with 3000MB it succeeds after a good two minutes.
time(1) says (for -j1):
433.30user 36.17system 7:49.84elapsed 99%CPU (0avgtext+0avgdata 108212maxresident)k 192inputs+1512outputs (0major+16409056minor)pagefaults 0swaps
129.74user 2.06system 2:13.27elapsed 98%CPU (0avgtext+0avgdata 1884568maxresident)k 236896inputs+1096outputs (795major+462129minor)pagefaults 0swaps
So with the patch it's more than three times faster, but needs more than seventeen times more memory. And I need a bigger VM. :-/
René