git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3] coccicheck: process every source file at once

From
René Scharfe <l.s.r@web.de>
Date
Oct 6, 2018, 08:42 UTC
Message-ID
<8b51c4c0-df51-7639-07fe-74d6ca3b5944@web.de>
In-Reply-To
<20181005190045.GA17482@sigill.intra.peff.net>
Am 05.10.2018 um 21:00 schrieb Jeff King:
Show 34 quoted lines
> On Fri, Oct 05, 2018 at 08:50:50PM +0200, SZEDER Gábor wrote:
> 
>> On Fri, Oct 05, 2018 at 12:59:01PM -0400, Jeff King wrote:
>>> On Fri, Oct 05, 2018 at 04:53:35PM +0000, Keller, Jacob E wrote:
>>>
>>>>> Are we OK with saying 1.3-1.8GB is necessary to run coccicheck? That
>>>>> doesn't feel like an exorbitant request for a developer-only tool these
>>>>> days, but I have noticed some people on the list tend to have lousier
>>>>> machines than I do. ;)
>>>>>
>>>>> -Peff
>>>>
>>>> It's probably not worth trying to make this more complicated and scale
>>>> up how many files we do at once based on the amount of available
>>>> memory on the system...
>>>
>>> Yeah, that sounds too complicated. At most I'd give a Makefile knob to
>>> say "spatch in batches of $(N)". But I'd prefer to avoid even that
>>> complexity if we can.
>>
>> But perhaps one more if-else, e.g.:
>>
>>   if test -n "$(COCCICHECK_ALL_AT_ONCE)"; then \
>>       <all at once from Jacob>
>>   else
>>       <old for loop>
>>   fi
>>
>> would be an acceptable compromise?  Dunno.
> 
> That's OK, too, assuming people would actually want to use it. I'm also
> OK shipping this (with the "make -j" fix you suggested) and seeing if
> anybody actually complains. I assume there are only a handful of people
> running coccicheck in the first place.

FWIW, my development environment is a virtual machine with 1200MB RAM and 900MB swap space. coccicheck takes almost eight minutes sequentially, and four and a half minutes with -j4.

Unsurprisingly, it fails after almost three minutes with the patch, reporting that it ran out of memory. With 2900MB it fails after almost two minutes, with 3000MB it succeeds after a good two minutes.

time(1) says (for -j1):

433.30user 36.17system 7:49.84elapsed 99%CPU (0avgtext+0avgdata 108212maxresident)k 192inputs+1512outputs (0major+16409056minor)pagefaults 0swaps

129.74user 2.06system 2:13.27elapsed 98%CPU (0avgtext+0avgdata 1884568maxresident)k 236896inputs+1096outputs (795major+462129minor)pagefaults 0swaps

So with the patch it's more than three times faster, but needs more than seventeen times more memory. And I need a bigger VM. :-/

René
Previous: Jacob KellerNext: Jeff King
Message 11 of 20 in “coccicheck: process every source file at once”
  1. coccicheck: process every source file at onceJacob Keller, Oct 2, 2018
  2. Jacob KellerOct 2, 2018
  3. Jacob KellerOct 5, 2018
  4. SZEDER GáborOct 5, 2018
  5. Jeff KingOct 5, 2018
  6. Keller, Jacob EOct 5, 2018
  7. Jeff KingOct 5, 2018
  8. SZEDER GáborOct 5, 2018
  9. Jeff KingOct 5, 2018
  10. Jacob KellerOct 5, 2018
  11. René ScharfeOct 6, 2018
  12. Jeff KingOct 9, 2018
  13. SZEDER GáborOct 5, 2018
  14. Jeff KingOct 5, 2018
  15. SZEDER GáborOct 5, 2018
  16. Jeff KingOct 9, 2018
  17. SZEDER GáborOct 10, 2018
  18. Jeff KingOct 10, 2018
  19. Beat BolliOct 7, 2018
  20. Beat BolliOct 7, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.