From: Andrzej Hunt Date: Fri, 11 Jun 2021 15:44:09 GMT Subject: Re: UNLEAK(), leak checking in the default tests etc. Message-ID: <18238547-0fbb-a9c4-a769-7e6d865b171a@ahunt.org> In-Reply-To: On 10/06/2021 18:36, Jeff King wrote: > On Thu, Jun 10, 2021 at 05:32:41PM +0200, Andrzej Hunt wrote: > >>> I do think it would be worth splitting out ASan from leak-checking. The >>> whole suite should run clean with regular ASan already, and we'd want to >>> find regressions there even in the tests that aren't leak-clean. I do >>> periodic ASan runs already; the main argument against doing it for every >>> CI run is just that's a lot more CPU. But maybe not enough to be >>> prohibitive? It's probably still way cheaper than running the test suite >>> on Windows. >> >> I've been running tests with ASAN in the Github Actions environment, and a >> single run takes just over 30 minutes [1] - which I believe is similar to >> the normal test jobs (they do run the test suite twice in that time I >> think). >> >> I've been doing the same with UBSAN, and that's even faster at 15-20 minutes >> [2]. However I get the impression that ASAN issues are both more common (at >> least on seen), and more impactful - so I would argue that ASAN should be >> prioritised if there's spare capacity. (I have no idea if ASAN+UBSAN can be >> combined, but I suspect that doing so would make the tests slower?) > > I routinely do SANITIZE=address,undefined since they are both useful > (and we do not trigger either in the current test suite). I never > measured the time of their combined use versus just one, but surely it's > faster the two-at-once approach is faster than running the test suite > twice. I'm seeing 33 minutes for SANITIZE=address,undefined - which is no slower than SANITIZE=address by itself (disclaimer: it's only one measurement): https://github.com/ahunt/git/runs/2795642716?check_suite_focus=true (The job's name is wrong but if you look in the logs you can confirm that it's using address+undefined.) The usual linux and mac test-jobs are actually from 24 to 30 minutes (the numbers seem a bit variable) - with the exception of one faster 10 minute job: https://github.com/git/git/actions/runs/925771097 vs https://github.com/git/git/actions/runs/927729395 So to summarise: adding an ASAN+UBSAN job would make things a bit slower, but not a huge amount slower.