From: James Ramsay Date: Thu, 12 Mar 2020 04:08:49 GMT Subject: [TOPIC 12/17] Test harness improvements Message-ID: In-Reply-To: 1. Jonathan N: Test harness is an important part of the development process, shapes what kinds of tests people write. What can we improve? 2. Peff: I love our test harness 3. Brian: It’s amazing for integration tests. For our C code it’s a lot harder to do unit tests. We sometimes have a portability issue, about POSIX shell vs others. 4. ZJ: I like how it also acts as a piece of documentation. 5. Jonathan N: If we had more unit tests: if I am working on refs, I might like to run all tests related to that. And now we have this lack of dependency graph between this 6. Peff: I’m super nervous about that. Tools like code coverage could do this. But I’ve seen cases where all new tests are green, and tests in the area I expected they succeed. But at some far corner it seems to fail. So you’re optimizing for speed, might be losing in correctness. I’m biased because I can run all tests on my computer in 1 minute. But for Windows this doesn’t seem to work that. 7. Peff: We can spend time on speeding up things, making it better parallelized for example. I’ll send some patches out on this. 8. Jonathan N: Really nice contribution to Git by David Barr, whose background was as a Java developer and thus the code was written in a Java way with clear API boundaries and unit tests. 9. Brian: Yes if your function is doing too much, it should be split up making it possible to test the separate pieces and then have a function that calls those and tests the end result.