Re: [RFC PATCH 0/2] add an external testing library for unit tests

From: brian m. carlson <sandals@crustytoothpaste.net>
Date: Apr 27, 2023, 21:35 UTC
Message-ID: <ZErqvkCIkytTCvZr@tapette.crustytoothpaste.net>
In-Reply-To: <20230427175007.902278-1-calvinwan@google.com>

On 2023-04-27 at 17:50:05, Calvin Wan wrote:

Show 11 quoted lines

> In our current testing environment, we spend a significant amount of
> effort crafting end-to-end tests for error conditions that could easily
> be captured by unit tests (or we simply forgo some hard-to-setup and
> rare error conditions). Unit tests additionally provide stability to the
> codebase and can simplify debugging through isolation. Turning parts of
> Git into libraries[1] gives us the ability to run unit tests on the
> libraries and to write unit tests in C. Writing unit tests in pure C,
> rather than with our current shell/test-tool helper setup, simplifies
> test setup, simplifies passing data around (no shell-isms required), and
> reduces testing runtime by not spawning a separate process for every
> test invocation.

I think this is a good idea. Like Junio said downthread, we need to have integration and end-to-end tests, and I think those will continue to compose the majority of our tests. However, having simple and easy to use unit tests would be really valuable for testing things like our hash and HMAC implementations, as well as a variety of other library functions, including our strbuf code.

At work, I recently ported a project from C with no unit tests to Rust with unit tests (and in both cases, our existing integration and functional testsuite) and found that writing unit tests let us have substantially more confidence in the correct functioning of our code.

I think it's great that we're using existing TAP functionality as well.

If you're looking for some proof-of-concept projects to illustrate why this is useful in v1, might I suggest some of the subsystems above? Assuming it lands, I plan on sending some tests for the percent-encoding in the strbuf code and some more aggressive testing of our block SHA-256 and HMAC implementations if nobody gets to it before me. (Mostly because this is stuff I wrote or touched and would like to have more confidence in.)

-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA