Re: [PATCH 00/12] Group reffiles tests

From: Patrick Steinhardt <ps@pks.im>
Date: Jan 18, 2024, 11:38 UTC
Message-ID: <ZakNqm0zyw8IiIhB@tanuki>
In-Reply-To: <xmqqv87rsan6.fsf@gitster.g>

On Wed, Jan 17, 2024 at 05:17:17PM -0800, Junio C Hamano wrote:

Show 27 quoted lines

> "John Cai via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
> > This series groups REFFILES specific tests together. These tests are
> > currently grouped together across the test suite based on functionality.
> > However, since they exercise low-level behavior specific to the refs backend
> > being used (in these cases, the ref-files backend), group them together
> > based on which refs backend they test. This way, in the near future when the
> > reftables backend gets upstreamed we can add tests that exercise the
> > reftables backend close by in the t06xx area.
> >
> > These patches also remove the REFFILES prerequisite, since all the tests in
> > t06xx are reffiles specific.
> 
> As we already have REFFILES lazy prereq, even _before_ we enable the
> reftable backend, I think that we should start t0600 and t0602 with
> 
> 	. ./test-lib.sh
> 	if ! test_have_prereq REFFILES
> 	then
> 		skip_all='skipping reffiles specific tests'
> 		test_done
> 	fi
> 
> which is more in line with the existing convention.  It is more
> efficient than "forcing t0600 and t0602 to run always with reffiles"
> when you have a CI job that uses reftable for all tests and another
> CI job that uses reffiles for all tests.

I think it depends. If we use the REFFILES prereq for the files-specific tests, then we should likely also use the REFTABLE prereq for the reftable-specific tests.

But that raises the question of whether we want to add a CI job that exercises code with the reftable backend for every major platform (Linux, macOS, Windows). If so then your proposal would be fine with me as we make sure that things work alright on all of them. But if we think that this would be too expensive then I'd like to at least have very basic test coverage on all platforms by always running these backend-specific tests.

Patrick