Re: [PATCH 0/8] t7900: untangle test dependencies

From: Junio C Hamano <gitster@pobox.com>
Date: Oct 17, 2023, 20:49 UTC
Message-ID: <xmqqv8b5ezz1.fsf@gitster.g>
In-Reply-To: <8cd788dc-7d16-4cfd-9f70-7889dcaa7199@app.fastmail.com>

"Kristoffer Haugsbakk" <code@khaugsbakk.name> writes:

Show 14 quoted lines

> On Tue, Oct 17, 2023, at 21:59, Junio C Hamano wrote:
>> It is kind-of surprising that with only 8 patches you can reach such
>> a state, but ...
>>
>>> # The tests that used to depend on each other should still pass
>>> # when run together
>>> ./t7900-maintenance.sh --quiet --run=setup,30,31 &&
>>
>> ... this puzzles me.  What does it mean for tests to "depend on each
>> other"?  Does this mean running #31 with or without running #30 runs
>> under different condition and potentially run different things?
>
> What I mean is that some preceding test has a side-effect that a test
> depends on.

I see. And 31 used to depend on the side effect of having ran 30, but in the updated test, the precondition 31 depends on is created by itself without relying on what 30 did (and in fact, perhaps in the updated test, 30 may rewind what it did as part of the clean-up process using test_when_finished). That makes sense.

Show 6 quoted lines

> I don't know what the policy is. :) My motivation was that I was working
> on something else which seemed to break the suite, then I tried to reduce
> the tests that were run to get rid of the noise (`--verbose`), but then it
> got confusing because I didn't know if I had really broken some tests
> myself or if more tests would start failing by only running a subset of
> them.

Yeah, it is a laudable goal, but I am not sure how practical it is to expect developers to maintain that propertly. Unless there is some automated test to enforce the independence of the tests, that is.

Thanks.