Re: [PATCH] tests: add an optional test to test git-annex
- From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
- Date
- May 17, 2017, 06:47 UTC
- Message-ID
- <CACBZZX4Jppr7ht7m444EjW4CDYX5CMvnxtStH4bF=A19TYKcZg@mail.gmail.com>
- In-Reply-To
- <xmqq7f1gyzep.fsf@gitster.mtv.corp.google.com>
On Wed, May 17, 2017 at 4:45 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 5 quoted lines
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > >> Add an optional test to test git-annex. It's guarded by a new >> EXTERNAL_TESTS environment variable. Running this test takes me 10 >> minutes.
[Re-arranged your mail because it worked better with my reply]
> I do not mind at all to place the simple reproduction recipe Joey > posted as a new test in our test suite, though. That kind of test > that catches changes to externally visible behaviour surely belongs > to our test suite.
This is not a replacement for having an isolated test for the issue Joey noted. We should have a separate patch for that, but I did not have time/interest in writing that up. This change is orthagonal to that.
Show 5 quoted lines
> Well, it is one thing to place git-annex under CI to make sure its > latest and greatest works together well with our latest and greatest > (and it may be something we want to see happen), but driving its > tests from our testsuite sounds like a tail wagging the dog, at > least to me.
To me this is just a question of:
* Is it the case that git-annex tests for a lot of edge cases we don't test for: Yes, probably. As evidenced by them spotting this regression, and not us.
* We can (and should) add a test for the specific breakage we caused in 2.13.0, but that's no replacement for other things annex may be covering & we may be missing which'll catch future breakages.
* It's a pretty established practice to test a library (git) along with its consumers (e.g. annex) before a major release.
* This allows us to do that at minimal cost. I think it makes sense to add this and integration tests for other similar utilities if they're similarly easy to integrate.