Re: [RFC PATCH] fetch: stop emitting duplicate transfer.credentialsInUrl=warn warnings

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Nov 1, 2022, 22:17 UTC
Message-ID: <221101.86y1su2u2p.gmgdl@evledraar.gmail.com>
In-Reply-To: <Y2GHjnuyuwGpY3II@nand.local>

On Tue, Nov 01 2022, Taylor Blau wrote:

Show 24 quoted lines

> On Tue, Nov 01, 2022 at 04:01:18AM +0100, Ævar Arnfjörð Bjarmason wrote:
>> > Yeah, I think it is crappy UX, too. It's just that I think the tests
>> > should not _asserting_ the bad behavior. At most, they should tolerate
>> > the bad behavior as a band-aid. So I think Dscho's patch is doing the
>> > right thing (and I do agree that we should fix the immediate CI pain by
>> > adjusting the tests, and letting the user-visible fix proceed
>> > independently).
>>
>> The tests aren't just asserting the bad behavior, they're also ensuring
>> that it doesn't get worse. 1 warning is ideal, 2-3 is bad, but
>> tolerable, but if we start emitting 500 of these it would be nice to
>> know.
>
> I admit that this kind of argument does not sway me.
>
> Is it likely that we would suddenly start spewing 500 such warnings? If
> we did, are there no other tests that would catch it? And even if *that*
> were the case, would nobody happen to notice it in the meantime either
> during development or when we queue an affected topic onto 'next' for
> wider testing?
>
> I guess the answer is that it's possible that we'd miss such a
> regression in all of those above places, but to me it seems extremely
> unlikely that we'd let such a regression through without noticing.

Literally 500? Probably not, that was hyperbole to make a point, but several, low tens? Yeah, I know of at least a couple in-tree off the top of my head.

The point, which I assumed was clear is that we literally wouldn't notice if it were 500, and that sort of thing is a common pattern in our tests. I.e. in most cases we'd ideally test_cmp known output (at least to the extent of assuring ourselves that we're getting it right).

Instead we often just grep it, or don't test it at all. Sometimes for a good reason (e.g. the output containing absolute paths), but more often than not for no good reason.