Re: [RFC/PATCH] tests: support testing with an arbitrary default branch (sort of)

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Nov 13, 2020, 22:00 UTC
Message-ID: <nycvar.QRO.7.76.6.2011132229480.18437@tvgsbejvaqbjf.bet>
In-Reply-To: <20201113191418.GA764688@coredump.intra.peff.net>

Hi Peff and Ævar,

On Fri, 13 Nov 2020, Jeff King wrote:

Show 17 quoted lines

> On Fri, Nov 13, 2020 at 05:13:20PM +0100, Ævar Arnfjörð Bjarmason wrote:
>
> >  * A lot of tests (but a small minority of the total) have master
> >    "master" hardcoded in some way. We now inventory them in
> >    tests-that-need-master.txt, we can still remove the names from that
> >    file and manually change the code later, but this accomplishes a
> >    clean test run with a relatively easy-to-review diff.
> >
> >    We ignore GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=<name> when it comes
> >    to these files, unless
> >    GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME_HARDER=true is set.
>
> I'm confused how this is better. We could just be setting
> GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME at the top of those files, couldn't
> we? (Likewise, I think annotating individual scripts is more
> decentralized than a magic pattern in test-lib.sh, though it amounts to
> the same thing).

I considered this alternative very briefly, and rejected it immediately because it would have implied a drawn-out limbo where the test suite will be inconsistent. That's not what I want.

> And if I understand the current state of Dscho's patches, we don't
> _have_ to convert any tests right now. We could just annotate those
> scripts which are not yet converted to have them use the old name.

And that's what I did in some cases, see e.g. 5d5f4ea30de (t5411: finish preparing for `main` being the default branch name, 2020-10-31).

It's just quite a ton of work to do this for all the affected test scripts, and I wanted to automate as much as possible of that process so that it'll be easier to cope with all the other topics that are in flight.

Therefore, I opted to "annotate the scripts" via a `case` in `test-lib.sh`. It should be quite a bit easier to understand, too, for the occasional reader, as it shows you the complete range of scripts that have been handled so far.

Show 5 quoted lines

> But I don't think we want to live in that state indefinitely. It's
> slightly annoying to have inconsistent naming within the tests. I'd be
> happy to switch individual tests at a leisurely pace over the next
> couple of months or whatever. But since Dscho has bothered to write all
> of the patches now, why not use them?

Yep, I am a bit puzzled why we need to consider other approaches. I would have thought that the more pressing issue is to verify that I

1) caught all the necessary conversions,

2) did not miss any non-trivial adjustments (such as `naster` and aligned
   comments).

At the same time, I am quite thankful for all the help Ævar provided; mentioning that I missed `naster` in t1400 was definitely super helpful.

Show 6 quoted lines

> I'm much more concerned about the lack of documentation changes
> associated with the final patch. We don't necessarily need to eradicate
> every mention of "master" from the documentation, but I think we do need
> to make sure that examples and instructions are consistent with how Git
> will actually behave. And that does need to happen at the same time as
> the user-visible flip of the default.

I am sorry. This is totally my fault. I should have described this much better in the cover letter. This patch series deliberately omits all the documentation changes.

I _did_ prepare them, they are pretty much ready-to-go, as part of the `inclusive-naming` branch I track in https://github.com/gitgitgadget/git/pull/655 (I use the branch name `use-main-as-default-branch-name-docs` in the `inclusive-naming` branch thicket to track the documentation changes).

The decision to separate them out into their own patch series was made consciously, to avoid having one big, honking, totally unreviewably large patch series.

Therefore, just like I fed the preparatory patch series in a slow cadence, I planned on feeding the `-docs` patch series once the dust settles on this patch series (i.e. once it hits `next`).

> I'm on the fence whether there should be a deprecation period or major
> version bump for the final patch, but making the tests flexible enough
> to handle the before and after state seems like it can be done uncoupled
> from the actual default-flip.

Hmm. On that matter, I wonder what the big fuss is all about. It's not like Git is forcing anybody to change their default branch. That's not at all what we're doing. To the contrary: _after_ many projects chose to change their default branch names, and _after_ GitHub started to follow that trend, Git added support for `init.defaultBranch` to accommodate that use case better. So in a sense, we're actually pretty late changing the fall-back of `init.defaultBranch`, at least from the perspective outside of the Git project.

There have been plenty of articles about this in the meantime, too, and I could imagine that most developers are at least aware that the shift away from `master` is happening, in many quite visible projects.

So to me, this whole discussion about whether this should bump the major version of Git seems a bit overblown. It's not like the median developer is creating new repositories on a regular basis, and if they do, chances are that they go with whatever branch name happens to be the initial one.

What is much more common is that developers clone existing projects. And guess what, many of those projects already use a different default branch name. And developers seem to accept that and just go on with their lives.

If it was up to me, I would reserve a major version increment to much bigger changes.

Ciao, Dscho