git can not be built for s390x since update to git-2.38.0

I am one of the zsystems Maintainers (s390x architecture) at openSUSE and git can not be built any more since the update from 2.37.3 to 2.38.0, and with 2.38.1 afterwards. We have tried to debug it.

There is no real test error output, but we can identify, that it is happening in the test part with check-chainlint.

[ 386s] + ./.make -j2 test [ 386s] /usr/bin/make -C git-gui gitexecdir='/usr/libexec/git' all [ 387s] /usr/bin/make -C gitk-git all [ 387s] /usr/bin/make -C templates SHELL_PATH='/bin/sh' PERL_PATH='/usr/bin/perl' [ 387s] make[1]: Entering directory '/home/abuild/rpmbuild/BUILD/git-2.38.1/templates' [ 387s] : no custom templates yet [ 387s] make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/git-2.38.1/templates' [ 387s] /usr/bin/make -C t/ all [ 387s] make[1]: Entering directory '/home/abuild/rpmbuild/BUILD/git-2.38.1/t' [ 387s] rm -f -r 'test-results' [ 387s] make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/git-2.38.1/t' [ 387s] make[1]: Entering directory '/home/abuild/rpmbuild/BUILD/git-2.38.1/t' [ 387s] make[1]: *** [Makefile:83: check-chainlint] Error 1 [ 387s] make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/git-2.38.1/t' [ 387s] make[1]: *** Waiting for unfinished jobs.... [ 388s] make: *** [Makefile:3075: test] Error 2 [ 388s] error: Bad exit status from /var/tmp/rpm-tmp.ztqAAp (%check) [ 388s]

We figured out, that the perl script have been changed there in the last month. I have to give the hint, that all versions of openSUSE for s390x are affected by this build failure. Builds are working for the other architectures.

How can we fix this issue?

Hi Sarah,

On Tue, 1 Nov 2022 at 19:38, Sarah Julia Kriesch <sarah.kriesch@opensuse.org> wrote:

You can disable this feature by setting the GIT_TEST_CHAIN_LINT environment variable to "0". That should hopefully allow you to run the tests successfully.

That shouldn't risk you anything, like reduce test coverage or similar. That the tests are chainlint-ok (roughly speaking, that they are written the way the Git project likes its tests) has already been established by many others at this point.

I'm cc-ing the resident chainlint expert.

Martin

On Tue, Nov 01, 2022 at 08:05:36PM +0100, Martin Ågren wrote:

Indeed. It would be nice to get some more helpful output from the chainlint machinery when whatever is happening here happens.

I've never not had chainlint output when I was expecting it (and, I often get chainlint output when I'm *not* expecting any ;-)), so I wonder what is going on here.

Thanks, Taylor

On Tue, 1 Nov 2022 at 20:05, Martin Ågren <martin.agren@gmail.com> wrote:

On further reading, it appears this error happens in the "check-chainlint" target without even checking this environment variable (that happens elsewhere). I doubt my original suggestion will be a complete workaround.

One approach might be to patch the "check-chainlint" target into a no-op, possibly combined with turning off the chainlinting as I mentioned above. My makefile-foo is unfortunately not sufficiently advanced to immediately spot what might be going on here.

At any rate, all of the above is just about working around the issue. As for a proper fix, let's hope someone like Eric can chime in.

Martin

On Tue, Nov 1, 2022 at 3:41 PM Martin Ågren <martin.agren@gmail.com> wrote:

Makefile-foo or not, there isn't much to go on here since the `make` output doesn't show the actual problem. Squinting at the `check-chainlint` target in "t/Makefile", I wonder if the problem is with the line-splicing and missing command termination; that is, it could just be a shell syntax error which is picked up by more strict shells such as `dash`. In particular, in this hunk of code:

    { \
        echo "# chainlint: $(CHAINLINTTMP_SQ)/tests" && \
        for i in $(CHAINLINTTESTS); do \
            echo "# chainlint: $$i" && \
            sed -e '/^[   ]*$$/d' chainlint/$$i.expect; \
        done \
    } >'$(CHAINLINTTMP_SQ)'/expect && \

if you add a semicolon after `done` like this:

    done; \

does that fix the problem?

If not, then you may want to remove the "@" from the "@mkdir -p ..." in the body of the `check-chainlint` rule in t/Makefile so that `make` will print out the actual command(s) it is running, and then invoke the target manually with `make -C t check-chainlint`. With the actual commands visible, copy/paste in the terminal to run each command individually and check the exit status $? after each command invocation to find which, if any, is failing.

Sarah Julia Kriesch wrote:

I don't know if it's of much help, but for what it's worth 2.38.0 and 2.38.1 been built in Fedora for s390x without issue. Build logs and artifacts for 2.38.1 can be found here:

    https://koji.fedoraproject.org/koji/buildinfo?buildID=2077983