threads / discuss / 58828

Re: git can not be built for s390x since update to git-2.38.0

Subject: Re: git can not be built for s390x since update to git-2.38.0

## tl;dr

5 messages between Nov 21, 2022 and Nov 22, 2022.

replies: 4people: 2as markdown or json

Eric Sunshine· Nov 21, 2022, 05:12 UTC · lore
[cc:+git; taking this back to the mailing list so others can weigh in]

On Wed, Nov 9, 2022 at 1:38 AM Sarah Julia Kriesch <sarah.kriesch@opensuse.org> wrote:

Show 25 quoted lines
>  > Gesendet: Sonntag, 06. November 2022 um 21:42 Uhr
>  > Von: "Eric Sunshine" <sunshine@sunshineco.com>
>  >
>  > On Sun, Nov 6, 2022 at 2:18 PM Sarah Julia Kriesch
>  > <sarah.kriesch@opensuse.org> wrote:
>  > > We have also identified this week, that chainlint.pl --stats says "0
>  > > tests". That can explain the empty output.
>  > >
>  > > ~/rpmbuild/BUILD/git-2.38.1/t> /usr/bin/perl chainlint.pl --stats
>  > > --emit-all chainlinttmp/tests
>  > > total: 0 workers, 0 scripts, 0 tests, 0 errors, 0.01s/0.01s (wall/user)
>  >
>  > Was `--stats` added for debugging this issue, or is that somehow part
>  > of your build process?
>  >
> Yes. We tried to debug that manually. Therefore, we have added the --stats.
>
>  > What is the content of the "chainlinttmp/tests" file? Is it empty?
>  >
> It is empty.
>
>  > It is quite curious that it reports 0 scripts. It should report 1
>  > script even if chainlinttmp/file is empty. This might point a finger
>  > at File::Glob::bsd_glob() returning an empty list for some reason, or
>  > the problem could be a failure with Perl "ithreads".

Do you happen to know if Perl is built with "ithreads" on your platform, and if so, is "ithread" support working?

The empty "chainlinttmp/tests" file also sounds suspicious. If you haven't already done so, I would recommend following the advice in [1] and running each command individually which Makefile would have run; this might allow you to isolate the problematic command or shell construct if such exists.

[1]: https://lore.kernel.org/git/CAPig+cS184iKk6icG6ML=X+6Ng=fJyfic8izFyp1hT40cChm4g@mail.gmail.com/
Ævar Arnfjörð Bjarmason· Nov 22, 2022, 14:35 UTC · re: Eric Sunshine · lore
On Mon, Nov 21 2022, Eric Sunshine wrote:
Show 40 quoted lines
> [cc:+git; taking this back to the mailing list so others can weigh in]
>
> On Wed, Nov 9, 2022 at 1:38 AM Sarah Julia Kriesch
> <sarah.kriesch@opensuse.org> wrote:
>>  > Gesendet: Sonntag, 06. November 2022 um 21:42 Uhr
>>  > Von: "Eric Sunshine" <sunshine@sunshineco.com>
>>  >
>>  > On Sun, Nov 6, 2022 at 2:18 PM Sarah Julia Kriesch
>>  > <sarah.kriesch@opensuse.org> wrote:
>>  > > We have also identified this week, that chainlint.pl --stats says "0
>>  > > tests". That can explain the empty output.
>>  > >
>>  > > ~/rpmbuild/BUILD/git-2.38.1/t> /usr/bin/perl chainlint.pl --stats
>>  > > --emit-all chainlinttmp/tests
>>  > > total: 0 workers, 0 scripts, 0 tests, 0 errors, 0.01s/0.01s (wall/user)
>>  >
>>  > Was `--stats` added for debugging this issue, or is that somehow part
>>  > of your build process?
>>  >
>> Yes. We tried to debug that manually. Therefore, we have added the --stats.
>>
>>  > What is the content of the "chainlinttmp/tests" file? Is it empty?
>>  >
>> It is empty.
>>
>>  > It is quite curious that it reports 0 scripts. It should report 1
>>  > script even if chainlinttmp/file is empty. This might point a finger
>>  > at File::Glob::bsd_glob() returning an empty list for some reason, or
>>  > the problem could be a failure with Perl "ithreads".
>
> Do you happen to know if Perl is built with "ithreads" on your
> platform, and if so, is "ithread" support working?
>
> The empty "chainlinttmp/tests" file also sounds suspicious. If you
> haven't already done so, I would recommend following the advice in [1]
> and running each command individually which Makefile would have run;
> this might allow you to isolate the problematic command or shell
> construct if such exists.
>
> [1]: https://lore.kernel.org/git/CAPig+cS184iKk6icG6ML=X+6Ng=fJyfic8izFyp1hT40cChm4g@mail.gmail.com/
I can also reproduce this on gcc202. It's the same issue with ending up
with an empty "tests" file.
	
	avar@gcc202:/run/user/1632/git/t$ uname -a
	Linux gcc202 6.0.0-4-sparc64-smp #1 SMP Debian 6.0.8-1 (2022-11-11) sparc64 GNU/Linux
	avar@gcc202:/run/user/1632/git/t$ lsb_release -a
	No LSB modules are available.
	Distributor ID: Debian
	Description:    Debian GNU/Linux bookworm/sid
	Release:        n/a
	Codename:       bookworm

The change at the tip of https://lore.kernel.org/git/221122.86cz9fbyln.gmgdl@evledraar.gmail.com/ fixes it for me, i.e. getting rid of the perl threads stuff...

That box is built with "useithreads=define", so it's not that it doesn't support threading in general.

Eric Sunshine· Nov 22, 2022, 16:16 UTC · re: Ævar Arnfjörð Bjarmason · lore

On Tue, Nov 22, 2022 at 9:41 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

Show 9 quoted lines
> On Mon, Nov 21 2022, Eric Sunshine wrote:
> > The empty "chainlinttmp/tests" file also sounds suspicious. If you
> > haven't already done so, I would recommend following the advice in [1]
> > and running each command individually which Makefile would have run;
> > this might allow you to isolate the problematic command or shell
> > construct if such exists.
>
> I can also reproduce this on gcc202. It's the same issue with ending up
> with an empty "tests" file.

Were you able to determine why the "tests" file is empty? The creation loop is simple enough:

    for i in $(CHAINLINTTESTS); do \
        echo "test_expect_success '$$i' '" && \
        sed -e '/^# LINT: /d' chainlint/$$i.test && \
        echo "'"; \
    done >'$(CHAINLINTTMP_SQ)'/tests && \
and the file is never changed after that (or shouldn't be).
Show 8 quoted lines
>         avar@gcc202:/run/user/1632/git/t$ uname -a
>         Linux gcc202 6.0.0-4-sparc64-smp #1 SMP Debian 6.0.8-1 (2022-11-11) sparc64 GNU/Linux
>         avar@gcc202:/run/user/1632/git/t$ lsb_release -a
>         No LSB modules are available.
>         Distributor ID: Debian
>         Description:    Debian GNU/Linux bookworm/sid
>         Release:        n/a
>         Codename:       bookworm
This is one of the compile-machines to which you have access?
> The change at the tip of
> https://lore.kernel.org/git/221122.86cz9fbyln.gmgdl@evledraar.gmail.com/
> fixes it for me, i.e. getting rid of the perl threads stuff...

I'm confused. How does a change to chainlint.pl -- which only reads the "tests" file -- fix the empty "tests" file?

> That box is built with "useithreads=define", so it's not that it doesn't
> support threading in general.
Ævar Arnfjörð Bjarmason· Nov 22, 2022, 22:01 UTC · re: Eric Sunshine · lore
On Tue, Nov 22 2022, Eric Sunshine wrote:
Show 22 quoted lines
> On Tue, Nov 22, 2022 at 9:41 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> On Mon, Nov 21 2022, Eric Sunshine wrote:
>> > The empty "chainlinttmp/tests" file also sounds suspicious. If you
>> > haven't already done so, I would recommend following the advice in [1]
>> > and running each command individually which Makefile would have run;
>> > this might allow you to isolate the problematic command or shell
>> > construct if such exists.
>>
>> I can also reproduce this on gcc202. It's the same issue with ending up
>> with an empty "tests" file.
>
> Were you able to determine why the "tests" file is empty? The creation
> loop is simple enough:
>
>     for i in $(CHAINLINTTESTS); do \
>         echo "test_expect_success '$$i' '" && \
>         sed -e '/^# LINT: /d' chainlint/$$i.test && \
>         echo "'"; \
>     done >'$(CHAINLINTTMP_SQ)'/tests && \
>
> and the file is never changed after that (or shouldn't be).

Sorry. I meant that the result of running *against* the non-empty test file is empty. I.e.:

	$ perl chainlint.pl chainlinttmp/tests ; echo $?
	0
Anyway, the "fix" here can also be:
	 $jobs = ncores() if $jobs < 1;
	+$jobs = 1 if $jobs < 1;
I.e. it's another case of bad /proc/cpuinfo parsing.
On that platform, this also works:
	
	diff --git a/t/chainlint.pl b/t/chainlint.pl
	index 976db4b8a0..e6bc476fc5 100755
	--- a/t/chainlint.pl
	+++ b/t/chainlint.pl
	@@ -656,7 +656,7 @@ sub ncores {
	        # Windows
	        return $ENV{NUMBER_OF_PROCESSORS} if exists($ENV{NUMBER_OF_PROCESSORS});
	        # Linux / MSYS2 / Cygwin / WSL
	-       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^processor\s*:/, <>)); } if -r '/proc/cpuinfo';
	+       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^(?:CPU\d+|processor\s*):/, <>)); } if -r '/proc/cpuinfo';
	        # macOS & BSD
	        return qx/sysctl -n hw.ncpu/ if $^O =~ /(?:^darwin$|bsd)/;
	        return 1;
But a generic guard against parse failure seems prudent.
Show 10 quoted lines
>>         avar@gcc202:/run/user/1632/git/t$ uname -a
>>         Linux gcc202 6.0.0-4-sparc64-smp #1 SMP Debian 6.0.8-1 (2022-11-11) sparc64 GNU/Linux
>>         avar@gcc202:/run/user/1632/git/t$ lsb_release -a
>>         No LSB modules are available.
>>         Distributor ID: Debian
>>         Description:    Debian GNU/Linux bookworm/sid
>>         Release:        n/a
>>         Codename:       bookworm
>
> This is one of the compile-machines to which you have access?

Yes, and you can get access to it too. The GCC farm is open for free software development, you only need to send in an E-Mail request, ssh key etc.

Show 6 quoted lines
>> The change at the tip of
>> https://lore.kernel.org/git/221122.86cz9fbyln.gmgdl@evledraar.gmail.com/
>> fixes it for me, i.e. getting rid of the perl threads stuff...
>
> I'm confused. How does a change to chainlint.pl -- which only reads
> the "tests" file -- fix the empty "tests" file?
It doesn't, sorry, see above.
Eric Sunshine· Nov 22, 2022, 22:20 UTC · re: Ævar Arnfjörð Bjarmason · lore

On Tue, Nov 22, 2022 at 5:10 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

Show 11 quoted lines
> On that platform, this also works:
>         diff --git a/t/chainlint.pl b/t/chainlint.pl
>         @@ -656,7 +656,7 @@ sub ncores {
>                 # Windows
>                 return $ENV{NUMBER_OF_PROCESSORS} if exists($ENV{NUMBER_OF_PROCESSORS});
>                 # Linux / MSYS2 / Cygwin / WSL
>         -       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^processor\s*:/, <>)); } if -r '/proc/cpuinfo';
>         +       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^(?:CPU\d+|processor\s*):/, <>)); } if -r '/proc/cpuinfo';
>                 # macOS & BSD
>                 return qx/sysctl -n hw.ncpu/ if $^O =~ /(?:^darwin$|bsd)/;
>                 return 1;
Thanks for digging into this. Your diagnosis matches the diagnosis in [1].
Show 6 quoted lines
> Anyway, the "fix" here can also be:
>
>          $jobs = ncores() if $jobs < 1;
>         +$jobs = 1 if $jobs < 1;
>
> I.e. it's another case of bad /proc/cpuinfo parsing.

Yup, I suggested the same in [2] as a would-be-nice-to-have-eventually patch, although only as a fallback, not as a proper fix for the reported problem since a real fix (as you have above and as in [1]) would return the correct core count.

[1]: https://lore.kernel.org/git/CANYNYEHXU8ivgAOa8EO5e9kOcbu6XF7rj+9EcSrbDQE+Rvyw_Q@mail.gmail.com/ [2]: https://lore.kernel.org/git/CAPig+cQ6_7wf6C280Rqi7mcTCiQp-n5GiLWTPazfcUcGFeZi0g@mail.gmail.com/

← back to recent threads