[FYI] How I compile on IRIX 6.5 with the MIPSpro compiler and ksh

This email describes how I have successfully compiled git on IRIX6.5 with the native MIPSpro compiler version 7.4.4m and successfully run nearly all of the tests using the Korn shell.

A series of 7 patches will follow this email. The patches work around a couple of flaws in the IRIX c99 compiler, and the not-quite-recent build environment.

A build script is provided at the end of this email.

The GIT_SKIP_TESTS environment variable was used to skip tests which still fail. The tests which still fail do so because of an ancient iconv, with one exception: the t5000 tests which fail are due to the installed gnu tar being too old.

Two lingering concerns:
  1) Building with -Ofast produced an executable which segfaulted and could not
     pass the tests.
  2) Building without THREADED_DELTA_SEARCH=1 produced an executable which
     segfaulted and could not pass the tests.

I have no debugger.

Otherwise, I've been using this executable somewhat and have not experienced any problems. Maybe it will be useful to others.

 Makefile                      |   12 ++++++++++++
 git-compat-util.h             |   12 +++++++++++-
 t/lib-git-svn.sh              |    3 +++
 t/t1002-read-tree-m-u-2way.sh |   10 ++++++----
 t/t9301-fast-export.sh        |    1 -
 t/test-lib.sh                 |    2 +-
 templates/Makefile            |    2 +-
 unpack-trees.c                |    3 ++-
 8 files changed, 36 insertions(+), 9 deletions(-)

-brandon

#!/bin/sh

GIT_SKIP_TESTS='
   t3900.1[0-289] t3900.2[023]
   t3901.*
   t5000.1[0-24-689] t5000.2[01]
   t5100.[56] t5100.1[02]
   t9301.4
'

export GIT_SKIP_TESTS

# NO_C99_FORMAT is required since this c99 compiler does not understand %td, # and probably not %zu.

exec gmake \
    CC=c99 \
    CFLAGS='-n32 -O2' \
    SHELL_PATH='/bin/ksh' \
    TAR=gtar \
    NO_C99_FORMAT=1 \
    NO_CURL=1 \
    NO_OPENSSL=1 \
    NO_TCLTK=1 \
    NO_PERL_MAKEMAKER=1 \
    THREADED_DELTA_SEARCH=1

---
 Makefile |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

Don't define _XOPEN_SOURCE Do define _SGI_SOURCE Declare the _xpg5 versions of *snprintf() along with wrapper macros

Defining _XOPEN_SOURCE prevents many of the common functions and macros
from being defined. _Not_ setting _XOPEN_SOURCE, and instead setting
_SGI_SOURCE, provides all of the XPG4, XPG5, BSD, POSIX functions and
declarations, _BUT_ provides a horribly broken snprintf(). The provided
snprintf() can not be worked around using git's compat workaround, since
SGI's vsnprintf() always returns the number of characters written into
the string, instead of -1 which is what git's compat version expects.
SGI does have a working snprintf(), but it is only provided when
_NO_XOPEN5 evaluates to zero, and this only happens if _XOPEN_SOURCE is
defined which, as mentioned above, prevents many other common functions
and defines. The working *snprintf() functions are named _xpg5_vsnprintf()
and _xpg5_snprintf(), so declarations for these two functions were added
to git-compat-util.h and macros were added for vsnprintf and snprintf to
call these versions.
---
 git-compat-util.h |   12 +++++++++++-
 1 files changed, 11 insertions(+), 1 deletions(-)

The c99 MIPSpro Compiler version 7.4.4m on IRIX 6.5 does not properly
initialize run-time initialized arrays. An array which is initialized with
fewer elements than the length of the array should have the unitialized
elements initialized to zero. This compiler does perform proper
initialization when static initialization parameters are used. So, work
around this by initializing with static elements, followed by a simple
assignment.
---
 unpack-trees.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

---

Hmm, I'm thinking that this is probably only an issue when path is not absolute, since I didn't have any problems installing the executables. Possibly, a more general fix is to provide the absolute path here.

-brandon

 templates/Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

Brandon Casey <casey@nrlssc.navy.mil> writes:

I do not see absolutely any reason to use install there.

I have to wonder why 9907721 (templates/Makefile: don't depend on local umask setting, 2008-02-28) did not do this instead:

	$(QUIET)umask 022 && ls *--* 2>/dev/null | \
	while read boilerplate; \
	do \
		case "$$boilerplate" in *~) continue ;; esac && \
		dst=`echo "$$boilerplate" | sed -e 's|^this|.|;s|--|/|g'` && \
		dir=`expr "$$dst" : '\(.*\)/'` && \
		mkdir -p blt/$$dir && \
		case "$$boilerplate" in \
		*--) ;; \
		*) cp $$boilerplate blt/$$dst ;; \
		esac || exit; \
	done && \
	date >$@

Junio C Hamano wrote:

That works just fine for me.

-brandon

From: Junio C Hamano <gitster@pobox.com>

The native install on some platforms (namely IRIX 6.5) treats non-absolute paths as being relative to the root directory rather than relative to the current directory. Work around this by avoiding install in this case since it is unnecessary, and instead depend on the local umask setting and use mkdir.

Tested-by: Brandon Casey <casey@nrlssc.navy.mil>
---

Junio C Hamano wrote:

works for me.

-brandon

 templates/Makefile |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

In ksh, if trap is called within a function with 0 or EXIT as its signal, then the trap will be executed at the time the function returns. This causes a problem in the test functions since 'trap - exit' is called within the test_done function in order to remove the trap which calls die() on exit. This means trap has to be called from the scripts top-level. Do so using an alias.

Additionally, there is some strangeness with respect to aliases and
sourced script files; the alias hack doesn't work. So call 'trap - 0'
directly in lib-git-svn.sh before calling the test_done function.
---
 t/lib-git-svn.sh |    3 +++
 t/test-lib.sh    |    2 +-
 2 files changed, 4 insertions(+), 1 deletions(-)

Brandon Casey <casey@nrlssc.navy.mil> writes:

Your alias test_done that calls function test_done look ugly and confusing beyond words. Perhaps test_done() can instead set a global variable and die() can notice it instead, like this? I haven't bothered to change the other "trap - exit" but I think you got the idea...

I will try your suggestions.

I also hope you noticed that these patches followed an FYI message and were not meant for direct inclusion. I hope you noticed that since I don't want you to waste your time. Maybe I should have put FYI in the [PATCH] string.

That being said, I (as always) appreciate the comments.

-brandon

Junio C Hamano wrote:

Brandon Casey <casey@nrlssc.navy.mil> writes:

Yeah, I think I got the subtle distinction between [PATCH FYI], [PATCH] and ones with and without sign-offs.

I've read "trap" section of 1003.1 2004 three times but I do not see the exact definition of "EXIT" condition mentioned. Is the behaviour to consider "function return" the EXIT condition considered a "shortcoming" (aka bug)?

Junio C Hamano wrote:

Heh, so subtle as to be unintentional. I think I mucked up that posting. I should have labeled them all FYI and numbered them. I did hope there would be some discussion that would result in a followup patch for submission.

No, I think it is the defined behavior for the Korn shell.

The SunOS 5.7 man page states:

     If sig  is  0 or EXIT and the trap  statement is exe-
     cuted inside the body of a function, then the com-
     mand arg is executed after the function completes.
     If sig is 0 or EXIT for a trap   set  outside  any
     function, the command arg is executed on exit from
     the shell.

The IRIX 6.5 man page has the exact same wording.

On linux, the man page for the public domain Korn shell states

    Functions  defined with the function reserved word are treated differ-
    ently in the following ways from functions defined with the  ()  nota-
    tion:

        ...

      - The EXIT trap, if set in a function, will be executed after the
        function returns.

But then later in the trap section:

    The original Korn shell’s DEBUG trap and the  handling  of  ERR
    and EXIT traps in functions are not yet implemented.

If Korn shell is to be supported, it seems that trap on EXIT must be avoided within functions. I think the only one is in git-bisect.

-brandon

Junio C Hamano wrote:

Yes that works and is much clearer. Tested on solaris and irix.

-brandon

Brandon Casey wrote:

I spoke too soon. Failing tests do not terminate the testing.

ksh does not place the exit status of the shell in $? or provide it as an argument to the trap function. Possibly the status of the 'exit' command is provided instead?

$ cat test.sh #!/bin/sh

die () {
    status=$?
    echo >&2 "die status: $status"
    exit $status
}

trap 'die' 0

echo "I am dying" exit 1

###### END TEST SCRIPT ######

linux $ /bin/sh test.sh I am dying die status: 1 linux $ echo $? 1

linux $ /usr/bin/ksh test.sh I am dying die status: 1 linux $ echo $? 1

IRIX $ /bin/ksh test.sh I am dying die status: 0 IRIX $ echo $? 0

SunOS $ /usr/xpg4/bin/sh test.sh I am dying die status: 0 SunOS $ echo $? 0

Ah, no, it looks like the status of the command executed immediately before exit is what the trap has access to in $?. Adding a call to false before exit in the above script causes the korn scripts to exit with status 1 on IRIX and SunOS.

#!/bin/sh

die () {
    status=$?
    echo >&2 "die status: $status"
    exit $status
}

trap 'die' 0

echo "I am dying" false exit 1

-brandon

2008/8/20 Brandon Casey <casey@nrlssc.navy.mil>

Just to report the same findings with AIX 5.3's /bin/sh (which is /bin/ksh), /bin/ksh93 and even /bin/bsh

Mike

Some old platforms have an old diff which doesn't have the -U option. 'git diff' can be used in its place. Adjust the comparison function to strip git's additional header lines to make this possible.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
---
 t/t1002-read-tree-m-u-2way.sh |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)

---

This is necessary if t9301.4 is included in the GIT_SKIP_TESTS environment variable.

-brandon

 t/t9301-fast-export.sh |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

Brandon Casey <casey@nrlssc.navy.mil> writes:

If it is not skipped what happens?  Does your change break this test?

How about attempting to unset always (because the test does not want to have the configuration) but ignore the result from the command (because the config may not have had anything to unset to begin with depending on what happened in the previous tests)?

Junio C Hamano wrote:

Yes, it was only meant as an example to be used along with the build script I supplied.

Yes, unsetting always and ignoring the result would allow this test to pass. It could also be moved into its own test so that it could be skipped using GIT_SKIP_TESTS, but still signal failure when it is not skipped.

-brandon

On platforms with a broken iconv it may be necessary to skip the fourth test in this script. Test four sets the i18n.commitencoding config variable. If test four is skipped, then a later unset of the i18n.commitencoding config variable will return a non-zero exit status and cause a test to fail. So move the 'config --unset' into its own test function which allows it to be skipped independently.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
---
 t/t9301-fast-export.sh |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

Brandon Casey <casey@nrlssc.navy.mil> writes:

I do not know if this is worth it, and I am reasonably sure this is not an optimal solution for this particular case.

Many existing pieces in the test scripts, I even suspect majority of them, do depend on earlier piece in the sequence to succeed and try to build on top of the state they have left. In the ideal world, especially when the software is young and flaky, perhaps that should not be the case and having test pieces that are as independent as each other would be easier to fix many tests that do not pass, but these days we expect all tests to pass, so running tests has become like reading the error output from compilers --- very often, the first error is the only one that counts.

Even though test-lib.sh does support skipping individual test pieces by adding t9301.12 to GIT_TEST_SKIP, it is not very useful in practice with the current set of tests, it would involve huge effort to make it reliably usable, and individual test pieces in the end result from such an effort will have to perform their own set-up from scratch, which would mean the test will take even longer to run, to catch occasional breakages.

For this particular case, what we are interested in testing is not that "config --unset" exits with 0 status. We are however interested in making sure that i18n.commitencoding is not set when the body of #12 runs.

So I think a more appropriate change would be something like this for this particular case.

 t/t9301-fast-export.sh |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

Junio C Hamano <gitster@pobox.com> writes:

Having said that, we may want to have an easier way to exclude certain classes of pieces, and also encourage test writers to group pieces that are related to these classes together.

For example, this introduces a new environment you can set, GIT_SKIP_TEST_CLASS, which is a space separated list of classes of features that you would want to exclude from the test. test_expect_success/failure can now take an optional "class token" as the first parameter (they traditionally took only two parameters, but with class token, they take three).

This example defines I18N class, and lets you exclude the one you were manually excluding with "GIT_SKIP_TESTS=t9301.4"

 t/t9301-fast-export.sh |    2 +-
 t/test-lib.sh          |   50 ++++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 45 insertions(+), 7 deletions(-)

Junio C Hamano schrieb:

Nice idea. Another class would be the tests that depend on that the filesystem supports symbolic links.

How do the tests look like if this token is the *last* argument?

To continue the idea, please look into t5302-pack-index.sh: We skip some tests if we don't have support for 64bit file offsets. Making these tests a "static" class would not be appropriate because the condition whether 64bit support is present is derived dynamically by the testsuite. What if we could write tests like this:

test_expect_success \
    'index v2: verify a pack with some 64-bit offsets' \
    'git verify-pack -v "test-3-${pack3}.pack"' \
    'test "$have_64bits"'

i.e. the 3rd argument is a condition that tells whether the test should be run. And in other cases the 3rd argument is the token that you propose:

test_expect_success 'iso-8859-1' '

     ...test goes here...

' I18N

Hm?

-- Hannes

Johannes Sixt <j.sixt@viscovery.net> writes:

I thought about it but rejected it because it is much easier to spot class tokens if it comes immediately after test_expect_xyzzy.

I suspect that certain classes of tests that need to be skipped can be autodetected inside test-lib.sh; it would be an independent topic to build on top of this. Your example of 64-bit may be one of them.

I hope these messages threaded correctly for everyone.

They show up correctly on gmane, but not for me in Thunderbird.

For me, the Message-Id of the first message is:
  giNXZFTxzY3B65dQob7CwvwwfSKlZpw_60oz81RxU5UN3PsTT_3dMQ@cipher.nrlssc.navy.mil
and this is what I provide to git-send-email.

But when I receive a message sent in such a way back from vger (i.e. using git-send-email to set In-Reply-To), the In-Reply-To field looks like:

  48A9FDCE.3070906@nrlssc.navy.mil

so thunderbird does not nest the replies beneath the original message.

-brandon

I would have liked to see 

Subject : patch [0/7] How to compile IRIX 6.5 w/ MIPSpro compiler/ksh ... Subject : patch [7/7] How to compile IRIX 6.5 w/ MIPSpro compiler/ksh

It would be easier to follow all your patches with all the other email on the lists.

Thanks,

-- Boyd Gerber <gerberb@zenez.com> ZENEZ 1042 East Fort Union #135, Midvale Utah 84047

Boyd Lynn Gerber wrote:

whoops, yeah, I should have done that. Sorry.

-brandon