v2, 6 partsBypass GCC attributes on NonStop platform where used.

> From: "Randall S. Becker" <rsbecker@nexbridge.com>
>
> * remote.c: force ignoring of GCC __attribute construct not supported
>   by c99 by defining it as an empty CPP macro.
>
> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>

> ---
>  remote.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/remote.c b/remote.c
> index 4e93753e1..c18f9de7f 100644
> --- a/remote.c
> +++ b/remote.c
> @@ -11,6 +11,10 @@
>  #include "mergesort.h"
>  #include "argv-array.h"
>
> +#if defined (__TANDEM)
> +#define __attribute(a)
> +#endif
> +
>  enum map_direction { FROM_SRC, FROM_DST };
>
>  static struct refspec s_tag_refspec = {
> --
> 2.16.0.31.gf1a482c
>

> From: "Randall S. Becker" <rsbecker@nexbridge.com>
> 
> * remote.c: force ignoring of GCC __attribute construct not supported
>   by c99 by defining it as an empty CPP macro.
> 
> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> ---
>  remote.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/remote.c b/remote.c
> index 4e93753e1..c18f9de7f 100644
> --- a/remote.c
> +++ b/remote.c
> @@ -11,6 +11,10 @@
>  #include "mergesort.h"
>  #include "argv-array.h"
>  
> +#if defined (__TANDEM)
> +#define __attribute(a)
> +#endif
> +

> > diff --git a/remote.c b/remote.c
> > index 4e93753e1..c18f9de7f 100644
> > --- a/remote.c
> > +++ b/remote.c
> > @@ -11,6 +11,10 @@
> >  #include "mergesort.h"
> >  #include "argv-array.h"
> >  
> > +#if defined (__TANDEM)
> > +#define __attribute(a)
> > +#endif
> > +
> 
> Hmm, the only use of __attribute() I can find is in compat/regex/.
> In particular, there is no use of __attribute() in regex.c.
> [__attribute__() is used in regex.c]
> 
> Is this an old patch which is no longer required?
> 
> puzzled.

> On Fri, Jan 19, 2018 at 08:28:48PM +0000, Ramsay Jones wrote:
> 
>>> diff --git a/remote.c b/remote.c
>>> index 4e93753e1..c18f9de7f 100644
>>> --- a/remote.c
>>> +++ b/remote.c
>>> @@ -11,6 +11,10 @@
>>>  #include "mergesort.h"
>>>  #include "argv-array.h"
>>>  
>>> +#if defined (__TANDEM)
>>> +#define __attribute(a)
>>> +#endif
>>> +
>>
>> Hmm, the only use of __attribute() I can find is in compat/regex/.
>> In particular, there is no use of __attribute() in regex.c.
>> [__attribute__() is used in regex.c]
>>
>> Is this an old patch which is no longer required?
>>
>> puzzled.

> I'm puzzled, too. The actual gcc thing is __attribute__(), and we
> already turn that into a noop via macro expansion if __GNUC__ is not
> defined (in git-compat-util.h, but see below).
> 
> __attribute(), without the trailing underscores, is used internally by
> the regex compat code (but it also converts that into a noop on non-GNUC
> platforms)>

> However the logic in git-compat-util is weird:
> 
>   #if defined(__HP_cc) && (__HP_cc >= 61000)
>   #define NORETURN __attribute__((noreturn))
>   #define NORETURN_PTR
>   #elif defined(__GNUC__) && !defined(NO_NORETURN)
>   #define NORETURN __attribute__((__noreturn__))
>   #define NORETURN_PTR __attribute__((__noreturn__))
>   #elif defined(_MSC_VER)
>   #define NORETURN __declspec(noreturn)
>   #define NORETURN_PTR
>   #else
>   #define NORETURN
>   #define NORETURN_PTR
>   #ifndef __GNUC__
>   #ifndef __attribute__
>   #define __attribute__(x)
>   #endif
>   #endif
>   #endif
> 
> Most of the conditional is dealing with NORETURN, but then we stick the
> __attribute()__ handling in the "else" block. Is it possible that this
> platform triggers __HP_cc, but doesn't actually understand
> __attribute__?

> On 19/01/18 21:20, Jeff King wrote:
> > On Fri, Jan 19, 2018 at 08:28:48PM +0000, Ramsay Jones wrote:
> >
> >>> diff --git a/remote.c b/remote.c
> >>> index 4e93753e1..c18f9de7f 100644
> >>> --- a/remote.c
> >>> +++ b/remote.c
> >>> @@ -11,6 +11,10 @@
> >>>  #include "mergesort.h"
> >>>  #include "argv-array.h"
> >>>
> >>> +#if defined (__TANDEM)
> >>> +#define __attribute(a)
> >>> +#endif
> >>> +
> >>
> >> Hmm, the only use of __attribute() I can find is in compat/regex/.
> >> In particular, there is no use of __attribute() in regex.c.
> >> [__attribute__() is used in regex.c]
> >>
> >> Is this an old patch which is no longer required?
> >>
> >> puzzled.
> 
> heh, I only just noticed that I (twice) wrote regex.c when I meant remote.c
> instead. Hopefully, that didn't cause too much confusion!
> 
> > I'm puzzled, too. The actual gcc thing is __attribute__(), and we
> > already turn that into a noop via macro expansion if __GNUC__ is not
> > defined (in git-compat-util.h, but see below).
> >
> > __attribute(), without the trailing underscores, is used internally by
> > the regex compat code (but it also converts that into a noop on
> > non-GNUC platforms)>
> 
> Indeed.
> 
> > However the logic in git-compat-util is weird:
> >
> >   #if defined(__HP_cc) && (__HP_cc >= 61000)
> >   #define NORETURN __attribute__((noreturn))
> >   #define NORETURN_PTR
> >   #elif defined(__GNUC__) && !defined(NO_NORETURN)
> >   #define NORETURN __attribute__((__noreturn__))
> >   #define NORETURN_PTR __attribute__((__noreturn__))
> >   #elif defined(_MSC_VER)
> >   #define NORETURN __declspec(noreturn)
> >   #define NORETURN_PTR
> >   #else
> >   #define NORETURN
> >   #define NORETURN_PTR
> >   #ifndef __GNUC__
> >   #ifndef __attribute__
> >   #define __attribute__(x)
> >   #endif
> >   #endif
> >   #endif
> >
> > Most of the conditional is dealing with NORETURN, but then we stick
> > the __attribute()__ handling in the "else" block. Is it possible that
> > this platform triggers __HP_cc, but doesn't actually understand
> > __attribute__?
> 
> That seems unlikely. However, that conditional looks a mess ... ;-)

>  This caused tests to seem to fail
>   while actually succeeding.
>
> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> ---
>  t/lib-git-daemon.sh | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/t/lib-git-daemon.sh b/t/lib-git-daemon.sh
> index 987d40680..955beecd9 100644
> --- a/t/lib-git-daemon.sh
> +++ b/t/lib-git-daemon.sh
> @@ -68,6 +68,7 @@ start_git_daemon() {
>                 test_skip_or_die $GIT_TEST_GIT_DAEMON \
>                         "git daemon failed to start"
>         fi
> +       trap '' EXIT
>  }
>
>  stop_git_daemon() {
> @@ -89,4 +90,6 @@ stop_git_daemon() {
>         fi
>         GIT_DAEMON_PID=
>         rm -f git_daemon_output
> +
> +       trap '' EXIT
>  }
> --
> 2.16.0.31.gf1a482c
>

> From: "Randall S. Becker" <rsbecker@nexbridge.com>
> 
> * t/lib-git-daemon.sh: fix incompatibilities with ksh traps not being
>   cleared automatically on platform. This caused tests to seem to fail
>   while actually succeeding.
> 
> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> ---
>  t/lib-git-daemon.sh | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/t/lib-git-daemon.sh b/t/lib-git-daemon.sh
> index 987d40680..955beecd9 100644
> --- a/t/lib-git-daemon.sh
> +++ b/t/lib-git-daemon.sh
> @@ -68,6 +68,7 @@ start_git_daemon() {
>  		test_skip_or_die $GIT_TEST_GIT_DAEMON \
>  			"git daemon failed to start"
>  	fi
> +	trap '' EXIT
>  }

> On Fri, Jan 19, 2018 at 12:34:04PM -0500, randall.s.becker@rogers.com wrote:
> 
> > From: "Randall S. Becker" <rsbecker@nexbridge.com>
> >
> > * t/lib-git-daemon.sh: fix incompatibilities with ksh traps not being
> >   cleared automatically on platform. This caused tests to seem to fail
> >   while actually succeeding.
> >
> > Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> > ---
> >  t/lib-git-daemon.sh | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/t/lib-git-daemon.sh b/t/lib-git-daemon.sh index
> > 987d40680..955beecd9 100644
> > --- a/t/lib-git-daemon.sh
> > +++ b/t/lib-git-daemon.sh
> > @@ -68,6 +68,7 @@ start_git_daemon() {
> >  		test_skip_or_die $GIT_TEST_GIT_DAEMON \
> >  			"git daemon failed to start"
> >  	fi
> > +	trap '' EXIT
> >  }
> 
> I don't think this can be right. The way these traps are supposed to work is:
> 
>   - as soon as test-lib.sh is loaded, we "trap die EXIT" to catch an
>     accidental death/exit from one of the scripts
> 
>   - when test_done is called, we clear the trap (i.e., it is OK to exit
>     now because the script has given us a positive indication that all
>     tests have been run)
> 
>   - while the child git-daemon is running, we'd want to clean up after
>     ourselves. So during start_git_daemon() we add our cleanup (followed
>     by the original "die", because shell traps don't push onto a stack).
>     And then at stop_git_daemon(), we restore the original die trap.
> 
> But this patch means that while git-daemon is running, we have no trap at all!
> That means that a failed test which causes us to exit would leave a child
> daemon running.
> 
> Furthermore, both of these functions now drop the 'die' trap entirely,
> meaning the script would fail to notice premature exit from one of the test
> snippets.
> 
> So while this may be papering over a problem on NonStop, I think it's
> breaking the intent of the traps.
> 
> I'm not sure what the root of the problem is, but it sounds like ksh may be
> triggering EXIT traps when we don't expect (during function returns?
> Subshell exits? Other?)

> On January 19, 2018 4:27 PM, Jeff King wrote:
> > On Fri, Jan 19, 2018 at 12:34:04PM -0500, randall.s.becker@rogers.com
> wrote:
> >
> > > From: "Randall S. Becker" <rsbecker@nexbridge.com>
> > >
> > > * t/lib-git-daemon.sh: fix incompatibilities with ksh traps not being
> > >   cleared automatically on platform. This caused tests to seem to fail
> > >   while actually succeeding.
> > >
> > > Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> > > ---
> > >  t/lib-git-daemon.sh | 3 +++
> > >  1 file changed, 3 insertions(+)
> > >
> > > diff --git a/t/lib-git-daemon.sh b/t/lib-git-daemon.sh index
> > > 987d40680..955beecd9 100644
> > > --- a/t/lib-git-daemon.sh
> > > +++ b/t/lib-git-daemon.sh
> > > @@ -68,6 +68,7 @@ start_git_daemon() {
> > >  		test_skip_or_die $GIT_TEST_GIT_DAEMON \
> > >  			"git daemon failed to start"
> > >  	fi
> > > +	trap '' EXIT
> > >  }
> >
> > I don't think this can be right. The way these traps are supposed to work is:
> >
> >   - as soon as test-lib.sh is loaded, we "trap die EXIT" to catch an
> >     accidental death/exit from one of the scripts
> >
> >   - when test_done is called, we clear the trap (i.e., it is OK to exit
> >     now because the script has given us a positive indication that all
> >     tests have been run)
> >
> >   - while the child git-daemon is running, we'd want to clean up after
> >     ourselves. So during start_git_daemon() we add our cleanup (followed
> >     by the original "die", because shell traps don't push onto a stack).
> >     And then at stop_git_daemon(), we restore the original die trap.
> >
> > But this patch means that while git-daemon is running, we have no trap at
> all!
> > That means that a failed test which causes us to exit would leave a
> > child daemon running.
> >
> > Furthermore, both of these functions now drop the 'die' trap entirely,
> > meaning the script would fail to notice premature exit from one of the
> > test snippets.
> >
> > So while this may be papering over a problem on NonStop, I think it's
> > breaking the intent of the traps.
> >
> > I'm not sure what the root of the problem is, but it sounds like ksh
> > may be triggering EXIT traps when we don't expect (during function
> returns?
> > Subshell exits? Other?)
> 
> The "unexpected" EXIT traps are exactly the issue we found when working
> with the platform support team. There was some discussion about what the
> right expectation was, and I was not able to have a change made to ksh on
> the platform. The problem may have a similar (identical?) root cause to the
> Perl exit issues we are also experiencing that is in their hands. I'm currently
> retesting without this change (results in 36 hours ☹ ). Is there a preferred
> way you would like me to bypass this except on NonStop? I can add a check
> based on uname.

> On January 19, 2018 4:27 PM, Jeff King wrote:
> > On Fri, Jan 19, 2018 at 12:34:04PM -0500, randall.s.becker@rogers.com
> wrote:
> >
> > > From: "Randall S. Becker" <rsbecker@nexbridge.com>
> > >
> > > * t/lib-git-daemon.sh: fix incompatibilities with ksh traps not being
> > >   cleared automatically on platform. This caused tests to seem to fail
> > >   while actually succeeding.
> > >
> > > Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> > > ---
> > >  t/lib-git-daemon.sh | 3 +++
> > >  1 file changed, 3 insertions(+)
> > >
> > > diff --git a/t/lib-git-daemon.sh b/t/lib-git-daemon.sh index
> > > 987d40680..955beecd9 100644
> > > --- a/t/lib-git-daemon.sh
> > > +++ b/t/lib-git-daemon.sh
> > > @@ -68,6 +68,7 @@ start_git_daemon() {
> > >  		test_skip_or_die $GIT_TEST_GIT_DAEMON \
> > >  			"git daemon failed to start"
> > >  	fi
> > > +	trap '' EXIT
> > >  }
> >
> > I don't think this can be right. The way these traps are supposed to work is:
> >
> >   - as soon as test-lib.sh is loaded, we "trap die EXIT" to catch an
> >     accidental death/exit from one of the scripts
> >
> >   - when test_done is called, we clear the trap (i.e., it is OK to exit
> >     now because the script has given us a positive indication that all
> >     tests have been run)
> >
> >   - while the child git-daemon is running, we'd want to clean up after
> >     ourselves. So during start_git_daemon() we add our cleanup (followed
> >     by the original "die", because shell traps don't push onto a stack).
> >     And then at stop_git_daemon(), we restore the original die trap.
> >
> > But this patch means that while git-daemon is running, we have no trap at
> all!
> > That means that a failed test which causes us to exit would leave a
> > child daemon running.
> >
> > Furthermore, both of these functions now drop the 'die' trap entirely,
> > meaning the script would fail to notice premature exit from one of the
> > test snippets.
> >
> > So while this may be papering over a problem on NonStop, I think it's
> > breaking the intent of the traps.
> >
> > I'm not sure what the root of the problem is, but it sounds like ksh
> > may be triggering EXIT traps when we don't expect (during function
> returns?
> > Subshell exits? Other?)
> 
> The "unexpected" EXIT traps are exactly the issue we found when working
> with the platform support team. There was some discussion about what the
> right expectation was, and I was not able to have a change made to ksh on
> the platform. The problem may have a similar (identical?) root cause to the
> Perl exit issues we are also experiencing that is in their hands. I'm currently
> retesting without this change (results in 36 hours ☹ ). Is there a preferred
> way you would like me to bypass this except on NonStop? I can add a check
> based on uname.

> From: "Randall S. Becker" <rsbecker@nexbridge.com>
> 
> * Makefile: Add TAR_EXTRACT_OPTIONS to allow platform options to be
>    specified if needed. The default is xof.
> 
> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> ---
>   Makefile | 6 +++++-
>   1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index 1a9b23b67..040e9eacd 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -429,6 +429,9 @@ all::
>   # running the test scripts (e.g., bash has better support for "set -x"
>   # tracing).
>   #
> +# Define TAR_EXTRACT_OPTIONS if you want to change the default behaviour
> +# from xvf to something else during installation.

> +#
>   # When cross-compiling, define HOST_CPU as the canonical name of the CPU on
>   # which the built Git will run (for instance "x86_64").
>   
> @@ -452,6 +455,7 @@ LDFLAGS =
>   ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS)
>   ALL_LDFLAGS = $(LDFLAGS)
>   STRIP ?= strip
> +TAR_EXTRACT_OPTIONS = xof
>   
>   # Create as necessary, replace existing, make ranlib unneeded.
>   ARFLAGS = rcs
> @@ -2569,7 +2573,7 @@ install: all
>   ifndef NO_GETTEXT
>   	$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(localedir_SQ)'
>   	(cd po/build/locale && $(TAR) cf - .) | \
> -	(cd '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR) xof -)
> +	(cd '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR) $(TAR_EXTRACT_OPTIONS) -)

> Am 19.01.2018 um 18:34 schrieb randall.s.becker@rogers.com:
> > From: "Randall S. Becker" <rsbecker@nexbridge.com>
> >
> > * Makefile: Add TAR_EXTRACT_OPTIONS to allow platform options to be
> >    specified if needed. The default is xof.
> >
> > Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> > ---
> >   Makefile | 6 +++++-
> >   1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/Makefile b/Makefile
> > index 1a9b23b67..040e9eacd 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -429,6 +429,9 @@ all::
> >   # running the test scripts (e.g., bash has better support for "set -x"
> >   # tracing).
> >   #
> > +# Define TAR_EXTRACT_OPTIONS if you want to change the default
> > +behaviour # from xvf to something else during installation.
> 
> "xof" instead of "xvf"?

> > +#
> >   # When cross-compiling, define HOST_CPU as the canonical name of the
> CPU on
> >   # which the built Git will run (for instance "x86_64").
> >
> > @@ -452,6 +455,7 @@ LDFLAGS =
> >   ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS)
> >   ALL_LDFLAGS = $(LDFLAGS)
> >   STRIP ?= strip
> > +TAR_EXTRACT_OPTIONS = xof
> >
> >   # Create as necessary, replace existing, make ranlib unneeded.
> >   ARFLAGS = rcs
> > @@ -2569,7 +2573,7 @@ install: all
> >   ifndef NO_GETTEXT
> >   	$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(localedir_SQ)'
> >   	(cd po/build/locale && $(TAR) cf - .) | \
> > -	(cd '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR) xof -)
> > +	(cd '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR)
> > +$(TAR_EXTRACT_OPTIONS) -)
> 
> Hmm.  TAR_EXTRACT_OPTIONS always needs to have f (or -f, or --file) at the
> end to go together with the following dash, meaning to extract from stdin.
> And x (or -x, or --extract) is probably needed in all cases as well.  So wouldn't
> it make more sense to only put the o (or -o, or
> --no-same-owner) into TAR_EXTRACT_OPTIONS and enforce x and f?

> 
> >   endif
> >   ifndef NO_PERL
> >   	$(MAKE) -C perl prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)'
> > install
> >

> On January 20, 2018 7:31 AM, René Scharfe wrote:
>> Am 19.01.2018 um 18:34 schrieb randall.s.becker@rogers.com:
>>> From: "Randall S. Becker" <rsbecker@nexbridge.com>
>>> 
>>> * Makefile: Add TAR_EXTRACT_OPTIONS to allow platform options to
>>> be specified if needed. The default is xof.
>>> 
>>> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com> --- 
>>> Makefile | 6 +++++- 1 file changed, 5 insertions(+), 1
>>> deletion(-)
>>> 
>>> diff --git a/Makefile b/Makefile index 1a9b23b67..040e9eacd
>>> 100644 --- a/Makefile +++ b/Makefile @@ -429,6 +429,9 @@ all:: #
>>> running the test scripts (e.g., bash has better support for "set
>>> -x" # tracing). # +# Define TAR_EXTRACT_OPTIONS if you want to
>>> change the default +behaviour # from xvf to something else during
>>> installation.
>> 
>> "xof" instead of "xvf"?
> 
> When I look at the parent commit, it says xof, so I wanted to
> preserve existing behaviour by default. Our install process wants to
> see the actual set of files, so we wanted to use xvof but that hardly
> seemed of general interest. I was hoping an option to control it
> would be agreeable.

> 
>>> +# # When cross-compiling, define HOST_CPU as the canonical name
>>> of the
>> CPU on
>>> # which the built Git will run (for instance "x86_64").
>>> 
>>> @@ -452,6 +455,7 @@ LDFLAGS = ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS) 
>>> ALL_LDFLAGS = $(LDFLAGS) STRIP ?= strip +TAR_EXTRACT_OPTIONS =
>>> xof
>>> 
>>> # Create as necessary, replace existing, make ranlib unneeded. 
>>> ARFLAGS = rcs @@ -2569,7 +2573,7 @@ install: all ifndef
>>> NO_GETTEXT $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(localedir_SQ)' 
>>> (cd po/build/locale && $(TAR) cf - .) | \ -	(cd
>>> '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR) xof -) +
>>> (cd '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR) 
>>> +$(TAR_EXTRACT_OPTIONS) -)
>> 
>> Hmm.  TAR_EXTRACT_OPTIONS always needs to have f (or -f, or --file)
>> at the end to go together with the following dash, meaning to
>> extract from stdin. And x (or -x, or --extract) is probably needed
>> in all cases as well.  So wouldn't it make more sense to only put
>> the o (or -o, or --no-same-owner) into TAR_EXTRACT_OPTIONS and
>> enforce x and f?
> 
> This is a good suggestion, and I'd love to do that, if I could
> guarantee a modern tar, which I can't. The platform comes with a
> really old-school tar from some old (seemingly BSD4.3) epoch that
> only takes one option set. There is a more modern tar that can be
> optionally installed if the sysadmin decides to that takes a slightly
> more modern set, which could support your request, and my team also
> has a gnu port that is very modern. I can't control what customers
> are choosing to have installed, unfortunately. Your point is well
> made and I am completely on board with it, but it introduces a
> configuration requirement.

> As with the broadening setbuf (patch 2/6) change, I would like to
> consider this for the future, having a slightly different more
> complex idea. I could introduce something like this:
> 
> 1. HAS_ANCIENT_TAR=UnfortunatelyYes in config.mak.uname that disables
> this capability all together 2. HAS_ANCIENT_TAR=AreYouKiddingMe
> (joke) then set up TAR_EXTRACT_ADDITIONAL_OPTIONS above and beyond
> the default, so --file, --no-same-owner would always be in effect for
> that operation.
> 
> The micro-project would also, logically, need to apply to other tar
> occurrences throughout the code and potentially need a test case
> written for it (not entirely sure what that would test, yet).
> Is that a reasonable approach?

> To: Randall S. Becker <rsbecker@nexbridge.com>; git@vger.kernel.org
> Subject: Re: [PATCH v2 2/6] Add tar extract install options override in
> installation processing.
> 
> Am 20.01.2018 um 14:44 schrieb Randall S. Becker:
> > On January 20, 2018 7:31 AM, René Scharfe wrote:
> >> Am 19.01.2018 um 18:34 schrieb randall.s.becker@rogers.com:
> >>> From: "Randall S. Becker" <rsbecker@nexbridge.com>
> >>>
> >>> * Makefile: Add TAR_EXTRACT_OPTIONS to allow platform options to be
> >>> specified if needed. The default is xof.
> >>>
> >>> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com> ---
> >>> Makefile | 6 +++++- 1 file changed, 5 insertions(+), 1
> >>> deletion(-)
> >>>
> >>> diff --git a/Makefile b/Makefile index 1a9b23b67..040e9eacd
> >>> 100644 --- a/Makefile +++ b/Makefile @@ -429,6 +429,9 @@ all:: #
> >>> running the test scripts (e.g., bash has better support for "set -x"
> >>> # tracing). # +# Define TAR_EXTRACT_OPTIONS if you want to change
> >>> the default +behaviour # from xvf to something else during
> >>> installation.
> >>
> >> "xof" instead of "xvf"?
> >
> > When I look at the parent commit, it says xof, so I wanted to preserve
> > existing behaviour by default. Our install process wants to see the
> > actual set of files, so we wanted to use xvof but that hardly seemed
> > of general interest. I was hoping an option to control it would be
> > agreeable.
> 
> Preserving the default is good. I meant that the default is "xof", but the
> added line implies it was "xvf" instead.
> 
> Seeing your actual use case confirms that my suggestion below would work
> for you.
> 
> >
> >>> +# # When cross-compiling, define HOST_CPU as the canonical name
> >>> of the
> >> CPU on
> >>> # which the built Git will run (for instance "x86_64").
> >>>
> >>> @@ -452,6 +455,7 @@ LDFLAGS = ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS)
> >>> ALL_LDFLAGS = $(LDFLAGS) STRIP ?= strip +TAR_EXTRACT_OPTIONS = xof
> >>>
> >>> # Create as necessary, replace existing, make ranlib unneeded.
> >>> ARFLAGS = rcs @@ -2569,7 +2573,7 @@ install: all ifndef NO_GETTEXT
> >>> $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(localedir_SQ)'
> >>> (cd po/build/locale && $(TAR) cf - .) | \ -	(cd
> >>> '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR) xof -) + (cd
> >>> '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR)
> >>> +$(TAR_EXTRACT_OPTIONS) -)
> >>
> >> Hmm.  TAR_EXTRACT_OPTIONS always needs to have f (or -f, or --file)
> >> at the end to go together with the following dash, meaning to extract
> >> from stdin. And x (or -x, or --extract) is probably needed in all
> >> cases as well.  So wouldn't it make more sense to only put the o (or
> >> -o, or --no-same-owner) into TAR_EXTRACT_OPTIONS and enforce x and
> f?
> >
> > This is a good suggestion, and I'd love to do that, if I could
> > guarantee a modern tar, which I can't. The platform comes with a
> > really old-school tar from some old (seemingly BSD4.3) epoch that only
> > takes one option set. There is a more modern tar that can be
> > optionally installed if the sysadmin decides to that takes a slightly
> > more modern set, which could support your request, and my team also
> > has a gnu port that is very modern. I can't control what customers are
> > choosing to have installed, unfortunately. Your point is well made and
> > I am completely on board with it, but it introduces a configuration
> > requirement.
> 
> Long options would be nice to nice to have, but are not my main point; I
> included them mainly to spare readers from firing up "man tar" to look up
> the meaning of the short ones.
> 
> I just meant to say that something like this here would make more sense
> because you always need x and f- anyway:
> 
> 	TAR_EXTRACT_OPTIONS = o
> 
> 	... ${TAR} x${TAR_EXTRACT_OPTIONS}f -
> 
> > As with the broadening setbuf (patch 2/6) change, I would like to
> > consider this for the future, having a slightly different more complex
> > idea. I could introduce something like this:
> >
> > 1. HAS_ANCIENT_TAR=UnfortunatelyYes in config.mak.uname that disables
> > this capability all together 2. HAS_ANCIENT_TAR=AreYouKiddingMe
> > (joke) then set up TAR_EXTRACT_ADDITIONAL_OPTIONS above and beyond
> the
> > default, so --file, --no-same-owner would always be in effect for that
> > operation.
> >
> > The micro-project would also, logically, need to apply to other tar
> > occurrences throughout the code and potentially need a test case
> > written for it (not entirely sure what that would test, yet).
> > Is that a reasonable approach?
> 
> As long as old-school dash-less flags suffice for our purposes (including
> yours) we can just keep using that style everywhere and avoid adding more
> settings.  It would be a different matter if we needed features that have no
> short flag, or are only offered by certain tar implementations.

> On Sat, Jan 20 2018, Randall S. Becker jotted:
> 
> > I will reissue this particular patch shortly.
> 
> It makes sense to base that submission on the next branch instead of master.
> I have a patch queued up there which adds two new tar invocations.
> 
> Also re your commit message see the formatting guide in
> Documentation/SubmittingPatches, in particular: instead of:
> 
>  - Add a brief subject line
> 
>  - Just make the body be a normal paragraph instead of a bullet-point
>    list with one item.

> From: "Randall S. Becker" <rsbecker@nexbridge.com>
>
> * wrapper.c: called setbuf(stream,0) to force pipe flushes not enabled by
>   default on the NonStop platform.
>

> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> ---
>  wrapper.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/wrapper.c b/wrapper.c
> index d20356a77..671cbb4b4 100644
> --- a/wrapper.c
> +++ b/wrapper.c
> @@ -403,6 +403,9 @@ FILE *xfdopen(int fd, const char *mode)
>         FILE *stream = fdopen(fd, mode);
>         if (stream == NULL)
>                 die_errno("Out of memory? fdopen failed");
> +#ifdef __TANDEM
> +       setbuf(stream,0);

> +#endif
>         return stream;
>  }
>
> --
> 2.16.0.31.gf1a482c
>

> From: "Randall S. Becker" <rsbecker@nexbridge.com>
> 
> * wrapper.c: called setbuf(stream,0) to force pipe flushes not enabled by
>   default on the NonStop platform.
> 
> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> ---
>  wrapper.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/wrapper.c b/wrapper.c
> index d20356a77..671cbb4b4 100644
> --- a/wrapper.c
> +++ b/wrapper.c
> @@ -403,6 +403,9 @@ FILE *xfdopen(int fd, const char *mode)
>  	FILE *stream = fdopen(fd, mode);
>  	if (stream == NULL)
>  		die_errno("Out of memory? fdopen failed");
> +#ifdef __TANDEM
> +	setbuf(stream,0);
> +#endif

>  	return stream;
>  }
>  
> -- 
> 2.16.0.31.gf1a482c
> 

> On Fri, Jan 19, 2018 at 12:33:59PM -0500, randall.s.becker@rogers.com
> wrote:
> > From: "Randall S. Becker" <rsbecker@nexbridge.com>
> >
> > * wrapper.c: called setbuf(stream,0) to force pipe flushes not enabled by
> >   default on the NonStop platform.
> >
> > Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> > ---
> >  wrapper.c | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/wrapper.c b/wrapper.c
> > index d20356a77..671cbb4b4 100644
> > --- a/wrapper.c
> > +++ b/wrapper.c
> > @@ -403,6 +403,9 @@ FILE *xfdopen(int fd, const char *mode)
> >  	FILE *stream = fdopen(fd, mode);
> >  	if (stream == NULL)
> >  		die_errno("Out of memory? fdopen failed");
> > +#ifdef __TANDEM
> > +	setbuf(stream,0);
> > +#endif
> 
> Reading the commit message, I would have expected someting similar to
> 
> #ifdef FORCE_PIPE_FLUSHES
> 	setbuf(stream,0);
> #endif
> 
> (Because other systems may need the tweak as well, some day) Of course
> you need to change that in the Makefile and config.mak.uname
> 
> >  	return stream;
> >  }

> On Fri, Jan 19, 2018 at 12:33:59PM -0500, randall.s.becker@rogers.com wrote:
>> From: "Randall S. Becker" <rsbecker@nexbridge.com>
>> 
>> * wrapper.c: called setbuf(stream,0) to force pipe flushes not enabled by
>>   default on the NonStop platform.
>> 
>> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
>> ---
>>  wrapper.c | 3 +++
>>  1 file changed, 3 insertions(+)
>> 
>> diff --git a/wrapper.c b/wrapper.c
>> index d20356a77..671cbb4b4 100644
>> --- a/wrapper.c
>> +++ b/wrapper.c
>> @@ -403,6 +403,9 @@ FILE *xfdopen(int fd, const char *mode)
>>  	FILE *stream = fdopen(fd, mode);
>>  	if (stream == NULL)
>>  		die_errno("Out of memory? fdopen failed");
>> +#ifdef __TANDEM
>> +	setbuf(stream,0);
>> +#endif
>
> Reading the commit message, I would have expected someting similar to
>
> #ifdef FORCE_PIPE_FLUSHES
> 	setbuf(stream,0);
> #endif
>
> (Because other systems may need the tweak as well, some day)
> Of course you need to change that in the Makefile and config.mak.uname

> Torsten Bögershausen <tboegi@web.de> writes:
> 
> > On Fri, Jan 19, 2018 at 12:33:59PM -0500, randall.s.becker@rogers.com
> wrote:
> >> From: "Randall S. Becker" <rsbecker@nexbridge.com>
> >>
> >> * wrapper.c: called setbuf(stream,0) to force pipe flushes not enabled by
> >>   default on the NonStop platform.
> >>
> >> Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
> >> ---
> >>  wrapper.c | 3 +++
> >>  1 file changed, 3 insertions(+)
> >>
> >> diff --git a/wrapper.c b/wrapper.c
> >> index d20356a77..671cbb4b4 100644
> >> --- a/wrapper.c
> >> +++ b/wrapper.c
> >> @@ -403,6 +403,9 @@ FILE *xfdopen(int fd, const char *mode)
> >>  	FILE *stream = fdopen(fd, mode);
> >>  	if (stream == NULL)
> >>  		die_errno("Out of memory? fdopen failed");
> >> +#ifdef __TANDEM
> >> +	setbuf(stream,0);
> >> +#endif
> >
> > Reading the commit message, I would have expected someting similar to
> >
> > #ifdef FORCE_PIPE_FLUSHES
> > 	setbuf(stream,0);
> > #endif
> >
> > (Because other systems may need the tweak as well, some day) Of course
> > you need to change that in the Makefile and config.mak.uname
> 
> I actually wouldn't have expected anything like that after reading the commit
> message.
> 
> First I thought it was describing only what it does (i.e. "let's use
> setbuf() to set the stream unbuffered on TANDEM"), which is a useless
> description that only says what it does which we can read from the diff, but
> "NonStop by default creates pipe that does not flush" is a potentially useful
> information the log message adds.
> But it is just "potentially"---we cannot read what exact problem the change is
> trying to address.  Making a pipe totally unbuffered is a heavy hammer that
> may not be an appropriate solution---it could be that we are missing calls to
> fflush() where we need and have been lucky because most of the systems
> we deal with do line-buffered by default, or something silly/implausible like
> that, and if that is the case, a more proper fix may be to add these missing
> fflush() to right places.
> 
> IOW, I do not see it explained clearly why this change is needed on any single
> platform---so "that issue may be shared by others, too"
> is a bit premature thing for me to listen to and understand, as "that issue" is
> quite unclear to me.

>> IOW, I do not see it explained clearly why this change is needed on any single
>> platform---so "that issue may be shared by others, too"
>> is a bit premature thing for me to listen to and understand, as "that issue" is
>> quite unclear to me.
>
> v4 might be a little better. The issue seems to be specific to
> NonStop that it's PIPE mechanism needs to have setbuf(pipe,NULL)
> called for git to be happy.  The default behaviour appears to be
> different on NonStop from other platforms from our testing. We get
> hung up waiting on pipes unless this is done.

> "Randall S. Becker" <rsbecker@nexbridge.com> writes:
> 
> >> IOW, I do not see it explained clearly why this change is needed on
> >> any single platform---so "that issue may be shared by others, too"
> >> is a bit premature thing for me to listen to and understand, as "that
> >> issue" is quite unclear to me.
> >
> > v4 might be a little better. The issue seems to be specific to NonStop
> > that it's PIPE mechanism needs to have setbuf(pipe,NULL) called for
> > git to be happy.  The default behaviour appears to be different on
> > NonStop from other platforms from our testing. We get hung up waiting
> > on pipes unless this is done.
> 
> I am afraid that that is not a "diagnosis" enough to allow us moving forward.
> We get hung up because...?  When the process that has the other end of
> pipe open exits, NonStop does not close the pipe properly?  Or NonStop
> does not flush the data buffered in the pipe?
> Would it help if a compat wrapper that does setbuf(fd, NULL) immediately
> before closing the fd, or some other more targetted mechanism, is used only
> on NonStop, for example?  Potentially megabytes of data can pass thru a
> pipe, and if the platform bug affects only at the tail end of the transfer,
> marking the pipe not to buffer at all at the beginning is too big a hammer to
> work it around.  With the explanation given so far, this still smells more like
> "we have futzed around without understanding why, and this happens to
> work."  It may be good enough for your purpose of making progress (after
> all, I'd imagine that you'd need to work this around one way or another to
> hunt for and fix more issues on the platform), but it does not sound like "we
> know what the problem is, and this is the best workaround for that", which is
> what we want if it wants to become part of the official codebase.

> 
> "Randall S. Becker" <rsbecker@nexbridge.com> writes:
> 
> >> IOW, I do not see it explained clearly why this change is needed on
> >> any single platform---so "that issue may be shared by others, too"
> >> is a bit premature thing for me to listen to and understand, as "that
> >> issue" is quite unclear to me.
> >
> > v4 might be a little better. The issue seems to be specific to NonStop
> > that it's PIPE mechanism needs to have setbuf(pipe,NULL) called for
> > git to be happy.  The default behaviour appears to be different on
> > NonStop from other platforms from our testing. We get hung up waiting
> > on pipes unless this is done.
> 
> I am afraid that that is not a "diagnosis" enough to allow us moving forward.
> We get hung up because...?  When the process that has the other end of
> pipe open exits, NonStop does not close the pipe properly?  Or NonStop
> does not flush the data buffered in the pipe?
> Would it help if a compat wrapper that does setbuf(fd, NULL) immediately
> before closing the fd, or some other more targetted mechanism, is used only
> on NonStop, for example?  Potentially megabytes of data can pass thru a
> pipe, and if the platform bug affects only at the tail end of the transfer,
> marking the pipe not to buffer at all at the beginning is too big a hammer to
> work it around.  With the explanation given so far, this still smells more like
> "we have futzed around without understanding why, and this happens to
> work."  It may be good enough for your purpose of making progress (after
> all, I'd imagine that you'd need to work this around one way or another to
> hunt for and fix more issues on the platform), but it does not sound like "we
> know what the problem is, and this is the best workaround for that", which is
> what we want if it wants to become part of the official codebase.