threads / discuss / 27523

Unable to fork off sideband demultiplexer

Subject: Unable to fork off sideband demultiplexer

## tl;dr

15 messages between Jun 1, 2011 and Jun 4, 2011.

replies: 14people: 5as markdown or json

Randy Brandenburg· Jun 1, 2011, 14:16 UTC · lore

I have searched high and low to find a resolution to this error I am seeing when attempting to push or pull from a repository (either remote or on the same machine). The error reported is as follows:

# git push error: cannot create thread: Unknown error fatal: fetch-pack: Unable to fork off sideband demultiplexer Broken pipe

# git pull error: cannot create thread: Unknown error fatal: receive-pack: Unable to fork off sideband demultiplexer Broken pipe

This is a Solaris 9 installation (git-1.7.5-sol9-sparc-local.gz obtained from 
Sunfreeware.com) including all of the required package dependencies listed. 
I followed the same install on a Solaris 10 box and have no issues. Not sure if 
this is a permissions issue or something not configured 
correctly.
 
Any help is greatly appreciated. 
Junio C Hamano· Jun 1, 2011, 17:15 UTC · re: Randy Brandenburg · lore

Re: Unable to fork off sideband demultiplexer

Randy Brandenburg <randy.brandenburg@woh.rr.com> writes:
> # git push
> error: cannot create thread: Unknown error 

Perhaps threading library does not really work on your platform? Does rebuilding git with "make NO_PTHREADS=YesPlease" help?

Jeff King· Jun 1, 2011, 17:35 UTC · re: Junio C Hamano · lore

Re: Unable to fork off sideband demultiplexer

On Wed, Jun 01, 2011 at 10:15:54AM -0700, Junio C Hamano wrote:
Show 7 quoted lines
> Randy Brandenburg <randy.brandenburg@woh.rr.com> writes:
> 
> > # git push
> > error: cannot create thread: Unknown error 
> 
> Perhaps threading library does not really work on your platform? Does
> rebuilding git with "make NO_PTHREADS=YesPlease" help?

I suspect it is more subtle than that. We've had several people recently reporting the same issue, and all are using pre-built binaries on Solaris 9. Given the weird errno value ("unknown error"), my guess is that the packages are built on Solaris 10, and there is some ABI incompatibility between the two platforms.

I'd be curious if building on Solaris 9, even without NO_PTHREADS set, fixes the issue.

-Peff
Randy Brandenburg· Jun 1, 2011, 19:10 UTC · re: Jeff King · lore

Re: Unable to fork off sideband demultiplexer

Jeff King <peff <at> peff.net> writes:
Show 12 quoted lines
> 
> I suspect it is more subtle than that. We've had several people recently
> reporting the same issue, and all are using pre-built binaries on
> Solaris 9. Given the weird errno value ("unknown error"), my guess is
> that the packages are built on Solaris 10, and there is some ABI
> incompatibility between the two platforms.
> 
> I'd be curious if building on Solaris 9, even without NO_PTHREADS set,
> fixes the issue.
> 
> -Peff
> 

Thank you for the insight. I will attempt to rebuild on the target platform and see what happens.

I will post the results when I have an answer on whther that helped or not.
Randy Brandenburg· Jun 2, 2011, 15:29 UTC · re: Randy Brandenburg · lore

Re: Unable to fork off sideband demultiplexer

Randy Brandenburg <randy.brandenburg <at> woh.rr.com> writes:
Show 16 quoted lines
> 
> Jeff King <peff <at> peff.net> writes:
> > 
> > I suspect it is more subtle than that. We've had several people recently
> > reporting the same issue, and all are using pre-built binaries on
> > Solaris 9. Given the weird errno value ("unknown error"), my guess is
> > that the packages are built on Solaris 10, and there is some ABI
> > incompatibility between the two platforms.
> > 
> > I'd be curious if building on Solaris 9, even without NO_PTHREADS set,
> > fixes the issue.
> > 
> > -Peff
> > 
> 
> Thank you for the insight. I will attempt to rebuild on the target platform 
and 
Show 5 quoted lines
> see what happens. 
> 
> I will post the results when I have an answer on whther that helped or not.
> 
> 

I rebuilt git-1.7.5 from source on the Solaris 9 platform with gcc-3.4.6, GNU make 3.82, and GNU install (from coreutils 8.11). Following these steps

---------------------------------------------------------------------------- # ./configure CC=gcc

Edit the Makefile - set the path to "ar", "gcc" and GNU "install". CC = /usr/local/bin/gcc AR = /usr/ccs/bin/ar INSTALL = /usr/local/bin/install Look for ifeq ($(uname_S),SunOS) and set INSTALL to /usr/lcoal/bin/install

# gmake NO_TCLTK=1 NO_CURL=1 NO_PTHREADS=YesPlease install prefix=/opt/sfw # git --version git version 1.7.5 ---------------------------------------------------------------------------- Preliminary testing indicates a working system -- will investigate thourough today and tomorrow.

Thanks for the tips!
- Randy
Jeff King· Jun 2, 2011, 19:29 UTC · re: Randy Brandenburg · lore

Re: Unable to fork off sideband demultiplexer

On Thu, Jun 02, 2011 at 03:29:04PM +0000, Randy Brandenburg wrote:
Show 5 quoted lines
> I rebuilt git-1.7.5 from source on the Solaris 9 platform with gcc-3.4.6, GNU 
> make 3.82, and GNU install (from coreutils 8.11). Following these steps
> 
> ----------------------------------------------------------------------------
> # ./configure CC=gcc

Hmm, I would have thought configure would set up most of what you needed below, but it's possible our autoconf is not very complete (unlike many projects, it's not necessary to use it to build).

Show 5 quoted lines
> Edit the Makefile - set the path to "ar", "gcc" and GNU "install".
> CC = /usr/local/bin/gcc
> AR = /usr/ccs/bin/ar
> INSTALL = /usr/local/bin/install
> Look for ifeq ($(uname_S),SunOS) and set INSTALL to /usr/lcoal/bin/install

Rather than edit the Makefile, you can put these in "config.mak" which is sourced automatically by the Makefile (after the defaults are set up, so you can override them).

Show 6 quoted lines
> # gmake NO_TCLTK=1 NO_CURL=1 NO_PTHREADS=YesPlease install prefix=/opt/sfw
> # git --version
> git version 1.7.5
> ----------------------------------------------------------------------------
> Preliminary testing indicates a working system -- will investigate thourough 
> today and tomorrow.

Great, I'm glad it's working. With NO_PTHREADS set, you will be fork()ing off the sideband process instead of using a thread. So you avoid pthreads altogether and it works, but you can't do multi-threaded delta compression.

If you build without NO_PTHREADS set on a Solaris 9 box, does it work on that same box? That would confirm or deny my ABI compatibility theory (and hopefully point us in the right direction for the sunfreeware people to build a binary that works on Solaris 9 and 10).

-Peff
Randy Brandenburg· Jun 3, 2011, 12:32 UTC · re: Jeff King · lore

Re: Unable to fork off sideband demultiplexer

Jeff King <peff <at> peff.net> writes:
Show 8 quoted lines
> 
> If you build without NO_PTHREADS set on a Solaris 9 box, does it work on
> that same box? That would confirm or deny my ABI compatibility theory
> (and hopefully point us in the right direction for the sunfreeware
> people to build a binary that works on Solaris 9 and 10).
> 
> -Peff
> 

I will attempt building without NO_PTHREADS set today along with the other suggestions about modifying the config.mak instead of the actual Makefile. Stay tuned.

- Randy
Randy Brandenburg· Jun 3, 2011, 13:30 UTC · re: Jeff King · lore

Re: Unable to fork off sideband demultiplexer

Jeff King <peff <at> peff.net> writes:
Show 10 quoted lines
> Great, I'm glad it's working. With NO_PTHREADS set, you will be
> fork()ing off the sideband process instead of using a thread. So you
> avoid pthreads altogether and it works, but you can't do multi-threaded
> delta compression.
> 
> If you build without NO_PTHREADS set on a Solaris 9 box, does it work on
> that same box? That would confirm or deny my ABI compatibility theory
> (and hopefully point us in the right direction for the sunfreeware
> people to build a binary that works on Solaris 9 and 10).
> 

Building without NO_PTHREADS set results in the original "broken" behaviour. I am staying with the working version in the interest of time. May investigate more later.

Thanks for all the insight and tips.
- Randy
Jeff King· Jun 3, 2011, 14:49 UTC · re: Randy Brandenburg · lore

Re: Unable to fork off sideband demultiplexer

On Fri, Jun 03, 2011 at 01:30:00PM +0000, Randy Brandenburg wrote:
Show 10 quoted lines
> > If you build without NO_PTHREADS set on a Solaris 9 box, does it
> > work on that same box? That would confirm or deny my ABI
> > compatibility theory (and hopefully point us in the right direction
> > for the sunfreeware people to build a binary that works on Solaris 9
> > and 10).
> > 
> 
> Building without NO_PTHREADS set results in the original "broken"
> behaviour. I am staying with the working version in the interest of
> time. May investigate more later.

Thanks, that's a good data point. I know you are probably out of time for fooling with such things, but if you get a chance, could you try building also with "-pthreads" in PTHREAD_CFLAGS? I'm wondering if the problem is as simple as that.

-Peff
Randy Brandenburg· Jun 3, 2011, 18:52 UTC · re: Jeff King · lore

Re: Unable to fork off sideband demultiplexer

Jeff King <peff <at> peff.net> writes:
Show 8 quoted lines
> 
> Thanks, that's a good data point. I know you are probably out of time
> for fooling with such things, but if you get a chance, could you try
> building also with "-pthreads" in PTHREAD_CFLAGS? I'm wondering if the
> problem is as simple as that.
> 
> -Peff
> 

OK, rebuilt with NO_PTHREADS=0 and PTHREAD_CFLAGS=-pthreads and it seems to be working. Just tested quickly for the basics (push and pull). I will be able to test more in depth next week, but I would assume that all will be okay.

Hope this helps.
-Randy
Jeff King· Jun 3, 2011, 19:24 UTC · re: Randy Brandenburg · lore

Re: Unable to fork off sideband demultiplexer

On Fri, Jun 03, 2011 at 06:52:32PM +0000, Randy Brandenburg wrote:
> OK, rebuilt with NO_PTHREADS=0 and PTHREAD_CFLAGS=-pthreads and it seems to be 
> working. Just tested quickly for the basics (push and pull). I will be able to 
> test more in depth next week, but I would assume that all will be
> okay.

I think that will still build without pthreads at all, as we check "ifdef NO_PTHREADS" in the Makefile. You have to use "NO_PTHREADS=".

-Peff
Brandon Casey· Jun 3, 2011, 22:18 UTC · re: Jeff King · lore

Re: Unable to fork off sideband demultiplexer

On 06/03/2011 09:49 AM, Jeff King wrote:
Show 17 quoted lines
> On Fri, Jun 03, 2011 at 01:30:00PM +0000, Randy Brandenburg wrote:
> 
>>> If you build without NO_PTHREADS set on a Solaris 9 box, does it
>>> work on that same box? That would confirm or deny my ABI
>>> compatibility theory (and hopefully point us in the right direction
>>> for the sunfreeware people to build a binary that works on Solaris 9
>>> and 10).
>>>
>>
>> Building without NO_PTHREADS set results in the original "broken"
>> behaviour. I am staying with the working version in the interest of
>> time. May investigate more later.
> 
> Thanks, that's a good data point. I know you are probably out of time
> for fooling with such things, but if you get a chance, could you try
> building also with "-pthreads" in PTHREAD_CFLAGS? I'm wondering if the
> problem is as simple as that.

I generally build on 5.7 using Sun WSPro compilers and then test that binary on 5.7 and 5.9. The latest that I have compiled is 1.7.5.3 and I haven't seen this thread error.

Also note, I only do the above for testing purposes (and fun :). Solaris is not a primary development platform, so it does not get a lot of active testing in practice. The 5.9 system doesn't even have a compiler installed, so I can't try to duplicate this problem.

-Brandon
Ben Walton· Jun 2, 2011, 19:36 UTC · re: Randy Brandenburg · lore

Re: Unable to fork off sideband demultiplexer

Excerpts from Randy Brandenburg's message of Wed Jun 01 10:16:32 -0400 2011:
Hi Randy,
Show 5 quoted lines
> This is a Solaris 9 installation (git-1.7.5-sol9-sparc-local.gz
> obtained from Sunfreeware.com) including all of the required package
> dependencies listed.  I followed the same install on a Solaris 10
> box and have no issues. Not sure if this is a permissions issue or
> something not configured correctly.

Have you considered the OpenCSW git[1] packages? I'm a little stale at 1.7.3 right now but I have 1.7.5 ready to go...

It's built with thread support and should work on both 9 and 10.

Thanks -Ben

[1] http://opencsw.org/packages/git -- Ben Walton Systems Programmer - CHASS University of Toronto C:416.407.5610 | W:416.978.4302

Randy Brandenburg· Jun 3, 2011, 12:29 UTC · re: Ben Walton · lore

Re: Unable to fork off sideband demultiplexer

Ben Walton <bwalton <at> artsci.utoronto.ca> writes:
Hey Ben,
> 
> Have you considered the OpenCSW git[1] packages?  I'm a little stale
> at 1.7.3 right now but I have 1.7.5 ready to go...
> 

I did download the 1.7.3.2 version using pkg-get as the next option if I couldnt get the 1.7.5 to build and install. I would be all for a working package install if available, since we will have to install Git on a developer machine each time one is upgraded from a standard install of Solaris 9. I would like to add a package to our install instead of actually building from source each time.

Please keep me posted when the package is available for download!

Thanks, Randy Brandenburg

Ben Walton· Jun 4, 2011, 19:32 UTC · re: Randy Brandenburg · lore

Re: Unable to fork off sideband demultiplexer

Excerpts from Randy Brandenburg's message of Fri Jun 03 08:29:21 -0400 2011:
Hi Randy,
> I did download the 1.7.3.2 version using pkg-get as the next option
Try pkgutil.  It's much more capable. :)
> Please keep me posted when the package is available for download!

I'll push these for release shortly, but in the meantime, you can get 1.7.5.4 packages here:

http://buildfarm.opencsw.org/experimental.html#git

Thanks -Ben -- Ben Walton Systems Programmer - CHASS University of Toronto C:416.407.5610 | W:416.978.4302

← back to recent threads