{"thread":{"id":"33908","subject":"git hangs on pthread_join","startedAt":"2013-05-23T13:01:43Z","lastAt":"2013-05-29T08:29:03Z","messageCount":4,"participants":["Ian Kumlien","Martin Fick","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"218262","messageId":"20130523130143.GA6376@pomac.netswarm.net","threadId":"33908","inReplyTo":null,"subject":"git hangs on pthread_join","fromName":"Ian Kumlien","fromEmail":"pomac@vapor.com","sentAt":"2013-05-23T13:01:43Z","receivedAt":"2013-05-23T13:01:43Z","isPatch":false,"sender":{"key":"pomac@vapor.com","avatar":null},"body":"Hi,\n\nI'm running a rather special configuration, basically i have a gerrit\nserver pushing\ngit data over openvpn connections (company regulations n' stuff)...\n\ngit 1.8.2.1 is started by xinetd\n...\n port = 9418\n socket_type = stream\n wait = no\n user = gerrit2\n server = /usr/bin/git\n server_args = daemon --inetd --syslog --export-all\n--enable=receive-pack --init-timeout=3 --timeout=180 --base-path=\n...\n nice = 10\n per_source = UNLIMITED\n instances = UNLIMITED\n flags = KEEPALIVE NODELAY\n---\n\nKeepalive and nodelay has been added post fact, the same goes for the\ntimeouts.\n\nI have found \"git receive-pack\"s that has been running for days/weeks\nwithout terminating....\n\nAttaching gdb and doing a trace results in:\n#0 0x0000003261207b35 in pthread_join () from /lib64/libpthread.so.0\n#1 0x00000000004ce58b in finish_async ()\n#2 0x000000000045744b in cmd_receive_pack ()\n#3 0x0000000000404851 in handle_internal_command ()\n#4 0x0000000000404c9d in main ()\n(sorry don't have any debug data for the binary packages apparenlty (rpms\nwas\nbuilt from the official source))\n\n(RHEL 5 machine with glibc 2.5-65.el5_7.1)\n\nAnyone that has any clues about what could be going wrong?\n"},{"id":"218302","messageId":"201305231345.38190.mfick@codeaurora.org","threadId":"33908","inReplyTo":"20130523130143.GA6376@pomac.netswarm.net","subject":"Re: git hangs on pthread_join","fromName":"Martin Fick","fromEmail":"mfick@codeaurora.org","sentAt":"2013-05-23T19:45:38Z","receivedAt":"2013-05-23T19:45:38Z","isPatch":false,"sender":{"key":"mfick@codeaurora.org","avatar":null},"body":"On Thursday, May 23, 2013 07:01:43 am you wrote:\n> \n> I'm running a rather special configuration, basically i\n> have a gerrit server pushing\n... \n> I have found \"git receive-pack\"s that has been running\n> for days/weeks without terminating....\n> \n... \n> Anyone that has any clues about what could be going\n> wrong? --\n\n\nHave you narrowed down whether this is a git client problem, \nor a server problem (gerrit in your case). Is this a \nrepeatable issue. Try the same operation against a clone of \nthe repo using just git. Check on the server side for .noz \nfiles in you repo (a jgit thing),\n\n-Martin\n"},{"id":"218687","messageId":"20130528175108.GA21210@sigill.intra.peff.net","threadId":"33908","inReplyTo":"20130523130143.GA6376@pomac.netswarm.net","subject":"Re: git hangs on pthread_join","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-05-28T17:51:09Z","receivedAt":"2013-05-28T17:51:09Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 23, 2013 at 03:01:43PM +0200, Ian Kumlien wrote:\n\n> git 1.8.2.1 is started by xinetd\n> [...]\n> I have found \"git receive-pack\"s that has been running for days/weeks\n> without terminating....\n> \n> Attaching gdb and doing a trace results in:\n> #0 0x0000003261207b35 in pthread_join () from /lib64/libpthread.so.0\n> #1 0x00000000004ce58b in finish_async ()\n> #2 0x000000000045744b in cmd_receive_pack ()\n> #3 0x0000000000404851 in handle_internal_command ()\n> #4 0x0000000000404c9d in main ()\n\nI recently fixed a deadlock that could happen in receive-pack when\nclients hung up before sending a valid pack header. The fix is commit\n49ecfa1, and it's in git v1.8.2.2.\n\nThe stack trace for the deadlock fixed by 49ecfa1 would have\nunpack_with_sideband between #1 and #2 above, but it is entirely\npossible that it is simply inlined in your build of git, depending on\nthe -O level of your build (it is a static function that is only called\nfrom one place). So it seems likely that it is the culprit.\n\n-Peff\n"},{"id":"218790","messageId":"20130529082903.GC22308@pomac.netswarm.net","threadId":"33908","inReplyTo":"20130528175108.GA21210@sigill.intra.peff.net","subject":"Re: git hangs on pthread_join","fromName":"Ian Kumlien","fromEmail":"pomac@vapor.com","sentAt":"2013-05-29T08:29:03Z","receivedAt":"2013-05-29T08:29:03Z","isPatch":false,"sender":{"key":"pomac@vapor.com","avatar":null},"body":"On Tue, May 28, 2013 at 01:51:09PM -0400, Jeff King wrote:\n> On Thu, May 23, 2013 at 03:01:43PM +0200, Ian Kumlien wrote:\n> \n> > git 1.8.2.1 is started by xinetd\n> > [...]\n> > I have found \"git receive-pack\"s that has been running for days/weeks\n> > without terminating....\n> > \n> > Attaching gdb and doing a trace results in:\n> > #0 0x0000003261207b35 in pthread_join () from /lib64/libpthread.so.0\n> > #1 0x00000000004ce58b in finish_async ()\n> > #2 0x000000000045744b in cmd_receive_pack ()\n> > #3 0x0000000000404851 in handle_internal_command ()\n> > #4 0x0000000000404c9d in main ()\n> \n> I recently fixed a deadlock that could happen in receive-pack when\n> clients hung up before sending a valid pack header. The fix is commit\n> 49ecfa1, and it's in git v1.8.2.2.\n\nWith dodgy connections this could easily happen =)\n\nReally nice catch!\n\n> The stack trace for the deadlock fixed by 49ecfa1 would have\n> unpack_with_sideband between #1 and #2 above, but it is entirely\n> possible that it is simply inlined in your build of git, depending on\n> the -O level of your build (it is a static function that is only called\n> from one place). So it seems likely that it is the culprit.\n\nYeah, since it's a RHEL 5 machine i don't even get a debug rpm package\n=P\n\nI will upgrade all machines and keep monitoring, thanks!\n\n> -Peff\n"}]}