{"thread":{"id":"21577","subject":"t5541-http-push hanging","startedAt":"2009-11-11T06:28:05Z","lastAt":"2009-11-12T05:18:26Z","messageCount":7,"participants":["Brian Gernhardt","Tay Ray Chuan","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"127294","messageId":"B17AB159-E217-4E1F-BEA3-97E5892C13F4@gernhardtsoftware.com","threadId":"21577","inReplyTo":null,"subject":"t5541-http-push hanging","fromName":"Brian Gernhardt","fromEmail":"brian@gernhardtsoftware.com","sentAt":"2009-11-11T06:28:05Z","receivedAt":"2009-11-11T06:28:05Z","isPatch":false,"sender":{"key":"brian@gernhardtsoftware.com","avatar":"https://avatars.githubusercontent.com/u/133455?v=4"},"body":"My build script for git has been hanging at t5541.2 and I haven't had the tuits to discover why.  Here's what I've gotten so far, in case anyone can figure it out faster:\n\n$ ./t5541-http-push.sh -v\nInitialized empty Git repository in /Users/brian/dev/git/t/trash directory.t5541-http-push/.git/\n* expecting success: \n\tcd \"$ROOT_PATH\" &&\n\tmkdir test_repo &&\n\tcd test_repo &&\n\tgit init &&\n\t: >path1 &&\n\tgit add path1 &&\n\ttest_tick &&\n\tgit commit -m initial &&\n\tcd - &&\n\tgit clone --bare test_repo test_repo.git &&\n\tcd test_repo.git &&\n\tgit config http.receivepack true &&\n\tORIG_HEAD=$(git rev-parse --verify HEAD) &&\n\tcd - &&\n\tmv test_repo.git \"$HTTPD_DOCUMENT_ROOT_PATH\"\n\nInitialized empty Git repository in /Users/brian/dev/git/t/trash directory.t5541-http-push/test_repo/.git/\n[master (root-commit) 0c973ae] initial\n 0 files changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 path1\n/Users/brian/dev/git/t/trash directory.t5541-http-push\nInitialized empty Git repository in /Users/brian/dev/git/t/trash directory.t5541-http-push/test_repo.git/\n/Users/brian/dev/git/t/trash directory.t5541-http-push\n*   ok 1: setup remote repository\n\n* expecting success: \n\tcd \"$ROOT_PATH\" &&\n\tgit clone $HTTPD_URL/smart/test_repo.git test_repo_clone\n\nInitialized empty Git repository in /Users/brian/dev/git/t/trash directory.t5541-http-push/test_repo_clone/.git/\nerror: RPC failed; result=22, HTTP code = 500\n^CFATAL: Unexpected exit with code 130\n\nbrian@hermes t (master)$ cd trash\\ directory.t5541-http-push/httpd/\nbrian@hermes httpd (master)$ cat error.log \n[Wed Nov 11 06:19:39 2009] [notice] Apache/2.2.13 (Unix) configured -- resuming normal operations\n[Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] git-http-backend(59490) malloc: *** error for object 0x100200340: incorrect checksum for freed object - object was probably modified after being freed.\n[Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] *** set a breakpoint in malloc_error_break to debug\n[Wed Nov 11 06:19:40 2009] [error] [client 127.0.0.1] Premature end of script headers: git-http-backend\n[Wed Nov 11 06:24:23 2009] [notice] caught SIGTERM, shutting down"},{"id":"127341","messageId":"be6fef0d0911110747k219e1fe0l7bd19bbc58b5ea3e@mail.gmail.com","threadId":"21577","inReplyTo":"B17AB159-E217-4E1F-BEA3-97E5892C13F4@gernhardtsoftware.com","subject":"Re: t5541-http-push hanging","fromName":"Tay Ray Chuan","fromEmail":"rctay89@gmail.com","sentAt":"2009-11-11T15:47:33Z","receivedAt":"2009-11-11T15:47:33Z","isPatch":false,"sender":{"key":"rctay89@gmail.com","avatar":"https://avatars.githubusercontent.com/u/61553?v=4"},"body":"Hi,\n\nOn Wed, Nov 11, 2009 at 2:28 PM, Brian Gernhardt\n<brian@gernhardtsoftware.com> wrote:\n> error: RPC failed; result=22, HTTP code = 500\n> ^CFATAL: Unexpected exit with code 130\n\nlooks like an apache/http-backend issue. What version of apache are you running?\n\nAlso, are you running out of the 'next' branch?\n\n-- \nCheers,\nRay Chuan\n"},{"id":"127342","messageId":"20091111155025.GN11919@spearce.org","threadId":"21577","inReplyTo":"B17AB159-E217-4E1F-BEA3-97E5892C13F4@gernhardtsoftware.com","subject":"Re: t5541-http-push hanging","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-11-11T15:50:25Z","receivedAt":"2009-11-11T15:50:25Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Brian Gernhardt <brian@gernhardtsoftware.com> wrote:\n> brian@hermes httpd (master)$ cat error.log \n> [Wed Nov 11 06:19:39 2009] [notice] Apache/2.2.13 (Unix) configured -- resuming normal operations\n> [Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] git-http-backend(59490) malloc: *** error for object 0x100200340: incorrect checksum for freed object - object was probably modified after being freed.\n> [Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] *** set a breakpoint in malloc_error_break to debug\n> [Wed Nov 11 06:19:40 2009] [error] [client 127.0.0.1] Premature end of script headers: git-http-backend\n> [Wed Nov 11 06:24:23 2009] [notice] caught SIGTERM, shutting down--\n\nYikes, looks like git-http-backend corrupted its heap.\n\nIs this by any chance Mac OS X ?\n\n-- \nShawn.\n"},{"id":"127349","messageId":"9D004E6C-52FF-441D-82A6-8D002401703A@gernhardtsoftware.com","threadId":"21577","inReplyTo":"be6fef0d0911110747k219e1fe0l7bd19bbc58b5ea3e@mail.gmail.com","subject":"Re: t5541-http-push hanging","fromName":"Brian Gernhardt","fromEmail":"brian@gernhardtsoftware.com","sentAt":"2009-11-11T16:53:24Z","receivedAt":"2009-11-11T16:53:24Z","isPatch":false,"sender":{"key":"brian@gernhardtsoftware.com","avatar":"https://avatars.githubusercontent.com/u/133455?v=4"},"body":"\nOn Nov 11, 2009, at 10:47 AM, Tay Ray Chuan wrote:\n\n> On Wed, Nov 11, 2009 at 2:28 PM, Brian Gernhardt\n> <brian@gernhardtsoftware.com> wrote:\n>> error: RPC failed; result=22, HTTP code = 500\n>> ^CFATAL: Unexpected exit with code 130\n> \n> looks like an apache/http-backend issue. What version of apache are you running?\n\nIt's Apache 2.2.13 on Mac OS X 10.6.2\n\n> Also, are you running out of the 'next' branch?\n\nYes.  That particular build was v1.6.5.2-372-gc05027d\n\nHowever I've been seeing it for a couple days.\n\n~~ Brian"},{"id":"127409","messageId":"20091112042645.GO11919@spearce.org","threadId":"21577","inReplyTo":"B17AB159-E217-4E1F-BEA3-97E5892C13F4@gernhardtsoftware.com","subject":"Re: t5541-http-push hanging","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-11-12T04:26:45Z","receivedAt":"2009-11-12T04:26:45Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Brian Gernhardt <brian@gernhardtsoftware.com> wrote:\n> My build script for git has been hanging at t5541.2 and I haven't had the tuits to discover why.  Here's what I've gotten so far, in case anyone can figure it out faster:\n...\n> Initialized empty Git repository in /Users/brian/dev/git/t/trash directory.t5541-http-push/test_repo_clone/.git/\n> error: RPC failed; result=22, HTTP code = 500\n> ^CFATAL: Unexpected exit with code 130\n...\n> \n> [Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] git-http-backend(59490) malloc: *** error for object 0x100200340: incorrect checksum for freed object - object was probably modified after being freed.\n> [Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] *** set a breakpoint in malloc_error_break to debug\n> [Wed Nov 11 06:19:40 2009] [error] [client 127.0.0.1] Premature end of script headers: git-http-backend\n\nProbably caused by a missing cast:\n\n\ndiff --git a/http-backend.c b/http-backend.c\nindex f8ea9d7..957e4ef 100644\n--- a/http-backend.c\n+++ b/http-backend.c\n@@ -136,7 +136,7 @@ static void hdr_str(const char *name, const char *value)\n \n static void hdr_int(const char *name, size_t value)\n {\n-\tformat_write(1, \"%s: %\" PRIuMAX \"\\r\\n\", name, value);\n+\tformat_write(1, \"%s: %\" PRIuMAX \"\\r\\n\", name, (uintmax_t)value);\n }\n \n static void hdr_date(const char *name, unsigned long when)\n\n-- \nShawn.\n"},{"id":"127411","messageId":"20091112044906.GQ11919@spearce.org","threadId":"21577","inReplyTo":"B17AB159-E217-4E1F-BEA3-97E5892C13F4@gernhardtsoftware.com","subject":"Re: t5541-http-push hanging","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-11-12T04:49:06Z","receivedAt":"2009-11-12T04:49:06Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Brian Gernhardt <brian@gernhardtsoftware.com> wrote:\n> My build script for git has been hanging at t5541.2 and I haven't had the tuits to discover why.  Here's what I've gotten so far, in case anyone can figure it out faster:\n...\n> \tgit clone $HTTPD_URL/smart/test_repo.git test_repo_clone\n...\n> [Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] git-http-backend(59490) malloc: *** error for object 0x100200340: incorrect checksum for freed object - object was probably modified after being freed.\n> [Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] *** set a breakpoint in malloc_error_break to debug\n\nI can't reproduce this on 10.5.8.  Can you try running it in the\ndebugger and putting a breakpoint where they suggest?  Maybe it\nwill help give us some context information:\n\n  REQUEST_METHOD=GET \\\n  PATH_TRANSLATED=`pwd`/.git/info/refs \\\n  QUERY_STRING=service=git-upload-pack \\\n  gdb ./git-http-backend\n\n  b malloc_error_break\n  run\n\nI suspect that's where our failure is, the code path for the actual\ncommon negotiation and data transfer is shorter, and less likely\nto need to do a memory allocation... and therefore a free.\n\n-- \nShawn.\n"},{"id":"127412","messageId":"BA539267-B705-4BE3-BF69-DCE5FBD35CEA@gernhardtsoftware.com","threadId":"21577","inReplyTo":"20091112044906.GQ11919@spearce.org","subject":"Re: t5541-http-push hanging","fromName":"Brian Gernhardt","fromEmail":"brian@gernhardtsoftware.com","sentAt":"2009-11-12T05:18:26Z","receivedAt":"2009-11-12T05:18:26Z","isPatch":false,"sender":{"key":"brian@gernhardtsoftware.com","avatar":"https://avatars.githubusercontent.com/u/133455?v=4"},"body":"\nOn Nov 11, 2009, at 11:49 PM, Shawn O. Pearce wrote:\n\n> Brian Gernhardt <brian@gernhardtsoftware.com> wrote:\n>> My build script for git has been hanging at t5541.2 and I haven't had the tuits to discover why.  Here's what I've gotten so far, in case anyone can figure it out faster:\n> ...\n>> \tgit clone $HTTPD_URL/smart/test_repo.git test_repo_clone\n> ...\n>> [Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] git-http-backend(59490) malloc: *** error for object 0x100200340: incorrect checksum for freed object - object was probably modified after being freed.\n>> [Wed Nov 11 06:19:39 2009] [error] [client 127.0.0.1] *** set a breakpoint in malloc_error_break to debug\n> \n> I can't reproduce this on 10.5.8.  Can you try running it in the\n> debugger and putting a breakpoint where they suggest?  Maybe it\n> will help give us some context information:\n> \n>  REQUEST_METHOD=GET \\\n>  PATH_TRANSLATED=`pwd`/.git/info/refs \\\n>  QUERY_STRING=service=git-upload-pack \\\n>  gdb ./git-http-backend\n> \n>  b malloc_error_break\n>  run\n> \n> I suspect that's where our failure is, the code path for the actual\n> common negotiation and data transfer is shorter, and less likely\n> to need to do a memory allocation... and therefore a free.\n\nInteresting.  This command runs to completion.  In the test environment it gives:\n\nExpires: Fri, 01 Jan 1980 00:00:00 GMT\nPragma: no-cache\nCache-Control: no-cache, max-age=0, must-revalidate\nContent-Type: application/x-git-upload-pack-advertisement\n\n001e# service=git-upload-pack\n0000009b0c973ae9bd51902a28466f3850b543fa66a6aaf4 HEADmulti_ack thin-pack side-band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed\n003f0c973ae9bd51902a28466f3850b543fa66a6aaf4 refs/heads/master\n0000\n\n(which looks right to me)\n\naccess.log has:\n127.0.0.1 - - [12/Nov/2009:04:54:38 +0000] \"GET /smart/test_repo.git/info/refs?service=git-upload-pack HTTP/1.1\" 200 256\n127.0.0.1 - - [12/Nov/2009:04:54:38 +0000] \"POST /smart/test_repo.git/git-upload-pack HTTP/1.1\" 500 538\n\nAfter a bit of trial and error, I got this:\n\ntest_repo.git (BARE:master)$ REQUEST_METHOD=POST PATH_TRANSLATED=`pwd`/git-upload-pack gdb ../../../../../git-http-backend GNU gdb 6.3.50-20050815 (Apple version gdb-1344) (Fri Jul  3 01:19:56 UTC 2009)\nCopyright 2004 Free Software Foundation, Inc.\nGDB is free software, covered by the GNU General Public License, and you are\nwelcome to change it and/or distribute copies of it under certain conditions.\nType \"show copying\" to see the conditions.\nThere is absolutely no warranty for GDB.  Type \"show warranty\" for details.\nThis GDB was configured as \"x86_64-apple-darwin\"...Reading symbols for shared libraries ..\nwarning: Could not find object file \"/sw/src/fink.build/libiconv-1.12-3/libiconv-1.12/lib/.libs/iconv.o\" - no debug information available for \"./iconv.c\".\n\n\nwarning: Could not find object file \"/sw/src/fink.build/libiconv-1.12-3/libiconv-1.12/lib/.libs/localcharset.o\" - no debug information available for \"./../libcharset/lib/localcharset.c\".\n\n\nwarning: Could not find object file \"/sw/src/fink.build/libiconv-1.12-3/libiconv-1.12/lib/.libs/relocatable.o\" - no debug information available for \"./relocatable.c\".\n\n.. done\n\n(gdb) b malloc_error_break \nBreakpoint 1 at 0x4189374bb86951\n(gdb) run\nStarting program: /Users/brian/dev/git/git-http-backend \nReading symbols for shared libraries +++. done\ngit-http-backend(2358) malloc: *** error for object 0x100200390: incorrect checksum for freed object - object was probably modified after being freed.\n*** set a breakpoint in malloc_error_break to debug\n\nBreakpoint 1, 0x00007fff84c19951 in malloc_error_break ()\n(gdb) bt\n#0  0x00007fff84c19951 in malloc_error_break ()\n#1  0x00007fff84c1aaa8 in szone_error ()\n#2  0x00007fff84b466e3 in tiny_free_list_remove_ptr ()\n#3  0x00007fff84b44df7 in szone_realloc ()\n#4  0x00007fff84b44b3b in malloc_zone_realloc ()\n#5  0x00007fff84b50bd3 in realloc ()\n#6  0x000000010004985f in xrealloc (ptr=0x100200370, size=66) at wrapper.c:59\n#7  0x0000000100041372 in strbuf_grow (sb=0x7fff5fbfed70, extra=<value temporarily unavailable, due to optimizations>) at strbuf.c:61\n#8  0x000000010001e194 in strbuf_addch [inlined] () at /Users/brian/dev/git/strbuf.h:95\n#9  0x000000010001e194 in add_path (out=0x7fff5fbfed70, path=0x100200320 \"/usr/local/libexec/git-core\") at exec_cmd.c:95\n#10 0x000000010001e229 in setup_path () at exec_cmd.c:104\n#11 0x000000010000226d in main (argc=<value temporarily unavailable, due to optimizations>, argv=<value temporarily unavailable, due to optimizations>) at http-backend.c:641\n(gdb) \n\nI'm lost as to why it's dying doing setup_path.  My guess is that some of the code before here is doing something bad to memory.\n\n~~ Brian"}]}