{"thread":{"id":"15166","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","startedAt":"2008-08-22T16:25:49Z","lastAt":"2008-08-22T21:36:18Z","messageCount":12,"participants":["Andrew Morton","Petr Baudis","Alan D. Brunelle","Björn Steinbrink","Jeff King","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"88156","messageId":"20080822092549.ddcb7e79.akpm@linux-foundation.org","threadId":"15166","inReplyTo":"48AEDD3D.4060507@hp.com","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-08-22T16:25:49Z","receivedAt":"2008-08-22T16:25:49Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Fri, 22 Aug 2008 11:37:33 -0400 \"Alan D. Brunelle\" <Alan.Brunelle@hp.com> wrote:\n\n> Andrew Morton wrote:\n> \n> > \n> > You could confirm/debug it with something along the lines of this:\n> > \n> > --- a/mm/vmalloc.c~a\n> > +++ a/mm/vmalloc.c\n> > @@ -214,7 +214,9 @@ __get_vm_area_node(unsigned long size, u\n> >  \tunsigned long align = 1;\n> >  \tunsigned long addr;\n> >  \n> > -\tBUG_ON(in_interrupt());\n> > +\tif (preempt_count() > 10)\n> > +\t\tprintk(\"%s: preempt_count()=%d\\n\", __func__, preempt_count());\n> > +\tWARN_ON(in_interrupt());\n> >  \tif (flags & VM_IOREMAP) {\n> >  \t\tint bit = fls(size);\n> >  \n> > _\n> > \n> > \n> > But this bug could be in practically anywhere in the kernel and\n> > bisection is by far the best way to find it.  It's sad and odd that\n> > bisection landed you on a merge commit.  I'd suggest that you persist\n> > with the bisection (please). \n> > http://www.kernel.org/doc/local/git-quick.html#example might be useful.\n> \n> Sorry, was off on other things for the last couple of days:\n> \n> I /did/ bisect it down to the aforementioned merge, the question is: How\n> to crack open that merge into it's composite pieces? (Where do I go to\n> bisect within that?)\n\nurgh, it's irritating when git-bisect directs you to a merge commit - it\nhasn't done it for me for ages.\n\nOne (probably wrong) approach is to run\n\n\tgitk 1c89ac55017f982355c7761e1c912c88c941483d\n\nthen peer at the output, work out which real commits were in that\nmerge.\n\nIt looks like the merge ended with\nb1b135c8d619cb2c7045d6ee4e48375882518bb5 and started with\n40c42076ebd362dc69210cccea101ac80b6d4bd4, so perhaps you can do\n\n\tgit bisect bad b1b135c8d619cb2c7045d6ee4e48375882518bb5\n\tgit bisect good 40c42076ebd362dc69210cccea101ac80b6d4bd4\n\nThat has an off-by-one error, in that it assumes that\n40c42076ebd362dc69210cccea101ac80b6d4bd4 is good, which we don't know. \nPerhaps one can pick the preceding commit in the `git log' output to\nfix that.\n\nAnyway, I've sent sufficient wrongness to the right list for us to work\nout how to do this for real ;)\n"},{"id":"88161","messageId":"20080822171651.GP10544@machine.or.cz","threadId":"15166","inReplyTo":"20080822092549.ddcb7e79.akpm@linux-foundation.org","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-08-22T17:16:51Z","receivedAt":"2008-08-22T17:16:51Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote:\n> On Fri, 22 Aug 2008 11:37:33 -0400 \"Alan D. Brunelle\" <Alan.Brunelle@hp.com> wrote:\n> \n> > I /did/ bisect it down to the aforementioned merge, the question is: How\n> > to crack open that merge into it's composite pieces? (Where do I go to\n> > bisect within that?)\n> \n> urgh, it's irritating when git-bisect directs you to a merge commit - it\n> hasn't done it for me for ages.\n\nHmm, but doesn't that happen only when it's actually really the merge\ncommit that introduces the bug? Both parents of the merge commit were\nmarked as good by the user, so...\n\n> One (probably wrong) approach is to run\n> \n> \tgitk 1c89ac55017f982355c7761e1c912c88c941483d\n> \n> then peer at the output, work out which real commits were in that\n> merge.\n> \n> It looks like the merge ended with\n> b1b135c8d619cb2c7045d6ee4e48375882518bb5 and started with\n> 40c42076ebd362dc69210cccea101ac80b6d4bd4, so perhaps you can do\n> \n> \tgit bisect bad b1b135c8d619cb2c7045d6ee4e48375882518bb5\n> \tgit bisect good 40c42076ebd362dc69210cccea101ac80b6d4bd4\n\n...I don't quite get this - according to the bisection log,\n\n\t# good: [b1b135c8d619cb2c7045d6ee4e48375882518bb5] fix spinlock recursion in hvc_console\n\nand now you want to mark it as bad?\n\nYou could try to revert some of the merged commits at the point of the\nmerge, though.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nThe next generation of interesting software will be done\non the Macintosh, not the IBM PC.  -- Bill Gates\n"},{"id":"88166","messageId":"20080822105136.a8432875.akpm@linux-foundation.org","threadId":"15166","inReplyTo":"20080822171651.GP10544@machine.or.cz","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-08-22T17:51:36Z","receivedAt":"2008-08-22T17:51:36Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Fri, 22 Aug 2008 19:16:51 +0200\nPetr Baudis <pasky@suse.cz> wrote:\n\n> On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote:\n> > On Fri, 22 Aug 2008 11:37:33 -0400 \"Alan D. Brunelle\" <Alan.Brunelle@hp.com> wrote:\n> > \n> > > I /did/ bisect it down to the aforementioned merge, the question is: How\n> > > to crack open that merge into it's composite pieces? (Where do I go to\n> > > bisect within that?)\n> > \n> > urgh, it's irritating when git-bisect directs you to a merge commit - it\n> > hasn't done it for me for ages.\n> \n> Hmm, but doesn't that happen only when it's actually really the merge\n> commit that introduces the bug? Both parents of the merge commit were\n> marked as good by the user, so...\n\nA merge commit doesn't contain any kernel changes?  It's the individual\ncommits (aka \"patches\") which were in that merge which broke stuff. \nConfused.\n\nWe're trying to dive inside that merge commit to find out which of the\nreal commits caused the regression.\n\n> > One (probably wrong) approach is to run\n> > \n> > \tgitk 1c89ac55017f982355c7761e1c912c88c941483d\n> > \n> > then peer at the output, work out which real commits were in that\n> > merge.\n> > \n> > It looks like the merge ended with\n> > b1b135c8d619cb2c7045d6ee4e48375882518bb5 and started with\n> > 40c42076ebd362dc69210cccea101ac80b6d4bd4, so perhaps you can do\n> > \n> > \tgit bisect bad b1b135c8d619cb2c7045d6ee4e48375882518bb5\n> > \tgit bisect good 40c42076ebd362dc69210cccea101ac80b6d4bd4\n> \n> ...I don't quite get this - according to the bisection log,\n> \n> \t# good: [b1b135c8d619cb2c7045d6ee4e48375882518bb5] fix spinlock recursion in hvc_console\n> \n> and now you want to mark it as bad?\n\n<what bisection log?>\n\nI assume that Alan's bisection search ended up saying that the merge\ncommit (1c89ac55017f982355c7761e1c912c88c941483d) was the first bad\ncommit.\n\nNow I don't know what's going on.\n\n> You could try to revert some of the merged commits at the point of the\n> merge, though.\n"},{"id":"88170","messageId":"48AF006C.9030705@hp.com","threadId":"15166","inReplyTo":"20080822105136.a8432875.akpm@linux-foundation.org","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Alan D. Brunelle","fromEmail":"alan.brunelle@hp.com","sentAt":"2008-08-22T18:07:40Z","receivedAt":"2008-08-22T18:07:40Z","isPatch":false,"sender":{"key":"alan.brunelle@hp.com","avatar":null},"body":"Andrew Morton wrote:\n> On Fri, 22 Aug 2008 19:16:51 +0200\n> Petr Baudis <pasky@suse.cz> wrote:\n> \n>> On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote:\n>>> On Fri, 22 Aug 2008 11:37:33 -0400 \"Alan D. Brunelle\" <Alan.Brunelle@hp.com> wrote:\n>>>\n>>>> I /did/ bisect it down to the aforementioned merge, the question is: How\n>>>> to crack open that merge into it's composite pieces? (Where do I go to\n>>>> bisect within that?)\n>>> urgh, it's irritating when git-bisect directs you to a merge commit - it\n>>> hasn't done it for me for ages.\n>> Hmm, but doesn't that happen only when it's actually really the merge\n>> commit that introduces the bug? Both parents of the merge commit were\n>> marked as good by the user, so...\n> \n> A merge commit doesn't contain any kernel changes?  It's the individual\n> commits (aka \"patches\") which were in that merge which broke stuff. \n> Confused.\n> \n> We're trying to dive inside that merge commit to find out which of the\n> real commits caused the regression.\n> \n>>> One (probably wrong) approach is to run\n>>>\n>>> \tgitk 1c89ac55017f982355c7761e1c912c88c941483d\n>>>\n>>> then peer at the output, work out which real commits were in that\n>>> merge.\n>>>\n>>> It looks like the merge ended with\n>>> b1b135c8d619cb2c7045d6ee4e48375882518bb5 and started with\n>>> 40c42076ebd362dc69210cccea101ac80b6d4bd4, so perhaps you can do\n>>>\n>>> \tgit bisect bad b1b135c8d619cb2c7045d6ee4e48375882518bb5\n>>> \tgit bisect good 40c42076ebd362dc69210cccea101ac80b6d4bd4\n>> ...I don't quite get this - according to the bisection log,\n>>\n>> \t# good: [b1b135c8d619cb2c7045d6ee4e48375882518bb5] fix spinlock recursion in hvc_console\n>>\n>> and now you want to mark it as bad?\n> \n> <what bisection log?>\n> \n> I assume that Alan's bisection search ended up saying that the merge\n> commit (1c89ac55017f982355c7761e1c912c88c941483d) was the first bad\n> commit.\n> \n> Now I don't know what's going on.\n> \n>> You could try to revert some of the merged commits at the point of the\n>> merge, though.\n\nYou're right - this is where the bisection ended up (in fact, just\nremoving this merge commit allowed later kernels to boot just fine).\n\nI've patched the kernel w/ your suggestion, and now I'm getting\nsomething a bit different - and I'm looking into this...\n\nBegin: Loading essential drivers... ...\n\n[    6.594525] fuse init (API version 7.9)\n\n[    6.618686] ACPI: SSDT CFFD0D0A, 08C4 (r1 HPQOEM  CPU_TM2        1\nMSFT  100000E)\n[    6.621481] BUG: unable to handle kernel NULL pointer dereference at\n0000000000000858\n[    6.625017] IP: [<ffffffff8025e282>] debug_mutex_add_waiter+0x32/0x80\n\n[    6.625017] PGD 21a456067 PUD 21a45f067 PMD 0\n\n[    6.625017] Oops: 0002 [1] SMP\n\n[    6.625017] CPU 1\n\n[    6.625017] Modules linked in: processor(+) fan thermal_sys fuse\n\n[    6.625017] Pid: 1259, comm: modprobe Not tainted 2.6.27-rc3 #24\n\n[    6.625017] RIP: 0010:[<ffffffff8025e282>]  [<ffffffff8025e282>]\ndebug_mutex_add_waiter+0x32/0x80\n\n[    6.625017] RSP: 0018:ffff88021a073998  EFLAGS: 00010002\n\n[    6.625017] RAX: 0000000000000000 RBX: ffff88021a0739d8 RCX:\n0000000000000000\n\n[    6.625017] RDX: 0000000000000001 RSI: ffff88021a0739d8 RDI:\nffffffffa0091a60\n\n[    6.625017] RBP: ffff88021a0739b8 R08: ffffffff811deea0 R09:\nffff8800a6fdb000\n\n[    6.625017] R10: ffffffffa008f524 R11: 0000000000000000 R12:\nffffffffa0091a60\n\n[    6.625017] R13: ffff88021a072000 R14: ffff88021a9840a0 R15:\nffffffffa0091a98\n\n[    6.625017] FS:  00007f40ce2de6e0(0000) GS:ffff88022fc02a00(0000)\nknlGS:0000000000000000\n\n[    6.625017] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b\n\n[    6.625017] CR2: 0000000000000858 CR3: 000000022d86b000 CR4:\n00000000000006e0\n\n[    6.625017] DR0: 0000000000000000 DR1: 0000000000000000 DR2:\n0000000000000000\n\n[    6.625017] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:\n0000000000000400\n\n[    6.625017] Process modprobe (pid: 1259, threadinfo ffff88021a072000,\ntask ffff88021a9840a0)\n[    6.625017] Stack:  0000000000000000 ffffffffa0091a60\n0000000000000246 ffffffffa008f524\n\n[    6.625017]  ffff88021a073a38 ffffffff8049f556 ffffffffa008f524\nffffffffa0091a18\n\n[    6.625017]  ffff88021a0739d8 ffff88021a0739d8 1111111111111111\n1111111111111111\n\n[    6.625017] Call Trace:\n\n[    6.625017]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n\n[    6.625017]  [<ffffffff8049f556>] mutex_lock_nested+0xa6/0x250\n\n[    6.625017]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n\n[    6.625017]  [<ffffffff80363584>] ? idr_pre_get+0x44/0x90\n\n[    6.625017]  [<ffffffffa008f524>] get_idr+0x44/0xa0 [thermal_sys]\n\n[    6.625017]  [<ffffffffa008fe43>]\nthermal_cooling_device_register+0x83/0x250 [thermal_sys]\n\n[    6.625017]  [<ffffffffa019b2a3>] acpi_processor_start+0x64b/0x774\n[processor]\n[    6.625017]  [<ffffffff8031a90b>] ? __sysfs_add_one+0x6b/0xa0\n\n[    6.625017]  [<ffffffff8031b9fc>] ? sysfs_do_create_link+0xbc/0x150\n\n[    6.625017]  [<ffffffff803a7f1e>] acpi_start_single_object+0x2d/0x52\n\n[    6.625017]  [<ffffffff803a9516>] acpi_device_probe+0x7e/0x92\n\n[    6.625017]  [<ffffffff803dd3ab>] driver_probe_device+0x9b/0x1a0\n\n[    6.625017]  [<ffffffff803dd536>] __driver_attach+0x86/0x90\n\n[    6.625017]  [<ffffffff803dd4b0>] ? __driver_attach+0x0/0x90\n\n[    6.625017]  [<ffffffff803dc8fd>] bus_for_each_dev+0x5d/0x90\n\n[    6.625017]  [<ffffffff803dd1ec>] driver_attach+0x1c/0x20\n\n[    6.625017]  [<ffffffff803dcf39>] bus_add_driver+0x1e9/0x260\n\n[    6.625017]  [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107\n[processor]\n\n[    6.625017]  [<ffffffff803dd70f>] driver_register+0x5f/0x140\n\n[    6.625017]  [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107\n[processor]\n\n[    6.625017]  [<ffffffff803a9826>] acpi_bus_register_driver+0x3e/0x40\n\n[    6.625017]  [<ffffffffa0222094>] acpi_processor_init+0x94/0x107\n[processor]\n\n[    6.625017]  [<ffffffff80209040>] _stext+0x40/0x180\n\n[    6.625017]  [<ffffffff802a88d1>] ? __vunmap+0xa1/0x110\n\n[    6.625017]  [<ffffffff80267642>] sys_init_module+0x142/0x1dc0\n\n[    6.625017]  [<ffffffff80367ad6>] ? __up_read+0x46/0xb0\n\n[    6.625017]  [<ffffffff8048e530>] ? cpu_down+0x0/0x70\n\n[    6.625017]  [<ffffffff8020c34b>] system_call_fastpath+0x16/0x1b\n\n[    6.625017]\n"},{"id":"88180","messageId":"20080822193730.GA1598@atjola.homenet","threadId":"15166","inReplyTo":"20080822105136.a8432875.akpm@linux-foundation.org","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-08-22T19:37:30Z","receivedAt":"2008-08-22T19:37:30Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2008.08.22 10:51:36 -0700, Andrew Morton wrote:\n> On Fri, 22 Aug 2008 19:16:51 +0200 Petr Baudis <pasky@suse.cz> wrote:\n> > On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote:\n> > > One (probably wrong) approach is to run\n> > > \n> > > \tgitk 1c89ac55017f982355c7761e1c912c88c941483d\n> > > \n> > > then peer at the output, work out which real commits were in that\n> > > merge.\n> > > \n> > > It looks like the merge ended with\n> > > b1b135c8d619cb2c7045d6ee4e48375882518bb5 and started with\n> > > 40c42076ebd362dc69210cccea101ac80b6d4bd4, so perhaps you can do\n> > > \n> > > \tgit bisect bad b1b135c8d619cb2c7045d6ee4e48375882518bb5\n> > > \tgit bisect good 40c42076ebd362dc69210cccea101ac80b6d4bd4\n> > \n> > ...I don't quite get this - according to the bisection log,\n> > \n> > \t# good: [b1b135c8d619cb2c7045d6ee4e48375882518bb5] fix spinlock recursion in hvc_console\n> > \n> > and now you want to mark it as bad?\n> \n> <what bisection log?>\n\nAlan provided his bisection log as an attachment to the original bug\nreport.\n\n> I assume that Alan's bisection search ended up saying that the merge\n> commit (1c89ac55017f982355c7761e1c912c88c941483d) was the first bad\n> commit.\n\nYep, and that's totally correct as far as bisect is concerned. The\nparents of that merge commit are:\n88fa08f67bee1a0c765237bdac106a32872f57d2\nb1b135c8d619cb2c7045d6ee4e48375882518bb5\n\nAnd Alan marked both of them as good.\n\nSo, unless Alan made a mistake during his bisection, each of the\nbranches is correct, but the merge did not lead to a correct result. So\nwhile there were no textual conflicts, there were still incompatible\nchanges regarding the code semantics and compatibility was not restored\nduring the merge.\n\nTo get an overview over what got merged together you can can use\nsomething like:\ngitk --left-right 1c89ac55017^1...1c89ac55017^2\n\nWhich shows all commits that were on only one side of the merge, with\nnice \"arrows\" that indicate from which side the commit is coming. The\nconflict should be between one commit from the left and one commit from\nthe right side, obviously.\n\nBjörn\n"},{"id":"88183","messageId":"48AF17C4.4060606@hp.com","threadId":"15166","inReplyTo":"20080822193730.GA1598@atjola.homenet","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Alan D. Brunelle","fromEmail":"alan.brunelle@hp.com","sentAt":"2008-08-22T19:47:16Z","receivedAt":"2008-08-22T19:47:16Z","isPatch":false,"sender":{"key":"alan.brunelle@hp.com","avatar":null},"body":"Björn Steinbrink wrote:\n> On 2008.08.22 10:51:36 -0700, Andrew Morton wrote:\n>> On Fri, 22 Aug 2008 19:16:51 +0200 Petr Baudis <pasky@suse.cz> wrote:\n>>> On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote:\n>>>> One (probably wrong) approach is to run\n>>>>\n>>>> \tgitk 1c89ac55017f982355c7761e1c912c88c941483d\n>>>>\n>>>> then peer at the output, work out which real commits were in that\n>>>> merge.\n>>>>\n>>>> It looks like the merge ended with\n>>>> b1b135c8d619cb2c7045d6ee4e48375882518bb5 and started with\n>>>> 40c42076ebd362dc69210cccea101ac80b6d4bd4, so perhaps you can do\n>>>>\n>>>> \tgit bisect bad b1b135c8d619cb2c7045d6ee4e48375882518bb5\n>>>> \tgit bisect good 40c42076ebd362dc69210cccea101ac80b6d4bd4\n>>> ...I don't quite get this - according to the bisection log,\n>>>\n>>> \t# good: [b1b135c8d619cb2c7045d6ee4e48375882518bb5] fix spinlock recursion in hvc_console\n>>>\n>>> and now you want to mark it as bad?\n>> <what bisection log?>\n> \n> Alan provided his bisection log as an attachment to the original bug\n> report.\n> \n>> I assume that Alan's bisection search ended up saying that the merge\n>> commit (1c89ac55017f982355c7761e1c912c88c941483d) was the first bad\n>> commit.\n> \n> Yep, and that's totally correct as far as bisect is concerned. The\n> parents of that merge commit are:\n> 88fa08f67bee1a0c765237bdac106a32872f57d2\n> b1b135c8d619cb2c7045d6ee4e48375882518bb5\n> \n> And Alan marked both of them as good.\n> \n> So, unless Alan made a mistake during his bisection, each of the\n> branches is correct, but the merge did not lead to a correct result. So\n> while there were no textual conflicts, there were still incompatible\n> changes regarding the code semantics and compatibility was not restored\n> during the merge.\n\nIt's important to note that even if I did make a mistake during the\nbisection process (and I certainly wouldn't discount that), recent\nkernels still fail: but when I take out that commit from a recent\nkernel, it fails.\n\nI put in Andrew's suggested patch (to help find things), and now I\nrepeatedly get the problems in the attached log.\n\nNot being an x86 knowledgeable person, I'm a bit concerned about the RSP\nvalue?! (I enabled stack overflow checking, but that didn't stop things.)\n\nAlan\n\n\nLoading, please wait...\nBegin: Loading essential drivers... ...\n[    6.604300] fuse init (API version 7.9)\n[    6.625529] ACPI: SSDT CFFD0D0A, 08C4 (r1 HPQOEM  CPU_TM2        1 MSFT  100000E)\n[    6.631954] BUG: unable to handle kernel NULL pointer dereference at 0000000000000858\n[    6.640583] IP: [<ffffffff8025e282>] debug_mutex_add_waiter+0x32/0x80\n[    6.640583] PGD 21a84f067 PUD 21ad55067 PMD 0 \n[    6.642421] Oops: 0002 [1] SMP \n[    6.642421] CPU 2 \n[    6.642421] Modules linked in: processor(+) fan thermal_sys fuse\n[    6.642421] Pid: 1259, comm: modprobe Not tainted 2.6.27-rc3 #26\n[    6.642421] RIP: 0010:[<ffffffff8025e282>]  [<ffffffff8025e282>] debug_mutex_add_waiter+0x32/0x80\n[    6.642421] RSP: 0018:ffff88021a1a5998  EFLAGS: 00010002\n[    6.642421] RAX: 0000000000000000 RBX: ffff88021a1a59d8 RCX: 0000000000000000\n[    6.642421] RDX: 0000000000000001 RSI: ffff88021a1a59d8 RDI: ffffffffa0091a60\n[    6.642421] RBP: ffff88021a1a59b8 R08: ffffffff811deea0 R09: ffff8800a6fed000\n[    6.642421] R10: ffffffffa008f524 R11: 0000000000000002 R12: ffffffffa0091a60\n[    6.642421] R13: ffff88021a1a4000 R14: ffff88021a34c0a0 R15: ffffffffa0091a98\n[    6.642421] FS:  00007f2c513606e0(0000) GS:ffff88022fc02e00(0000) knlGS:0000000000000000\n[    6.642421] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b\n[    6.642421] CR2: 0000000000000858 CR3: 000000021a8f7000 CR4: 00000000000006e0\n[    6.642421] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000\n[    6.642421] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400\n[    6.642421] Process modprobe (pid: 1259, threadinfo ffff88021a1a4000, task ffff88021a34c0a0)\n[    6.642421] Stack:  0000000000000000 ffffffffa0091a60 0000000000000246 ffffffffa008f524\n[    6.642421]  ffff88021a1a5a38 ffffffff8049f556 ffffffffa008f524 ffffffffa0091a18\n[    6.642421]  ffff88021a1a59d8 ffff88021a1a59d8 1111111111111111 1111111111111111\n[    6.642421] Call Trace:\n[    6.642421]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    6.642421]  [<ffffffff8049f556>] mutex_lock_nested+0xa6/0x250\n[    6.642421]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    6.642421]  [<ffffffff80363584>] ? idr_pre_get+0x44/0x90\n[    6.642421]  [<ffffffffa008f524>] get_idr+0x44/0xa0 [thermal_sys]\n[    6.642421]  [<ffffffffa008fe43>] thermal_cooling_device_register+0x83/0x250 [thermal_sys]\n[    6.642421]  [<ffffffffa019b2a3>] acpi_processor_start+0x64b/0x774 [processor]\n[    6.642421]  [<ffffffff8031a90b>] ? __sysfs_add_one+0x6b/0xa0\n[    6.642421]  [<ffffffff8031b9fc>] ? sysfs_do_create_link+0xbc/0x150\n[    6.642421]  [<ffffffff803a7f1e>] acpi_start_single_object+0x2d/0x52\n[    6.642421]  [<ffffffff803a9516>] acpi_device_probe+0x7e/0x92\n[    6.642421]  [<ffffffff803dd3ab>] driver_probe_device+0x9b/0x1a0\n[    6.642421]  [<ffffffff803dd536>] __driver_attach+0x86/0x90\n[    6.642421]  [<ffffffff803dd4b0>] ? __driver_attach+0x0/0x90\n[    6.642421]  [<ffffffff803dc8fd>] bus_for_each_dev+0x5d/0x90\n[    6.642421]  [<ffffffff803dd1ec>] driver_attach+0x1c/0x20\n[    6.642421]  [<ffffffff803dcf39>] bus_add_driver+0x1e9/0x260\n[    6.642421]  [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor]\n[    6.642421]  [<ffffffff803dd70f>] driver_register+0x5f/0x140\n[    6.642421]  [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor]\n[    6.642421]  [<ffffffff803a9826>] acpi_bus_register_driver+0x3e/0x40\n[    6.642421]  [<ffffffffa0222094>] acpi_processor_init+0x94/0x107 [processor]\n[    6.642421]  [<ffffffff80209040>] _stext+0x40/0x180\n[    6.642421]  [<ffffffff802a88d1>] ? __vunmap+0xa1/0x110\n[    6.642421]  [<ffffffff80267642>] sys_init_module+0x142/0x1dc0\n[    6.642421]  [<ffffffff80367ad6>] ? __up_read+0x46/0xb0\n[    6.642421]  [<ffffffff8048e530>] ? cpu_down+0x0/0x70\n[    6.642421]  [<ffffffff8020c34b>] system_call_fastpath+0x16/0x1b\n[    6.642421] \n[    6.642421] \n[    6.642421] Code: 20 48 89 5d e8 4c 89 65 f0 48 89 f3 4c 89 6d f8 8b 47 08 49 89 d5 49 89 fc 89 c2 25 ff ff 00 00 c1 ea 10 39 c2 74 1d 49 8b 4 \n[    6.652421] RIP  [<ffffffff8025e282>] debug_mutex_add_waiter+0x32/0x80\n[    6.652421]  RSP <ffff88021a1a5998>\n[    6.652421] CR2: 0000000000000858\n[    6.652421] ---[ end trace 53d26a9a6cd6aa38 ]---\n[    6.991546] ------------[ cut here ]------------\n[    6.996188] kernel BUG at kernel/sched.c:1155!\n[    7.005292] invalid opcode: 0000 [2] SMP \n[    7.005292] CPU 2 \n[    7.005292] Modules linked in: processor(+) fan thermal_sys fuse\n[    7.011541] Pid: 1259, comm: modprobe Tainted: G      D   2.6.27-rc3 #26\n[    7.011541] RIP: 0010:[<ffffffff8022cc2b>]  [<ffffffff8022cc2b>] resched_task+0x6b/0x70\n[    7.011541] RSP: 0018:ffff88022f0efe98  EFLAGS: 00010046\n[    7.011541] RAX: 00000000000006ee RBX: ffff880028092e00 RCX: ffffffff81046000\n[    7.011541] RDX: 00000000000006ee RSI: ffff88021a34c0a0 RDI: ffffffff805ee660\n[    7.011541] RBP: ffff88022f0efe98 R08: 0000000000000000 R09: 0000000000000001\n[    7.011541] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88021a34c0a0\n[    7.011541] R13: 0000000000000001 R14: ffff880028092e98 R15: ffff88021a34c0d8\n[    7.011541] FS:  00007f2c513606e0(0000) GS:ffff88022fc02e00(0000) knlGS:0000000000000000\n[    7.011541] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b\n[    7.011541] CR2: 0000000000000858 CR3: 000000021a8f7000 CR4: 00000000000006e0\n[    7.011541] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000\n[    7.011541] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400\n[    7.011541] Process modprobe (pid: 1259, threadinfo ffff88021a1a4000, task ffff88021a34c0a0)\n[    7.011541] Stack:  ffff88022f0efed8 ffffffff8023774f ffff88022f0efed8 ffff8800280b6e00\n[    7.011541]  ffffffff80238510 ffff8800280b2b20 ffff8800280b2aa0 7fffffffffffffff\n[    7.011541]  ffff88022f0efef8 ffffffff8023855d ffff8800280b2aa0 ffff8800280b7660\n[    7.011541] Call Trace:\n[    7.011541]  <IRQ>  [<ffffffff8023774f>] task_tick_fair+0xbf/0xd0\n[    7.011541]  [<ffffffff80238510>] ? hrtick+0x0/0x90\n[    7.011541]  [<ffffffff8023855d>] hrtick+0x4d/0x90\n[    7.011541]  [<ffffffff80254f65>] __run_hrtimer+0x95/0xb0\n[    7.011541]  [<ffffffff80255c4b>] hrtimer_interrupt+0x11b/0x190\n[    7.011541]  [<ffffffff8021d643>] smp_apic_timer_interrupt+0x83/0xc0\n[    7.011541]  [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70\n[    7.011541]  <EOI>  [<ffffffff804a12eb>] ? _spin_unlock_irq+0x2b/0x30\n[    7.011541]  [<ffffffff804a0b75>] ? __down_read+0xa5/0xb7\n[    7.011541]  [<ffffffff8026fad5>] ? acct_collect+0x45/0x1d0\n[    7.011541]  [<ffffffff8049fb57>] ? down_read+0x37/0x40\n[    7.011541]  [<ffffffff8026fad5>] ? acct_collect+0x45/0x1d0\n[    7.011541]  [<ffffffff8026fad5>] ? acct_collect+0x45/0x1d0\n[    7.011541]  [<ffffffff8023f27d>] ? do_exit+0x18d/0x9b0\n[    7.011541]  [<ffffffff803c7d19>] ? do_unblank_screen+0x19/0x130\n[    7.011541]  [<ffffffff804a1a17>] ? oops_end+0x87/0x90\n[    7.011541]  [<ffffffff804a3cd3>] ? do_page_fault+0x663/0x800\n[    7.011541]  [<ffffffff804a15ed>] ? error_exit+0x0/0x9a\n[    7.011541]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    7.011541]  [<ffffffff8025e282>] ? debug_mutex_add_waiter+0x32/0x80\n[    7.011541]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    7.011541]  [<ffffffff8049f556>] ? mutex_lock_nested+0xa6/0x250\n[    7.011541]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    7.011541]  [<ffffffff80363584>] ? idr_pre_get+0x44/0x90\n[    7.011541]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    7.011541]  [<ffffffffa008fe43>] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys]\n[    7.011541]  [<ffffffffa019b2a3>] ? acpi_processor_start+0x64b/0x774 [processor]\n[    7.011541]  [<ffffffff8031a90b>] ? __sysfs_add_one+0x6b/0xa0\n[    7.011541]  [<ffffffff8031b9fc>] ? sysfs_do_create_link+0xbc/0x150\n[    7.011541]  [<ffffffff803a7f1e>] ? acpi_start_single_object+0x2d/0x52\n[    7.011541]  [<ffffffff803a9516>] ? acpi_device_probe+0x7e/0x92\n[    7.011541]  [<ffffffff803dd3ab>] ? driver_probe_device+0x9b/0x1a0\n[    7.011541]  [<ffffffff803dd536>] ? __driver_attach+0x86/0x90\n[    7.011541]  [<ffffffff803dd4b0>] ? __driver_attach+0x0/0x90\n[    7.011541]  [<ffffffff803dc8fd>] ? bus_for_each_dev+0x5d/0x90\n[    7.011541]  [<ffffffff803dd1ec>] ? driver_attach+0x1c/0x20\n[    7.011541]  [<ffffffff803dcf39>] ? bus_add_driver+0x1e9/0x260\n[    7.011541]  [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor]\n[    7.011541]  [<ffffffff803dd70f>] ? driver_register+0x5f/0x140\n[    7.011541]  [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor]\n[    7.011541]  [<ffffffff803a9826>] ? acpi_bus_register_driver+0x3e/0x40\n[    7.011541]  [<ffffffffa0222094>] ? acpi_processor_init+0x94/0x107 [processor]\n[    7.011541]  [<ffffffff80209040>] ? _stext+0x40/0x180\n[    7.011541]  [<ffffffff802a88d1>] ? __vunmap+0xa1/0x110\n[    7.011541]  [<ffffffff80267642>] ? sys_init_module+0x142/0x1dc0\n[    7.011541]  [<ffffffff80367ad6>] ? __up_read+0x46/0xb0\n[    7.011541]  [<ffffffff8048e530>] ? cpu_down+0x0/0x70\n[    7.011541]  [<ffffffff8020c34b>] ? system_call_fastpath+0x16/0x1b\n[    7.011541] \n[    7.011541] \n[    7.011541] Code: 8b 47 08 8b 50 1c 65 8b 04 25 24 00 00 00 39 c2 74 0d 0f ae f0 48 8b 47 08 f6 40 18 04 74 02 c9 c3 89 d7 ff 15 1f 7b 3c 00 c \n[    7.021541] RIP  [<ffffffff8022cc2b>] resched_task+0x6b/0x70\n[    7.021541]  RSP <ffff88022f0efe98>\n[    7.021541] ---[ end trace 53d26a9a6cd6aa38 ]---\n[    7.021541] Kernel panic - not syncing: Aiee, killing interrupt handler!\n[    7.021541] ------------[ cut here ]------------\n[    7.021541] WARNING: at kernel/smp.c:328 smp_call_function_mask+0x25a/0x260()\n[    7.021541] Modules linked in: processor(+) fan thermal_sys fuse\n[    7.021541] Pid: 1259, comm: modprobe Tainted: G      D   2.6.27-rc3 #26\n[    7.021541] \n[    7.021541] Call Trace:\n[    7.021541]  <IRQ>  [<ffffffff8023bacf>] warn_on_slowpath+0x5f/0x80\n[    7.021541]  [<ffffffff802650ca>] smp_call_function_mask+0x25a/0x260\n[    7.021541]  [<ffffffff8036957d>] ? string+0x3d/0xd0\n[    7.021541]  [<ffffffff80369a4b>] ? vsnprintf+0x43b/0x720\n[    7.021541]  [<ffffffff8036957d>] ? string+0x3d/0xd0\n[    7.021541]  [<ffffffff8036957d>] ? string+0x3d/0xd0\n[    7.021541]  [<ffffffff80369a4b>] ? vsnprintf+0x43b/0x720\n[    7.021541]  [<ffffffff80368d1e>] ? number+0x2ae/0x2d0\n[    7.021541]  [<ffffffff80368d1e>] ? number+0x2ae/0x2d0\n[    7.021541]  [<ffffffff80269e4d>] ? kallsyms_lookup+0x5d/0xa0\n[    7.021541]  [<ffffffff80368d1e>] ? number+0x2ae/0x2d0\n[    7.021541]  [<ffffffff80369a4b>] ? vsnprintf+0x43b/0x720\n[    7.021541]  [<ffffffff80369d98>] ? sprintf+0x68/0x70\n[    7.021541]  [<ffffffff8036957d>] ? string+0x3d/0xd0\n[    7.021541]  [<ffffffff804a3f63>] ? __atomic_notifier_call_chain+0x83/0xa0\n[    7.021541]  [<ffffffff804a3ee0>] ? __atomic_notifier_call_chain+0x0/0xa0\n[    7.021541]  [<ffffffff804a0eb6>] ? _spin_unlock+0x26/0x30\n[    7.021541]  [<ffffffff8021c470>] ? stop_this_cpu+0x0/0x30\n[    7.021541]  [<ffffffff80265110>] smp_call_function+0x40/0x50\n[    7.021541]  [<ffffffff8021c4f3>] native_smp_send_stop+0x23/0x40\n[    7.021541]  [<ffffffff8023be1f>] panic+0xaf/0x190\n[    7.021541]  [<ffffffff8023cc77>] ? printk+0x67/0x70\n[    7.021541]  [<ffffffff8049f4a9>] ? mutex_unlock+0x9/0x10\n[    7.021541]  [<ffffffff80256c91>] ? blocking_notifier_call_chain+0x11/0x20\n[    7.021541]  [<ffffffff8023f9f9>] do_exit+0x909/0x9b0\n[    7.021541]  [<ffffffff803c7d19>] ? do_unblank_screen+0x19/0x130\n[    7.021541]  [<ffffffff804a1a17>] oops_end+0x87/0x90\n[    7.021541]  [<ffffffff8020e08e>] die+0x5e/0x90\n[    7.021541]  [<ffffffff804a1f20>] do_trap+0x130/0x150\n[    7.021541]  [<ffffffff8020e662>] do_invalid_op+0x92/0xb0\n[    7.021541]  [<ffffffff8022cc2b>] ? resched_task+0x6b/0x70\n[    7.021541]  [<ffffffff8025788e>] ? sched_clock_cpu+0x14e/0x190\n[    7.021541]  [<ffffffff804a15ed>] error_exit+0x0/0x9a\n[    7.021541]  [<ffffffff8022cc2b>] ? resched_task+0x6b/0x70\n[    7.021541]  [<ffffffff8023774f>] task_tick_fair+0xbf/0xd0\n[    7.021541]  [<ffffffff80238510>] ? hrtick+0x0/0x90\n[    7.021541]  [<ffffffff8023855d>] hrtick+0x4d/0x90\n[    7.021541]  [<ffffffff80254f65>] __run_hrtimer+0x95/0xb0\n[    7.021541]  [<ffffffff80255c4b>] hrtimer_interrupt+0x11b/0x190\n[    7.021541]  [<ffffffff8021d643>] smp_apic_timer_interrupt+0x83/0xc0\n[    7.021541]  [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70\n[    7.021541]  <EOI>  [<ffffffff804a12eb>] ? _spin_unlock_irq+0x2b/0x30\n[    7.021541]  [<ffffffff804a0b75>] ? __down_read+0xa5/0xb7\n[    7.021541]  [<ffffffff8026fad5>] ? acct_collect+0x45/0x1d0\n[    7.021541]  [<ffffffff8049fb57>] ? down_read+0x37/0x40\n[    7.021541]  [<ffffffff8026fad5>] ? acct_collect+0x45/0x1d0\n[    7.021541]  [<ffffffff8026fad5>] ? acct_collect+0x45/0x1d0\n[    7.021541]  [<ffffffff8023f27d>] ? do_exit+0x18d/0x9b0\n[    7.021541]  [<ffffffff803c7d19>] ? do_unblank_screen+0x19/0x130\n[    7.021541]  [<ffffffff804a1a17>] ? oops_end+0x87/0x90\n[    7.021541]  [<ffffffff804a3cd3>] ? do_page_fault+0x663/0x800\n[    7.021541]  [<ffffffff804a15ed>] ? error_exit+0x0/0x9a\n[    7.021541]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    7.021541]  [<ffffffff8025e282>] ? debug_mutex_add_waiter+0x32/0x80\n[    7.021541]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    7.021541]  [<ffffffff8049f556>] ? mutex_lock_nested+0xa6/0x250\n[    7.021541]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    7.021541]  [<ffffffff80363584>] ? idr_pre_get+0x44/0x90\n[    7.021541]  [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys]\n[    7.021541]  [<ffffffffa008fe43>] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys]\n[    7.021541]  [<ffffffffa019b2a3>] ? acpi_processor_start+0x64b/0x774 [processor]\n[    7.021541]  [<ffffffff8031a90b>] ? __sysfs_add_one+0x6b/0xa0\n[    7.021541]  [<ffffffff8031b9fc>] ? sysfs_do_create_link+0xbc/0x150\n[    7.021541]  [<ffffffff803a7f1e>] ? acpi_start_single_object+0x2d/0x52\n[    7.021541]  [<ffffffff803a9516>] ? acpi_device_probe+0x7e/0x92\n[    7.021541]  [<ffffffff803dd3ab>] ? driver_probe_device+0x9b/0x1a0\n[    7.021541]  [<ffffffff803dd536>] ? __driver_attach+0x86/0x90\n[    7.021541]  [<ffffffff803dd4b0>] ? __driver_attach+0x0/0x90\n[    7.021541]  [<ffffffff803dc8fd>] ? bus_for_each_dev+0x5d/0x90\n[    7.021541]  [<ffffffff803dd1ec>] ? driver_attach+0x1c/0x20\n[    7.021541]  [<ffffffff803dcf39>] ? bus_add_driver+0x1e9/0x260\n[    7.021541]  [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor]\n[    7.021541]  [<ffffffff803dd70f>] ? driver_register+0x5f/0x140\n[    7.021541]  [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor]\n[    7.021541]  [<ffffffff803a9826>] ? acpi_bus_register_driver+0x3e/0x40\n[    7.021541]  [<ffffffffa0222094>] ? acpi_processor_init+0x94/0x107 [processor]\n[    7.021541]  [<ffffffff80209040>] ? _stext+0x40/0x180\n[    7.021541]  [<ffffffff802a88d1>] ? __vunmap+0xa1/0x110\n[    7.021541]  [<ffffffff80267642>] ? sys_init_module+0x142/0x1dc0\n[    7.021541]  [<ffffffff80367ad6>] ? __up_read+0x46/0xb0\n[    7.021541]  [<ffffffff8048e530>] ? cpu_down+0x0/0x70\n[    7.021541]  [<ffffffff8020c34b>] ? system_call_fastpath+0x16/0x1b\n[    7.021541] \n[    7.021541] ---[ end trace 53d26a9a6cd6aa38 ]---\n[    7.990001] BUG: spinlock lockup on CPU#0, swapper/0, ffff8800280b6e00\n[    7.990001] Pid: 0, comm: swapper Tainted: G      D W 2.6.27-rc3 #26\n[    7.990001] \n[    7.990001] Call Trace:\n[    7.990001]  <IRQ>  [<ffffffff8036ee4e>] _raw_spin_lock+0x18e/0x1a0\n[    7.990001]  [<ffffffff804a0fc4>] _spin_lock+0x34/0x40\n[    7.990001]  [<ffffffff80230ed6>] ? sched_rt_period_timer+0x126/0x1e0\n[    7.990001]  [<ffffffff80230ed6>] sched_rt_period_timer+0x126/0x1e0\n[    7.990001]  [<ffffffff80230db0>] ? sched_rt_period_timer+0x0/0x1e0\n[    7.990001]  [<ffffffff80254f65>] __run_hrtimer+0x95/0xb0\n[    7.990001]  [<ffffffff80255c4b>] hrtimer_interrupt+0x11b/0x190\n[    7.990001]  [<ffffffff8021d643>] smp_apic_timer_interrupt+0x83/0xc0\n[    7.990001]  [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70\n[    7.990001]  <EOI>  [<ffffffff802138a0>] ? mwait_idle+0x40/0x50\n[    7.990001]  [<ffffffff8020a8d2>] ? enter_idle+0x22/0x30\n[    7.990001]  [<ffffffff8020ae5c>] ? cpu_idle+0x4c/0xc0\n[    7.990001]  [<ffffffff8048d7d1>] ? rest_init+0x61/0x70\n[    7.990001] \n\n CTRL-A Z for help |115200 8N1 | NOR | Minicom 2.3-rc | VT102 | Online 00:03                                                                     \n\n\n"},{"id":"88184","messageId":"20080822194846.GA31356@coredump.intra.peff.net","threadId":"15166","inReplyTo":"20080822193730.GA1598@atjola.homenet","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-08-22T19:48:46Z","receivedAt":"2008-08-22T19:48:46Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Aug 22, 2008 at 09:37:30PM +0200, Björn Steinbrink wrote:\n\n> Yep, and that's totally correct as far as bisect is concerned. The\n> parents of that merge commit are:\n> 88fa08f67bee1a0c765237bdac106a32872f57d2\n> b1b135c8d619cb2c7045d6ee4e48375882518bb5\n> \n> And Alan marked both of them as good.\n> \n> So, unless Alan made a mistake during his bisection, each of the\n> branches is correct, but the merge did not lead to a correct result. So\n> while there were no textual conflicts, there were still incompatible\n> changes regarding the code semantics and compatibility was not restored\n> during the merge.\n\nOne thing that I have seen proposed (but never tried myself) is that you\ncan linearize the changes using \"rebase -i\" (or cherry-picking), and\nthen bisect that result. That is, given a history\n\nA-B-C-D\n \\   /\n  E-F\n\nwhere the merge \"D\" introduces the bug, you could try creating:\n\n  A-B-C-E'-F'\n\nand bisecting that. And you should know that C is good from your\nprevious bisection, but that F' probably is not, since it should be\ntextually the same as D (unless, of course, you had textual conflicts\nduring the rebase that you fixed up differently).\n\nSo in essence you are testing each of E and F, but based on the other\nwork. So you should be able to find the one patch that causes the\nconflict. And depending on the conflict, you may get more information by\ndoing it the other way. I.e.,:\n\n  A-E-F-B'-C'\n\n-Peff\n"},{"id":"88188","messageId":"48AF1ECC.7000205@hp.com","threadId":"15166","inReplyTo":"48AF17C4.4060606@hp.com","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Alan D. Brunelle","fromEmail":"alan.brunelle@hp.com","sentAt":"2008-08-22T20:17:16Z","receivedAt":"2008-08-22T20:17:16Z","isPatch":false,"sender":{"key":"alan.brunelle@hp.com","avatar":null},"body":"Alan D. Brunelle wrote:\n> Björn Steinbrink wrote:\n>> On 2008.08.22 10:51:36 -0700, Andrew Morton wrote:\n>>> On Fri, 22 Aug 2008 19:16:51 +0200 Petr Baudis <pasky@suse.cz> wrote:\n>>>> On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote:\n>>>>> One (probably wrong) approach is to run\n>>>>>\n>>>>> \tgitk 1c89ac55017f982355c7761e1c912c88c941483d\n>>>>>\n>>>>> then peer at the output, work out which real commits were in that\n>>>>> merge.\n>>>>>\n>>>>> It looks like the merge ended with\n>>>>> b1b135c8d619cb2c7045d6ee4e48375882518bb5 and started with\n>>>>> 40c42076ebd362dc69210cccea101ac80b6d4bd4, so perhaps you can do\n>>>>>\n>>>>> \tgit bisect bad b1b135c8d619cb2c7045d6ee4e48375882518bb5\n>>>>> \tgit bisect good 40c42076ebd362dc69210cccea101ac80b6d4bd4\n>>>> ...I don't quite get this - according to the bisection log,\n>>>>\n>>>> \t# good: [b1b135c8d619cb2c7045d6ee4e48375882518bb5] fix spinlock recursion in hvc_console\n>>>>\n>>>> and now you want to mark it as bad?\n>>> <what bisection log?>\n>> Alan provided his bisection log as an attachment to the original bug\n>> report.\n>>\n>>> I assume that Alan's bisection search ended up saying that the merge\n>>> commit (1c89ac55017f982355c7761e1c912c88c941483d) was the first bad\n>>> commit.\n>> Yep, and that's totally correct as far as bisect is concerned. The\n>> parents of that merge commit are:\n>> 88fa08f67bee1a0c765237bdac106a32872f57d2\n>> b1b135c8d619cb2c7045d6ee4e48375882518bb5\n>>\n>> And Alan marked both of them as good.\n>>\n>> So, unless Alan made a mistake during his bisection, each of the\n>> branches is correct, but the merge did not lead to a correct result. So\n>> while there were no textual conflicts, there were still incompatible\n>> changes regarding the code semantics and compatibility was not restored\n>> during the merge.\n> \n> It's important to note that even if I did make a mistake during the\n> bisection process (and I certainly wouldn't discount that), recent\n> kernels still fail: but when I take out that commit from a recent\n> kernel, it fails.\n\ns/fails/succeeds/\n\n> \n> I put in Andrew's suggested patch (to help find things), and now I\n> repeatedly get the problems in the attached log.\n> \n> Not being an x86 knowledgeable person, I'm a bit concerned about the RSP\n> value?! (I enabled stack overflow checking, but that didn't stop things.)\n> \n> Alan\n> \n"},{"id":"88196","messageId":"7v7ia8ahgu.fsf@gitster.siamese.dyndns.org","threadId":"15166","inReplyTo":"20080822105136.a8432875.akpm@linux-foundation.org","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-08-22T21:05:21Z","receivedAt":"2008-08-22T21:05:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Andrew Morton <akpm@linux-foundation.org> writes:\n\n> On Fri, 22 Aug 2008 19:16:51 +0200\n> Petr Baudis <pasky@suse.cz> wrote:\n>\n>> On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote:\n> ...\n>> > urgh, it's irritating when git-bisect directs you to a merge commit - it\n>> > hasn't done it for me for ages.\n>> \n>> Hmm, but doesn't that happen only when it's actually really the merge\n>> commit that introduces the bug? Both parents of the merge commit were\n>> marked as good by the user, so...\n>\n> A merge commit doesn't contain any kernel changes?  It's the individual\n> commits (aka \"patches\") which were in that merge which broke stuff. \n> Confused.\n>\n> We're trying to dive inside that merge commit to find out which of the\n> real commits caused the regression.\n\nYou may find neither parents were buggy, but the result of the merge is.\n\nA trivial example is when one branch changes the semantics of an existing\nfunction and converts all the call sites to the updated semantics, while\nthe other branch adds a new call site that still relies on the old\nbehaviour of that function.  The merge most likely won't textually\nconflict, and neither git merge nor quilt patch would report conflicts,\nbut the end result is that the new call site added by the latter branch\nnow gets an unexpected outcome from the function and can misbehave.  You\ncannot blame the breakage to either branch for such a breakage.\n"},{"id":"88201","messageId":"20080822141651.fe16ed99.akpm@linux-foundation.org","threadId":"15166","inReplyTo":"7v7ia8ahgu.fsf@gitster.siamese.dyndns.org","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Andrew Morton","fromEmail":"akpm@linux-foundation.org","sentAt":"2008-08-22T21:16:51Z","receivedAt":"2008-08-22T21:16:51Z","isPatch":false,"sender":{"key":"akpm@linux-foundation.org","avatar":null},"body":"On Fri, 22 Aug 2008 14:05:21 -0700\nJunio C Hamano <gitster@pobox.com> wrote:\n\n> Andrew Morton <akpm@linux-foundation.org> writes:\n> \n> > On Fri, 22 Aug 2008 19:16:51 +0200\n> > Petr Baudis <pasky@suse.cz> wrote:\n> >\n> >> On Fri, Aug 22, 2008 at 09:25:49AM -0700, Andrew Morton wrote:\n> > ...\n> >> > urgh, it's irritating when git-bisect directs you to a merge commit - it\n> >> > hasn't done it for me for ages.\n> >> \n> >> Hmm, but doesn't that happen only when it's actually really the merge\n> >> commit that introduces the bug? Both parents of the merge commit were\n> >> marked as good by the user, so...\n> >\n> > A merge commit doesn't contain any kernel changes?  It's the individual\n> > commits (aka \"patches\") which were in that merge which broke stuff. \n> > Confused.\n> >\n> > We're trying to dive inside that merge commit to find out which of the\n> > real commits caused the regression.\n> \n> You may find neither parents were buggy, but the result of the merge is.\n> \n> A trivial example is when one branch changes the semantics of an existing\n> function and converts all the call sites to the updated semantics, while\n> the other branch adds a new call site that still relies on the old\n> behaviour of that function.  The merge most likely won't textually\n> conflict, and neither git merge nor quilt patch would report conflicts,\n> but the end result is that the new call site added by the latter branch\n> now gets an unexpected outcome from the function and can misbehave.  You\n> cannot blame the breakage to either branch for such a breakage.\n\nSure, but\n\na) whoever added a change like that without also causing the build to\n   break is slappable and\n\nb) there are now two commits (one in each branch) either one of\n   which, when applied on top of the other branch will introduce the\n   regression.\n\n   That's useful infomation, but we don't know how to get\n   git-bisect to give it to us.\n\n\nIt's pretty simple.  If git-bisect tells us that the regression was\nintroduced by a merge commit, we want to perform a bisection within\nthat merge's individual commits.\n"},{"id":"88205","messageId":"20080822212121.GA31546@coredump.intra.peff.net","threadId":"15166","inReplyTo":"20080822141651.fe16ed99.akpm@linux-foundation.org","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-08-22T21:21:21Z","receivedAt":"2008-08-22T21:21:21Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Aug 22, 2008 at 02:16:51PM -0700, Andrew Morton wrote:\n\n> Sure, but\n> \n> a) whoever added a change like that without also causing the build to\n>    break is slappable and\n\nWell, the point is to find out _who_ to slap. ;)\n\n> b) there are now two commits (one in each branch) either one of\n>    which, when applied on top of the other branch will introduce the\n>    regression.\n> \n>    That's useful infomation, but we don't know how to get\n>    git-bisect to give it to us.\n>\n> It's pretty simple.  If git-bisect tells us that the regression was\n> introduced by a merge commit, we want to perform a bisection within\n> that merge's individual commits.\n\nI mentioned linearizing the two lines of development in another\nmessage in this thread. But there are, of course, two linearizations,\nneither of which might be possible to generate automatically, and one\nmight give you a better answer than the other.\n\n-Peff\n"},{"id":"88209","messageId":"20080822213618.GC1598@atjola.homenet","threadId":"15166","inReplyTo":"20080822141651.fe16ed99.akpm@linux-foundation.org","subject":"Re: Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-08-22T21:36:18Z","receivedAt":"2008-08-22T21:36:18Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2008.08.22 14:16:51 -0700, Andrew Morton wrote:\n> It's pretty simple.  If git-bisect tells us that the regression was\n> introduced by a merge commit, we want to perform a bisection within\n> that merge's individual commits.\n\nbisect already did that. It asked for the left side and the right side,\nboth were good. What you can still do is creating a _new_ history where\nthe commits are not in parallel but linearized, like Jeff described. But\nthat's (in general) not a trivial task as you need to reapply individual\ncommit patches which can cause conflicts that were already solved in the\nexisting merges to show up again. Or, it can even produce new conflicts,\nfor example when a commit on one side was reverted before the merge\nhappened. In that case, you get into a state with changes that were\nnever visible before.\n\nBjörn\n"}]}