RHEL5.6+: Panic in raw_ctl_ioctl called by Oracle emagent due to memory corruption of raw_devices table
Issue
- RHEL5 servers kernel crashes, 10/300 servers crashed while running an informational gathering script provided by EMC.
- Upgrading from Red Hat Enterprise Linux 5.5 to Red Hat Enterprise Linux 5.6 causes the system to panic due to corruption of raw_devices table
-
Crash dump shows the following backtrace
Pid: 11265, comm: raw Not tainted 2.6.18-238.5.1.el5 #1 RIP: 0010:[<ffffffff801b8d5d>] [<ffffffff801b8d5d>] raw_ctl_ioctl+0x193/0x1fb RSP: 0018:ffff810ef5ed9e68 EFLAGS: 00010286 RAX: ffff81057b39d080 RBX: 000000000001e490 RCX: 0000000000000000 RDX: ffff000000000000 RSI: 00007fff12a67638 RDI: ffffffff8033f770 Process raw (pid: 11265, threadinfo ffff810ef5ed8000, task ffff8101606290c0) Stack: 00007fff00001e49 0000000000000000 0000000000000000 0000000000030002 0000000000000000 ffff810510c9d1c0 00007fff12a67620 000000000000ac01 0000000000000003 ffffffff8004241d ffff810510c9d1c0 00007fff12a67620 Call Trace: [] do_ioctl+0x55/0x6b [] vfs_ioctl+0x457/0x4b9 [] remove_vma+0x5d/0x64 [] sys_ioctl+0x59/0x78 [] system_call+0x7e/0x83 Code: 8b 02 c1 e8 14 89 c0 48 89 44 24 08 8b 02 25 ff ff 0f 00 48 RIP [] raw_ctl_ioctl+0x193/0x1fb
Environment
- Red Hat Enterprise Linux 5
- Kernels at least RHEL5.6 and above
- Seen on 5.6 2.6.18-238.5.1.el5, 2.6.18-348.1.1.el5, 2.6.18-348.4.1.el5, 2.6.18-407.el5
- iSCSI loaded, including rdma_* modules
- CCISS driver used for local storage
- Oracle
- HP firmware 8.6.3
- Problem did not occur on Red Hat Enterprise Linux 5.5 (2.6.18-194.el5)
- However, due to the small nature of the memory corruption, this could just be chance (kernel memory still corrupted, but in less fatal area)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Close
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.