[RHEL 4.4] Panic, bad page state

Issue

We just had a kernel panic with the following output:

2010 Feb 4 20:31:24 localhost kern.emerg kernel: Bad page state at free\_hot\_cold_page (in process 'cmanicd', page 000001022fb3bb38)  
2010 Feb 4 20:31:24 localhost kern.emerg kernel: flags:0x01000014 mapping:0000000000000000 mapcount:1 count:0  
2010 Feb 4 20:31:24 localhost kern.emerg kernel: Backtrace:  
2010 Feb 4 20:31:24 localhost kern.warning kernel:  
2010 Feb 4 20:31:24 localhost kern.warning kernel: Call Trace:<ffffffff8015dc67>{bad\_page+112} <ffffffff8015e349>{free\_hot\_cold\_page+130}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff8015e402>{_\_pagevec\_free+39} <ffffffff801645e7>{release_pages+366}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff801715ee>{free\_pages\_and\_swap\_cache+118} <ffffffff801688c9>{unmap_vmas+1440}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff8016d9f3>{exit_mmap+144} <ffffffff80135df3>{mmput+102}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff8013a292>{do\_exit+646} <ffffffff8014399a>{sys\_rt_sigprocmask+88}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff8013ad6c>{sys\_exit\_group+0} <ffffffff80126257>{cstar\_do\_call+27}  
2010 Feb 4 20:31:24 localhost kern.warning kernel:  
2010 Feb 4 20:31:24 localhost kern.emerg kernel: Trying to fix it up, but a reboot is needed  
2010 Feb 4 20:31:24 localhost kern.emerg kernel: Bad page state at prep\_new\_page (in process 'cmanicd', page 000001022fb3bb38)  
2010 Feb 4 20:31:24 localhost kern.emerg kernel: flags:0x01000004 mapping:0000000000000000 mapcount:1 count:1  
2010 Feb 4 20:31:24 localhost kern.emerg kernel: Backtrace:  
2010 Feb 4 20:31:24 localhost kern.warning kernel:  
2010 Feb 4 20:31:24 localhost kern.warning kernel: Call Trace:<ffffffff8015dc67>{bad\_page+112} <ffffffff8015e625>{buffered\_rmqueue+520}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff8015e763>{_\_alloc\_pages+211} <ffffffff801698e2>{do\_wp\_page+484}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff8016aa58>{handle\_mm\_fault+1235} <ffffffff801342e7>{_\_wake\_up_sync+74}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff801243f4>{do\_page\_fault+520} <ffffffff80132fca>{schedule_tail+202}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: <ffffffff80110d91>{error_exit+0}  
2010 Feb 4 20:31:24 localhost kern.emerg kernel: Trying to fix it up, but a reboot is needed  
2010 Feb 4 20:31:24 localhost kern.warning kernel: \---\---\----- [cut here ] \---\---\--- [please bite here ] \---\---\---  
2010 Feb 4 20:31:24 localhost kern.alert kernel: Kernel BUG at rmap:479  
2010 Feb 4 20:31:24 localhost kern.emerg kernel: invalid operand: 0000 [1] SMP  
2010 Feb 4 20:31:24 localhost kern.warning kernel: CPU 1  
2010 Feb 4 20:31:24 localhost kern.warning kernel: Modules linked in: cpqci(U) ipmi\_si(U) ipmi\_devintf(U) ipmi\_msghandler(U) 8021q mptctl mptbase sg arptable\_filter arp\_tables md5 ipv6 nfs lockd nfs\_acl sunrpc autofs4 i2c\_dev i2c\_core dm\_mirror dm\_multipath dm\_mod button battery ac joydev uhci\_hcd ohci\_hcd ehci\_hcd k8\_edac edac\_mc bnx2(U) ext3 jbd cciss sd\_mod scsi\_mod  
2010 Feb 4 20:31:24 localhost kern.warning kernel: Pid: 22599, comm: cmanicd Tainted: P B 2.6.9-55.0.2.ELsmp  
2010 Feb 4 20:31:24 localhost kern.warning kernel: RIP: 0010:[<ffffffff8016fac3>] <ffffffff8016fac3>{page\_remove\_rmap+39}  
2010 Feb 4 20:31:24 localhost kern.warning kernel: RSP: 0018:0000010203c7ddd8 EFLAGS: 00010286  
2010 Feb 4 20:31:24 localhost kern.warning kernel: RAX: 00000000ffffffff RBX: 0000010008008360 RCX: 0000010008008360  
2010 Feb 4 20:31:24 localhost kern.warning kernel: RDX: 0000000000000000 RSI: 000000000021a3a1 RDI: 000001022fb3bb38  
2010 Feb 4 20:31:24 localhost kern.warning kernel: RBP: 000001022fb3bb38 R08: ffffffffffffffff R09: 0000000000000000  
2010 Feb 4 20:31:24 localhost kern.warning kernel: R10: 0000000000000000 R11: 0000000000000256 R12: 0000000000000000  
2010 Feb 4 20:31:24 localhost kern.warning kernel: R13: 0000010223b1d600 R14: 000000021a3a1000 R15: 00000000000c0000  
2010 Feb 4 20:31:24 localhost kern.warning kernel: FS: 0000002a955873e0(0000) GS:ffffffff804edc00(0000) knlGS:0000000055be3bb0  
2010 Feb 4 20:31:24 localhost kern.warning kernel: CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b  
2010 Feb 4 20:31:24 localhost kern.warning kernel: CR2: 00000000001a4e30 CR3: 0000000228538000 CR4: 00000000000006e0  
2010 Feb 4 20:31:24 localhost kern.warning kernel: Process cmanicd (pid: 22599, threadinfo 0000010203c7c000, task 00000101f3cd57f0)  
2010 Feb 4 20:31:24 localhost kern.warning kernel: Stack: 0000000000000000 ffffffff8016886a 0000000000200000 0000000009af4000  
2010 Feb 4 20:31:24 localhost kern.warning kernel: 0000010009ee0260 0000000009800000 000001007d82d000 0000000009af4000  
2010 Feb 4 20:31:24 localhost kern.warning kernel: 00000000096f4000 0000010008008360