Why do the server got stuck stuck for 30 minutes and then recovered back alone ?
Issue
The server got stuck and any of the instances were reachable during 30 minutes, after that, the server recovered alone. The following traces were present on /var/log/messages
Mar 27 12:38:11 myserver kernel: Call Trace:
Mar 27 12:38:11 myserver kernel: [<ffffffff8163a909>] schedule+0x29/0x70
Mar 27 12:38:11 myserver kernel: [<ffffffff8163c0c5>] rwsem_down_write_failed+0x115/0x220
Mar 27 12:38:11 myserver kernel: [<ffffffff81301813>] call_rwsem_down_write_failed+0x13/0x20
Mar 27 12:38:11 myserver kernel: [<ffffffff8128be70>] ? file_map_prot_check+0xd0/0xd0
Mar 27 12:38:11 myserver kernel: [<ffffffff81639b5d>] ? down_write+0x2d/0x30
Mar 27 12:38:11 myserver kernel: [<ffffffff81187a1c>] vm_mmap_pgoff+0x8c/0xe0
Mar 27 12:38:11 myserver kernel: [<ffffffff8108fddb>] ? recalc_sigpending+0x1b/0x50
Mar 27 12:38:11 myserver kernel: [<ffffffff81090731>] ? __set_task_blocked+0x41/0xa0
Mar 27 12:38:11 myserver kernel: [<ffffffff8119cb86>] SyS_mmap_pgoff+0x116/0x270
Mar 27 12:38:11 myserver kernel: [<ffffffff81019712>] SyS_mmap+0x22/0x30
Mar 27 12:38:11 myserver kernel: [<ffffffff81645909>] system_call_fastpath+0x16/0x1b
Mar 27 12:38:11 myserver kernel: INFO: task qemu-kvm:30279 blocked for more than 120 seconds.
Mar 27 12:38:11 myserver kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 27 12:38:11 myserver kernel: qemu-kvm D ffff883f26f0f300 0 30279 1 0x00000080
Mar 27 12:38:11 myserver kernel: ffff887df0d33de8 0000000000000086 ffff8863eaf25c00 ffff887df0d33fd8
Mar 27 12:38:11 myserver kernel: ffff887df0d33fd8 ffff887df0d33fd8 ffff8863eaf25c00 ffff8863eaf25c00
Mar 27 12:38:11 myserver kernel: ffff887ee991f738 ffffffffffffffff ffff887ee991f740 00000000000000a8
Environment
- Openstack Nova
- RHEV
- KSM
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.