Why do the server got stuck stuck for 30 minutes and then recovered back alone ?

Solution In Progress - Updated -

Issue

The server got stuck and any of the instances were reachable during 30 minutes, after that, the server recovered alone. The following traces were present on /var/log/messages

Mar 27 12:38:11 myserver kernel: Call Trace:
Mar 27 12:38:11 myserver kernel: [<ffffffff8163a909>] schedule+0x29/0x70
Mar 27 12:38:11 myserver kernel: [<ffffffff8163c0c5>] rwsem_down_write_failed+0x115/0x220
Mar 27 12:38:11 myserver kernel: [<ffffffff81301813>] call_rwsem_down_write_failed+0x13/0x20
Mar 27 12:38:11 myserver kernel: [<ffffffff8128be70>] ? file_map_prot_check+0xd0/0xd0
Mar 27 12:38:11 myserver kernel: [<ffffffff81639b5d>] ? down_write+0x2d/0x30
Mar 27 12:38:11 myserver kernel: [<ffffffff81187a1c>] vm_mmap_pgoff+0x8c/0xe0
Mar 27 12:38:11 myserver kernel: [<ffffffff8108fddb>] ? recalc_sigpending+0x1b/0x50
Mar 27 12:38:11 myserver kernel: [<ffffffff81090731>] ? __set_task_blocked+0x41/0xa0
Mar 27 12:38:11 myserver kernel: [<ffffffff8119cb86>] SyS_mmap_pgoff+0x116/0x270
Mar 27 12:38:11 myserver kernel: [<ffffffff81019712>] SyS_mmap+0x22/0x30
Mar 27 12:38:11 myserver kernel: [<ffffffff81645909>] system_call_fastpath+0x16/0x1b
Mar 27 12:38:11 myserver kernel: INFO: task qemu-kvm:30279 blocked for more than 120 seconds.
Mar 27 12:38:11 myserver kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 27 12:38:11 myserver kernel: qemu-kvm        D ffff883f26f0f300     0 30279      1 0x00000080
Mar 27 12:38:11 myserver kernel: ffff887df0d33de8 0000000000000086 ffff8863eaf25c00 ffff887df0d33fd8
Mar 27 12:38:11 myserver kernel: ffff887df0d33fd8 ffff887df0d33fd8 ffff8863eaf25c00 ffff8863eaf25c00
Mar 27 12:38:11 myserver kernel: ffff887ee991f738 ffffffffffffffff ffff887ee991f740 00000000000000a8

Environment

  • Openstack Nova
  • RHEV
  • KSM

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content