RHEL6: Watchdog detected hard LOCKUP on cpu X due to runqueue spinlock lock inversion

Issue

System panicked with:

Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 11
Pid: 27559, comm: java Not tainted 2.6.32-358.el6.x86_64 #1
Call Trace:
 <NMI>  [<ffffffff8150cfc8>] ? panic+0xa7/0x16f
 [<ffffffff810e10fd>] ? watchdog_overflow_callback+0xcd/0xd0
 [<ffffffff81116f00>] ? __perf_event_overflow+0xb0/0x2a0
 [<ffffffff8101a7bd>] ? x86_perf_event_update+0x5d/0xb0
 [<ffffffff8101b74d>] ? x86_perf_event_set_period+0xdd/0x170
 [<ffffffff81117524>] ? perf_event_overflow+0x14/0x20
 [<ffffffff810207e2>] ? intel_pmu_handle_irq+0x192/0x300
 [<ffffffff81512ba6>] ? kprobe_exceptions_notify+0x16/0x430
 [<ffffffff81511719>] ? perf_event_nmi_handler+0x39/0xb0
 [<ffffffff815131d5>] ? notifier_call_chain+0x55/0x80
 [<ffffffff8151323a>] ? atomic_notifier_call_chain+0x1a/0x20
 [<ffffffff8109cbfe>] ? notify_die+0x2e/0x30
 [<ffffffff81510e9b>] ? do_nmi+0x1bb/0x340
 [<ffffffff81510760>] ? nmi+0x20/0x30
 [<ffffffff8150ffce>] ? _spin_lock+0x1e/0x30
 <<EOE>>  [<ffffffff81055bbe>] ? double_lock_balance+0x4e/0x80
 [<ffffffff81064124>] ? pull_rt_task+0xc4/0x190
 [<ffffffff810642d5>] ? pre_schedule_rt+0x25/0x30
 [<ffffffff8150d412>] ? schedule+0x132/0x3b2
 [<ffffffff8109bae5>] ? __hrtimer_start_range_ns+0x1a5/0x470
 [<ffffffff8109b1a1>] ? lock_hrtimer_base+0x31/0x60
 [<ffffffff8150f068>] ? schedule_hrtimeout_range+0xc8/0x160
 [<ffffffff8109ae00>] ? hrtimer_wakeup+0x0/0x30
 [<ffffffff8109bde4>] ? hrtimer_start_range_ns+0x14/0x20
 [<ffffffff811c775d>] ? ep_poll+0x2ad/0x330
 [<ffffffff81063310>] ? default_wake_function+0x0/0x20
 [<ffffffff811c78a5>] ? sys_epoll_wait+0xc5/0xe0
 [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b