NMI Watchdog caused kernel panic using Intel Xeon E3 / E5 v3 CPU when server is idle

Issue

• NMI Watchdog encountered when the system is idle. Visible as the following in a vmcore captured during the event:

Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 22
Pid: 0, comm: swapper Tainted: P           ---------------    2.6.32-504.16.2.el6.x86_64 #1
Call Trace:
 <NMI>  [<ffffffff8152971c>] ? panic+0xa7/0x16f
 [<ffffffff81014959>] ? sched_clock+0x9/0x10
 [<ffffffff810ea6cd>] ? watchdog_overflow_callback+0xcd/0xd0
 [<ffffffff81120e97>] ? __perf_event_overflow+0xa7/0x240
 [<ffffffff8101dacd>] ? x86_perf_event_set_period+0xdd/0x170
 [<ffffffff811214e4>] ? perf_event_overflow+0x14/0x20
 [<ffffffff81024892>] ? intel_pmu_handle_irq+0x202/0x410
 [<ffffffff8152ea19>] ? perf_event_nmi_handler+0x39/0xb0
 [<ffffffff815304d5>] ? notifier_call_chain+0x55/0x80
 [<ffffffff8153053a>] ? atomic_notifier_call_chain+0x1a/0x20
 [<ffffffff810a4f5e>] ? notify_die+0x2e/0x30
 [<ffffffff8152e19b>] ? do_nmi+0x1bb/0x340
 [<ffffffff8152da60>] ? nmi+0x20/0x30
 [<ffffffff812eaae1>] ? intel_idle+0xb1/0x170
 <<EOE>>  [<ffffffff81427238>] ? menu_select+0x178/0x390
 [<ffffffff81426117>] ? cpuidle_idle_call+0xa7/0x140
 [<ffffffff81009fc6>] ? cpu_idle+0xb6/0x110
 [<ffffffff8152328d>] ? start_secondary+0x2be/0x301

• Missed hrtimer interrupt