Kernel panic due to hard lockup when attempting to trigger multiple sysrq-t events

Solution Verified - Updated -

Issue

  • NMI watchdog detected hard lockup when issuing multiple SysRg-T magic key sequences.

  • The system crashed with the following call traces:

crash> log
[..]
[1141515.690181] Kernel panic - not syncing: Hard LOCKUP
[1141515.690182] CPU: 14 PID: 75114 Comm: peer_down.sh Kdump: loaded Tainted: GOEL ------------ T 3.10.0-1127.18.2.el7.x86_64 #1
[1141515.690183] Hardware name: HPE Synergy 480 Gen10/Synergy 480 Gen10 Compute Module, BIOS I42 03/09/2020
[1141515.690183] Call Trace:
[1141515.690185]  <NMI>  [<ffffffff9637ffa5>] dump_stack+0x19/0x1b
[1141515.690186]  [<ffffffff96379541>] panic+0xe8/0x21f
[1141515.690188]  [<ffffffff96392ed2>] ? system_call_fastpath+0x25/0x2a
[1141515.690191]  [<ffffffff95c9ba6f>] nmi_panic+0x3f/0x40
[1141515.690193]  [<ffffffff95d4f6f1>] watchdog_overflow_callback+0x121/0x140
[1141515.690194]  [<ffffffff95da90e7>] __perf_event_overflow+0x57/0x100
[1141515.690196]  [<ffffffff95db28e4>] perf_event_overflow+0x14/0x20
[1141515.690197]  [<ffffffff95c0a9a0>] handle_pmi_common+0x1a0/0x250
[1141515.690200]  [<ffffffff95c0ac7f>] intel_pmu_handle_irq+0xcf/0x1d0
[1141515.690202]  [<ffffffff96389031>] perf_event_nmi_handler+0x31/0x50
[1141515.690203]  [<ffffffff9638a93c>] nmi_handle.isra.0+0x8c/0x150
[1141515.690205]  [<ffffffff9638ab5d>] do_nmi+0x15d/0x460
[1141515.690207]  [<ffffffff96389d9c>] end_repeat_nmi+0x1e/0x81
[1141515.690213]  [<ffffffffc0491250>] ? sys_imageblit+0x250/0x1000 [sysimgblt]
[1141515.690217]  [<ffffffffc0491250>] ? sys_imageblit+0x250/0x1000 [sysimgblt]
[1141515.690221]  [<ffffffffc0491250>] ? sys_imageblit+0x250/0x1000 [sysimgblt]
[1141515.690228]  <EOE>  [<ffffffffc0553737>] drm_fb_helper_sys_imageblit+0x17/0x40 [drm_kms_helper]
[1141515.690233]  [<ffffffffc053776b>] mga_imageblit+0x1b/0x40 [mgag200]
[1141515.690234]  [<ffffffff9600a62d>] bit_putcs+0x31d/0x5a0
[1141515.690238]  [<ffffffffc053762f>] ? mga_dirty_update+0x1ef/0x310 [mgag200]
[1141515.690242]  [<ffffffffc053777f>] ? mga_imageblit+0x2f/0x40 [mgag200]
[1141515.690244]  [<ffffffff9600a2d3>] ? bit_cursor+0x663/0x6a0
[1141515.690246]  [<ffffffff96005a2b>] fbcon_putcs+0x12b/0x160
[1141515.690247]  [<ffffffff9600a310>] ? bit_cursor+0x6a0/0x6a0
[1141515.690249]  [<ffffffff96005d77>] fbcon_redraw.isra.24+0x197/0x200
[1141515.690250]  [<ffffffff96006ca4>] fbcon_scroll+0x364/0xda0
[1141515.690252]  [<ffffffff9607e1ff>] scrup+0x16f/0x180
[1141515.690253]  [<ffffffff9607e2b0>] lf+0xa0/0xb0
[1141515.690254]  [<ffffffff9607e592>] vt_console_print+0x2d2/0x430
[1141515.690256]  [<ffffffff95c9cfb3>] call_console_drivers.constprop.19+0x93/0xf0
[1141515.690257]  [<ffffffff95c9e7b8>] console_unlock+0x468/0x4b0
[1141515.690258]  [<ffffffff95c9ebc4>] vprintk_emit+0x3c4/0x510
[1141515.690260]  [<ffffffff95c9ef79>] vprintk_default+0x29/0x40
[1141515.690261]  [<ffffffff963796d8>] printk+0x60/0x77
[1141515.690263]  [<ffffffff95cf5cc4>] print_rt_rq+0x214/0x2b0
[1141515.690264]  [<ffffffff95cee055>] print_rt_stats+0x95/0xd0
[1141515.690266]  [<ffffffff95cf45a4>] print_cpu+0x2d4/0xe40
[1141515.690268]  [<ffffffff95cf5d91>] sysrq_sched_debug_show+0x31/0x60
[1141515.690270]  [<ffffffff95cdac29>] show_state_filter+0xa9/0xd0
[1141515.690272]  [<ffffffff96073940>] sysrq_handle_showstate+0x10/0x20
[1141515.690273]  [<ffffffff960740dd>] __handle_sysrq+0x10d/0x170
[1141515.690274]  [<ffffffff96074548>] write_sysrq_trigger+0x28/0x40
[1141515.690276]  [<ffffffff95ec6280>] proc_reg_write+0x40/0x80
[1141515.690277]  [<ffffffff95e4d1d0>] vfs_write+0xc0/0x1f0
[1141515.690279]  [<ffffffff95e4df9f>] SyS_write+0x7f/0xf0
[1141515.690281]  [<ffffffff96392ed2>] system_call_fastpath+0x25/0x2a
[..]

Environment

  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise Linux 8.2
  • kernel
  • SysRq

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content