Oracle 12c throwing error: kernel: NMI watchdog: BUG: soft lockup - CPU#X stuck for Xs!
Issue
o Repeated soft lockup warnings noted in dmesg
or syslog
crash> log | grep "BUG: soft lockup" | head
[1543147.645191] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 23s! [ofsFcbPrCb:108055]
[1543175.646127] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 23s! [ofsFcbPrCb:108055]
[1543215.647464] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 22s! [ofsFcbPrCb:108055]
[1543243.648400] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 22s! [ofsFcbPrCb:108055]
[1543271.649335] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 22s! [ofsFcbPrCb:108055]
[1543299.650272] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 22s! [ofsFcbPrCb:108055]
[1543327.651209] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 22s! [ofsFcbPrCb:108055]
[1543355.652144] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 22s! [ofsFcbPrCb:108055]
[1543395.653481] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 23s! [ofsFcbPrCb:108055]
[1543423.654418] NMI watchdog: BUG: soft lockup - CPU#62 stuck for 23s! [ofsFcbPrCb:108055]
o Stack traces similar to below:
crash> bt
PID: 108055 TASK: ffff8801196a9fa0 CPU: 62 COMMAND: "ofsFcbPrCb"
#0 [ffff887e7d988e48] crash_nmi_callback at ffffffff810502b7
#1 [ffff887e7d988e58] nmi_handle at ffffffff816b892c
#2 [ffff887e7d988eb0] do_nmi at ffffffff816b8b5d
#3 [ffff887e7d988ef0] end_repeat_nmi at ffffffff816b7d89
[exception RIP: lockref_put_or_lock+0x8]
RIP: ffffffff81338418 RSP: ffff880100777c10 RFLAGS: 00000286
RAX: 0000000000000000 RBX: ffff880107ccfbc0 RCX: 0000000200000000
RDX: 0000000000000000 RSI: ffff880f8096b448 RDI: ffff880107ccfc18
RBP: ffff880100777c18 R8: 0000000000000101 R9: ffff880100777bf0
R10: 0000000000000000 R11: 0000000000000000 R12: ffff880107ccfc18
R13: ffff880f8096af68 R14: ffff880f8096b330 R15: ffff880f8096aed0
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
--- <NMI exception stack> ---
#4 [ffff880100777c10] lockref_put_or_lock at ffffffff81338418
#5 [ffff880100777c20] dput at ffffffff8121d530
#6 [ffff880100777c40] OfsFcbPruneVCB at ffffffffc10d8f48 [oracleacfs]
#7 [ffff880100777d00] OfsFcbPrunerCb at ffffffffc10d91f3 [oracleacfs]
#8 [ffff880100777d40] KsOsdInvokeCallback at ffffffffc064f8ca [oracleoks]
#9 [ffff880100777d98] Ks_invokeCallback at ffffffffc0698751 [oracleoks]
#10 [ffff880100777db8] Ks_processRequests at ffffffffc0699f33 [oracleoks]
#11 [ffff880100777e58] Ks_main at ffffffffc069a5cb [oracleoks]
#12 [ffff880100777e98] KsKthreadRun at ffffffffc064f82a [oracleoks]
#13 [ffff880100777ec8] kthread at ffffffff810b4031
#14 [ffff880100777f50] ret_from_fork at ffffffff816c0577
Environment
o Red Hat Enterprise Linux Server release 7
o Oracle 12c
o kernel- 3.10.0-693.21.1.el7.x86_64
o oracleacfs
driver in use
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.