soft lockups in opal_get_rtc_time()/opal_set_rtc_time() on ppc64le under RHEL7
Issue
Kernel experiences numerous softlockup events that look like:
[8800618.529482] NMI watchdog: BUG: soft lockup - CPU#106 stuck for 22s! [systemd-timedat:178328]
[8800618.529594] Modules linked in: unix_diag 8021q garp mrp ip6t_rpfilter bonding ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6
nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip
6_tables iptable_filter i2c_dev ext4 mbcache jbd2 ses enclosure scsi_transport_sas sg ofpart powernv_flash ibmpowernv shpchp ipmi_powernv mtd ipmi_msghandler at24 powernv_rng opal_prd i2c_opal binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect
[8800618.530433] sysimgblt fb_sys_fops ttm drm ahci libahci i40e libata ixgbe megaraid_sas i2c_core mdio ptp pps_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ipmi_devintf]
[8800618.530692] CPU: 106 PID: 178328 Comm: systemd-timedat Not tainted 3.10.0-693.37.4.el7.ppc64le #1
[8800618.530762] task: c00000394969ab80 ti: c000003338770000 task.ti: c000003338770000
[8800618.530822] NIP: c00000000077e338 LR: c000000000083898 CTR: 00000000300378e8
[8800618.530882] REGS: c000003338753970 TRAP: 0901 Not tainted (3.10.0-693.37.4.el7.ppc64le)
[8800618.530952] MSR: 9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE> CR: 44022882 XER: 20000000
[8800618.531103] CFAR: c0000000000838d0 SOFTE: 1
GPR00: 0000000030005128 c000003338753bf0 c00000000121f200 0000000000000000
GPR04: 0000000044022882 c00000000077e338 9000000000009033 0000000000000422
GPR08: 0000000000000000 00000000300b3218 c0000000000838b4 9000000000001003
GPR12: c00000000008388c c000000007b9ba00
[8800618.531478] NIP [c00000000077e338] opal_get_rtc_time+0x98/0xb0
[8800618.531530] LR [c000000000083898] opal_return+0xc/0x48
[8800618.531570] Call Trace:
[8800618.531592] [c000003338753bf0] [c00000000077e338] opal_get_rtc_time+0x98/0xb0 (unreliable)
[8800618.531673] [c000003338753c30] [c0000000007769fc] rtc_read_time+0xac/0xf0
[8800618.531733] [c000003338753c70] [c000000000778b68] rtc_dev_ioctl+0x478/0x610
[8800618.531793] [c000003338753d20] [c00000000034c748] do_vfs_ioctl+0x428/0x890
[8800618.531887] [c000003338753dd0] [c00000000034cc74] SyS_ioctl+0xc4/0xe0
[8800618.531949] [c000003338753e30] [c00000000000a284] system_call+0x38/0xfc
[8800618.532008] Instruction dump: ...
Eventually, kernel panics.
Environment
- RHEL7
- Power machine with OPAL (OpenPower Abstraction Layer)/OpenPOWER firmware
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.