AMD system panics after updating to kernel-3.10.0-327.4.5.el7

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 7, kernel-3.10.0-327.4.5.el7

  • AMD Phenom(tm) II X6 CPU

Issue

  • The system won't boot after update to kernel kernel-3.10.0-327.4.5.el7.x86_64

  • The system presents a crash with sysfs_show_current_clocksources instruction in stack trace.

Resolution

  • Update the kernel to 3.10.0-514.el7 (released with RHSA-2016-2574) or later, which contains fixes for this issue.

  • This fix is part of RHEL7.3GA and later. This fix was investigated in a private Bugzilla 1265283.

  • As a workaround, revert your system to a previous Kernel version.

Root Cause

  • This issue is a livelock race condition when tick_do_broadcast() is called and the clocksource has just switched to a new clocksource.

  • In this case the broadcast percpu event handler will be called which will attempt to reacquire the tick_broadcast_lock(), and then cause the deadlock.

Diagnostic Steps

  • Sample stack trace:
[11000.649584] BUG: unable to handle kernel NULL pointer dereference at 0000000000000030
[11000.649663] IP: [<ffffffff810da94f>] sysfs_show_current_clocksources+0x2f/0x50
[11000.649733] PGD 271fec067 PUD 273091067 PMD 0 
[11000.649779] Oops: 0000 [#1] SMP 
[11000.649811] Modules linked in: xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter nf_nat nf_conntrack bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop dm_mod ext4 mbcache jbd2 pcspkr iTCO_wdt iTCO_vendor_support coretemp sg acpi_power_meter kvm_intel i7core_edac edac_core lpc_ich mfd_core kvm i2c_i801 tpm_infineon shpchp acpi_cpufreq ipmi_si ipmi_devintf ipmi_msghandler ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic crct10dif_common mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit drm_kms_helper ttm drm crc32c_intel serio_raw e1000e ahci i2c_core libahci libata ptp pps_core
[11000.650497] CPU: 7 PID: 6670 Comm: cat Not tainted 3.10.0-327.4.5.el7.x86_64 #1
[11000.650559] Hardware name: FUJITSU                          PRIMERGY RX100 S6             /D2863, BIOS 6.00 Rev. 1.08.2863.A1           03/01/2010
[11000.650665] task: ffff880271835c00 ti: ffff88027236c000 task.ti: ffff88027236c000
[11000.650727] RIP: 0010:[<ffffffff810da94f>]  [<ffffffff810da94f>] sysfs_show_current_clocksources+0x2f/0x50
[11000.650811] RSP: 0018:ffff88027236fe88  EFLAGS: 00010246
[11000.650856] RAX: 0000000000000000 RBX: ffff8802725a5000 RCX: ffff88027236ffd8
[11000.650914] RDX: ffffffff81851d84 RSI: 0000000000001000 RDI: ffff8802725a5000
[11000.650973] RBP: ffff88027236fe90 R08: ffffffff818779b7 R09: ffffea0009c96980
[11000.651031] R10: 0000000000003f80 R11: 0000000000000246 R12: ffff88027236ff48
[11000.651089] R13: 0000000000010000 R14: ffff880273ce1540 R15: ffff880273a13070
[11000.651149] FS:  00007f90da946740(0000) GS:ffff88027fdc0000(0000) knlGS:0000000000000000
[11000.651232] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11000.651282] CR2: 0000000000000030 CR3: 000000026e954000 CR4: 00000000000007e0
[11000.651340] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11000.651399] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11000.651456] Stack:
[11000.651475]  ffffffff8198b000 ffff88027236feb0 ffffffff813f1ba0 ffff88027236feb0
[11000.651546]  ffff880273ce1560 ffff88027236ff00 ffffffff8125920a ffffffff8198ac90
[11000.651616]  ffffffff816bdd50 0000000000a08000 ffff88027062ce00 0000000000a08000
[11000.651686] Call Trace:
[11000.651714]  [<ffffffff813f1ba0>] dev_attr_show+0x20/0x60
[11000.651763]  [<ffffffff8125920a>] sysfs_read_file+0x9a/0x1a0
[11000.651813]  [<ffffffff811de3bc>] vfs_read+0x9c/0x170
[11000.651857]  [<ffffffff811def0f>] SyS_read+0x7f/0xe0
[11000.651903]  [<ffffffff816458c9>] system_call_fastpath+0x16/0x1b
[11000.651952] Code: 90 55 48 c7 c7 20 b0 98 81 48 89 e5 53 48 89 d3 e8 67 e1 55 00 48 8b 05 00 c6 cd 00 48 89 df 48 c7 c2 84 1d 85 81 be 00 10 00 00 <48> 8b 48 30 31 c0 e8 46 4d 22 00 48 c7 c7 20 b0 98 81 89 c3 e8 
[11000.656788] RIP  [<ffffffff810da94f>] sysfs_show_current_clocksources+0x2f/0x50
[11000.659172]  RSP <ffff88027236fe88>
[11000.661536] CR2: 0000000000000030

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.