PowerPC server can't boot after upgrade to 3.10.0-327.4.5.el7
Issue
After upgrade to kernel 3.10.0-327.4.5.el7 ppc64 or ppc64le servers can't boot due to panic early in boot process
[ 61.271589] Unable to handle kernel paging request for data at address 0x00000018
[ 61.271607] Faulting instruction address: 0xc0000000004a7380
[ 61.271612] Oops: Kernel access of bad area, sig: 11 [#1]
[ 61.271615] SMP NR_CPUS=2048 NUMA pSeries
[ 61.271619] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_comment xt_multiport xt_conntrack nf_conntrack iptable_filter pseries_rng ses enclosure sg nfsd auth_rpcgss nfs_acl lockd sunrpc grace binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_common ibmvscsi scsi_transport_srp ibmveth scsi_tgt ipr libata dm_mirror dm_region_hash dm_log dm_mod
[ 61.271666] CPU: 0 PID: 657 Comm: kworker/0:10 Tainted: G W ------------ 3.10.0-327.4.5.el7.ppc64 #1
[ 61.271671] Workqueue: events .topology_work_fn
[ 61.271675] task: c0000007e6721030 ti: c0000007e7d70000 task.ti: c0000007e7d70000
[ 61.271680] NIP: c0000000004a7380 LR: c00000000012c5cc CTR: 0000000000000000
[ 61.271684] REGS: c0000007e7d73520 TRAP: 0300 Tainted: G W ------------ (3.10.0-327.4.5.el7.ppc64)
[ 61.271688] MSR: 8000000100009032 <SF,EE,ME,IR,DR,RI> CR: 48848024 XER: 00000000
[ 61.271699] CFAR: c000000000009368 DAR: 0000000000000018 DSISR: 40000000 SOFTE: 1
GPR00: c00000000012c5cc c0000007e7d737a0 c0000000013822d0 0000000000000018
GPR04: 0000000000000800 0000000000000000 0000000000000000 0000000000000000
GPR08: 0000000000000000 00000000010b0000 c000000000d36840 0000000000000000
GPR12: 0000000000000000 c000000007b30000 c0000007e6b61400 00000000000000b8
GPR16: 0000000000000800 c0000007e538fe00 0000000000000000 c0000007e538fe00
GPR20: 00000000000000f8 c0000000013d09d8 0000000000000001 0000000000000001
GPR24: c00000000129c147 0000000000000020 0000000000000018 0000000000000800
GPR28: 0000000000000020 0000000000000110 0000000000000000 0000000000000010
[ 61.271760] NIP [c0000000004a7380] .__bitmap_weight+0x70/0x100
[ 61.271765] LR [c00000000012c5cc] .build_sched_domains+0xd3c/0x10b0
[ 61.271768] Call Trace:
[ 61.271772] [c0000007e7d737a0] [c0000000002be454] .kmem_cache_alloc_node_trace+0xb4/0x3a0 (unreliable)
[ 61.271780] [c0000007e7d73850] [c00000000012c5cc] .build_sched_domains+0xd3c/0x10b0
[ 61.271788] [c0000007e7d739a0] [c00000000012d154] .partition_sched_domains+0x484/0x510
[ 61.271794] [c0000007e7d73aa0] [c000000000196518] .rebuild_sched_domains+0x68/0xa0
[ 61.271799] [c0000007e7d73b30] [c000000000063b50] .topology_work_fn+0x10/0x30
[ 61.271804] [c0000007e7d73ba0] [c0000000000ff7d0] .process_one_work+0x1d0/0x690
[ 61.271809] [c0000007e7d73c50] [c0000000000ffe10] .worker_thread+0x180/0x500
[ 61.271814] [c0000007e7d73d30] [c00000000010be28] .kthread+0xe8/0xf0
[ 61.271820] [c0000007e7d73e30] [c00000000000a470] .ret_from_kernel_thread+0x58/0x68
[ 61.271824] Instruction dump:
[ 61.271826] 409d00b4 3bbcffff 3be3fff8 7bbd1f48 3bc00000 7fa3ea14 48000018 60000000
[ 61.271835] 60000000 60000000 60000000 60420000 <e87f0009> 4bbc72b9 60000000 7fbfe840
[ 61.271847] ---[ end trace fd234da35d8d8a95 ]---
If the vmcore is not collected following warning can be observed in logs shortly before crash
[ 61.271435] ------------[ cut here ]------------
[ 61.271439] WARNING: at kernel/sched/core.c:6277
[ 61.271442] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ipt_REJECT nf_
conntrack_ipv4 nf_defrag_ipv4 xt_comment xt_multiport xt_conntrack nf_conntrack iptable_filter
pseries_rng ses enclosure sg nfsd auth_rpcgss nfs_acl lockd sunrpc grace binfmt_misc ip_tables
ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_common ibmvscsi sc
si_transport_srp ibmveth scsi_tgt ipr libata dm_mirror dm_region_hash dm_log dm_mod
[ 61.271485] CPU: 0 PID: 657 Comm: kworker/0:10 Not tainted 3.10.0-327.4.5.el7.ppc64 #1
[ 61.271490] Workqueue: events .topology_work_fn
[ 61.271493] task: c0000007e6721030 ti: c0000007e7d70000 task.ti: c0000007e7d70000
[ 61.271495] NIP: c00000000012c900 LR: c00000000012c5f8 CTR: 0000000000000000
[ 61.271497] REGS: c0000007e7d735d0 TRAP: 0700 Not tainted (3.10.0-327.4.5.el7.ppc64)
[ 61.271498] MSR: 8000000100029032 <SF,EE,ME,IR,DR,RI> CR: 48848022 XER: 00000000
[ 61.271503] CFAR: c00000000012c5bc SOFTE: 1
GPR00: c00000000012c5f8 c0000007e7d73850 c0000000013822d0 0000000000000010
GPR04: 0000000000000800 0000000000000000 0000000000000000 0000000000000000
GPR08: 0000000000000000 00000000010b0000 c000000000d36840 0000000000000000
GPR12: 0000000000000000 c000000007b30000 c0000007e6b61400 00000000000000b8
GPR16: 0000000000000800 c0000007e538fe00 0000000000000000 c0000007e538fe00
GPR20: 00000000000000f8 c0000000013d09d8 0000000000000001 0000000000000001
GPR24: c00000000129c147 0000000000000017 0000000000000000 c0000007e31f7e00
GPR28: c0000000013d0a90 c0000007e6b61400 0000000000000000 c0000000013d5ebc
[ 61.271532] NIP [c00000000012c900] .build_sched_domains+0x1070/0x10b0
[ 61.271535] LR [c00000000012c5f8] .build_sched_domains+0xd68/0x10b0
[ 61.271537] Call Trace:
[ 61.271540] [c0000007e7d73850] [c00000000012c5f8] .build_sched_domains+0xd68/0x10b0 (unrelia
ble)
[ 61.271543] [c0000007e7d739a0] [c00000000012d154] .partition_sched_domains+0x484/0x510
[ 61.271547] [c0000007e7d73aa0] [c000000000196518] .rebuild_sched_domains+0x68/0xa0
[ 61.271550] [c0000007e7d73b30] [c000000000063b50] .topology_work_fn+0x10/0x30
[ 61.271553] [c0000007e7d73ba0] [c0000000000ff7d0] .process_one_work+0x1d0/0x690
[ 61.271555] [c0000007e7d73c50] [c0000000000ffe10] .worker_thread+0x180/0x500
[ 61.271558] [c0000007e7d73d30] [c00000000010be28] .kthread+0xe8/0xf0
[ 61.271562] [c0000007e7d73e30] [c00000000000a470] .ret_from_kernel_thread+0x58/0x68
[ 61.271564] Instruction dump:
[ 61.271566] 4bfffc1c 3ae00000 4bfffc04 7e10a838 4bfffe40 7fe3fb78 48192a29 60000000
[ 61.271572] 39200000 3ae0fff4 f9210078 4bfffbf0 <0fe00000> 4bfffcbc 89580006 694a0001
Environment
RedHat Enterprise Linux 7.2 on ppc64/ppc64le
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
