System with 3rd party module crashed at RIP __kmalloc in sunrpc codepath.

Solution Unverified - Updated -

Issue

  • A system crashed at _kmalloc in sunrpc codepath.
  • This system installs third party module: sisap and sisevt.
  • Call Trace is like below.
[27426691.011118] stack segment: 0000 [#1] SMP PTI
[27426691.065334] CPU: 11 PID: 128321 Comm: kworker/u129:3 Kdump: loaded Tainted: P           OE    --------- -  - 4.18.0-193.14.3.el8_2.x86_64 #1
[27426691.219362] Hardware name: XXXXX
[27426691.363016] Workqueue: rpciod rpc_async_schedule [sunrpc]                                                                                              
[27426691.430729] RIP: 0010:__kmalloc+0xa0/0x200                   
[27426691.482861] Code: 01 00 00 4d 8b 07 65 49 8b 50 08 65 4c 03 05 5f a6 97 77 49 8b 28 48 85 ed 0f 84 21 01 00 00 41 8b 47 20 4d 8b 07 48 8d 4a 01 <48> 8b
 5c 05 00 48 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 c5 41 8b
[27426691.710732] RSP: 0018:ffffb7bf605bfbf8 EFLAGS: 00010286
[27426691.776389] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000020c0e0ee
[27426691.864898] RDX: 0000000020c0e0ed RSI: 0000000000600000 RDI: ffff997407c07c00
[27426691.953409] RBP: ff997b6f0e4078ff R08: 000000000002e040 R09: 0000000000000001
[27426692.041919] R10: ffffb7bf605bfd1c R11: ffff9992ff6e8bc0 R12: 0000000000600000
[27426692.130429] R13: 0000000000000004 R14: ffff997407c07c00 R15: ffff997407c07c00
[27426692.218943] FS:  0000000000000000(0000) GS:ffff9992ff6c0000(0000) knlGS:0000000000000000
[27426692.318897] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[27426692.390768] CR2: 00007fe27e07e000 CR3: 00000034eee0a002 CR4: 00000000003626e0
[27426692.479278] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[27426692.567790] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[27426692.656303] Call Trace:
[27426692.688683]  ? xdr_stream_decode_string_dup+0x75/0x100 [sunrpc]
[27426692.762674]  xdr_stream_decode_string_dup+0x75/0x100 [sunrpc]
[27426692.834604]  decode_getfattr_attrs+0x7a5/0x1090 [nfsv4]
[27426692.900239]  ? select_task_rq_fair+0xad8/0xdb0
[27426692.956557]  decode_getfattr_generic.constprop.117+0x148/0x240 [nfsv4]
[27426693.037797]  ? rpc_exit+0x30/0x30 [sunrpc]
[27426693.089944]  ? call_bind_status+0x240/0x240 [sunrpc]
[27426693.152491]  nfs4_xdr_dec_open_noattr+0xc7/0x100 [nfsv4]
[27426693.219173]  ? rpc_decode_header+0xec/0x560 [sunrpc]
[27426693.281688]  call_decode+0x168/0x190 [sunrpc]
[27426693.336957]  __rpc_execute+0x85/0x340 [sunrpc]
[27426693.393234]  ? finish_task_switch+0xd7/0x2b0
[27426693.447454]  rpc_async_schedule+0x29/0x40 [sunrpc]
[27426693.507892]  process_one_work+0x1a7/0x3b0
[27426693.558982]  worker_thread+0x30/0x390
[27426693.605908]  ? create_worker+0x1a0/0x1a0
[27426693.655940]  kthread+0x112/0x130
[27426693.697674]  ? kthread_flush_work_fn+0x10/0x10
[27426693.753948]  ret_from_fork+0x35/0x40
[27426693.799836] Modules linked in: sisap(POE) sisevt(PE) nf_conntrack_netlink vxlan ip6_udp_tunnel udp_tunnel xt_CT xt_mac xt_set xt_physdev ip_set_hash_ne
t ip_set vhost_net vhost tap tun veth xsk_diag raw_diag unix_diag af_packet_diag netlink_diag tcp_diag udp_diag inet_diag rpcsec_gss_krb5 auth_rpcgss overlay
 nfsv4 dns_resolver nfs lockd grace fscache sch_ingress bonding rpcrdma ib_isert iscsi_target_mod ib_iser ib_srpt target_core_mod ib_srp scsi_transport_srp i
b_ipoib rdma_ucm ib_umad iw_cxgb4 ib_uverbs rdma_cm iw_cm ib_cm ib_core nfnetlink_cttimeout ipmi_ssif intel_rapl_msr mxm_wmi iTCO_wdt iTCO_vendor_support nf_
log_ipv6 nf_log_ipv4 nf_log_common intel_rapl_common nft_limit nls_utf8 isofs sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nft_counter kvm_intel kv
m irqbypass intel_cstate intel_uncore mei_me xt_LOG intel_rapl_perf joydev pcspkr mei i2c_i801 lpc_ich xt_limit xt_multiport xt_comment xt_state ipmi_si xt_c
onntrack nft_compat ipmi_devintf ipmi_msghandler wmi
[27426693.799873]  acpi_power_meter
...
[27426694.835845]  nf_tables nfnetlink openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_conncount nf_
nat nf_conntrack br_netfilter bridge stp llc ip_tables xfs libcrc32c dm_multipath sd_mod sg mgag200 drm_vram_helper i2c_algo_bit drm_kms_helper syscopyarea s
ysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crc32_pclmul ahci crc32c_intel drm libahci ixgbe ghash_clmulni_intel libata megaraid_sas be2net dca mdi
o sunrpc dm_mirror dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio cxgb4i cxgb4 libcxgbi libcxgb qla4xxx iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscs
i scsi_transport_iscsi

Environment

  • Red Hat Enterprise Linux 8.2
  • kernel-4.18.0-193.14.3.el8_2.x86_64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content