RHEL7.5: smb2.1 client on 4.14.0-49.6.1.el7a.ppc64le sees cifsd kernel panic due to NULL sock.sk_wq value and sock.sk_flags contains SOCK_DEAD
Issue
- An smb2.1 client on 4.14.0-49.6.1.el7a.ppc64le repeatedly crashes in the cifsd recvmsg code path due to accessing an orphaned sock structure
[202312.365627] Status code returned 0xc000000d STATUS_INVALID_PARAMETER
[202312.365676] CIFS VFS: disabling echoes and oplocks
[202312.365703] CIFS VFS: Send error in read = -22
[202316.242199] CIFS VFS: Free previous auth_key.response = c000003fdc51ea00
[202316.243656] Status code returned 0xc00000dc STATUS_INVALID_SERVER_STATE
[202316.243710] CIFS VFS: Send error in read = -5
[203412.081965] Status code returned 0xc000000d STATUS_INVALID_PARAMETER
[203412.082026] CIFS VFS: Send error in read = -22
[203412.296701] Status code returned 0xc000000d STATUS_INVALID_PARAMETER
[203412.296744] CIFS VFS: disabling echoes and oplocks
[203412.296770] CIFS VFS: Send error in read = -22
[203414.797934] CIFS VFS: Free previous auth_key.response = c0002039710bba00
[203414.799723] Status code returned 0xc00000dc STATUS_INVALID_SERVER_STATE
[203414.799769] CIFS VFS: Send error in read = -5
[267244.949246] CIFS VFS: disabling echoes and oplocks
[267244.951363] Unable to handle kernel paging request for data at address 0x00000000
[267244.951411] Faulting instruction address: 0xc000000000c68674
[267244.951449] Oops: Kernel access of bad area, sig: 11 [#1]
[267244.951477] LE SMP NR_CPUS=2048 NUMA PowerNV
[267244.951509] Modules linked in: nvidia_uvm(POE) arc4 md4 nls_utf8 cifs ccm sctp_diag sctp libcrc32c tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag nfsv3 nfs_acl rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_uverbs(OE) ib_umad(OE) mlx5_fpga_tools(OE) mlx5_ib(OE) mlx5_core(OE) mlxfw(OE) cxl mlx4_en(OE) mlx4_ib(OE) ib_core(OE) mlx4_core(OE) mlx_compat(OE) devlink i2c_dev dm_mirror dm_region_hash dm_log dm_mod at24 ofpart sg powernv_flash shpchp mtd opal_prd ipmi_powernv uio_pdrv_genirq uio ibmpowernv i2c_opal auth_rpcgss knem(OE) sunrpc binfmt_misc tcp_htcp ip_tables ext4 mbcache jbd2 raid1 sd_mod nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) ast i2c_algo_bit drm_kms_helper ttm syscopyarea sysfillrect
[267244.951918] sysimgblt fb_sys_fops drm ahci libahci libata tg3 ipmi_devintf i2c_core ipmi_msghandler ptp pps_core [last unloaded: devlink]
[267244.951991] CPU: 112 PID: 18188 Comm: cifsd Kdump: loaded Tainted: P OE ------------ 4.14.0-49.6.1.el7a.ppc64le #1
[267244.952053] task: c000003f7eb5b500 task.stack: c000003fd6a04000
[267244.952088] NIP: c000000000c68674 LR: c0000000001b5178 CTR: c0000000001920d0
[267244.952130] REGS: c000003fd6a07600 TRAP: 0300 Tainted: P OE ------------ (4.14.0-49.6.1.el7a.ppc64le)
[267244.952184] MSR: 9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE> CR: 28182b42 XER: 20040000
[267244.952230] CFAR: c0000000001b5174 DAR: 0000000000000000 DSISR: 40000000 SOFTE: 0
GPR00: c0000000001b5178 c000003fd6a07880 c0000000014c8200 0000000000000000
GPR04: c000003fd6a07948 0000000000000001 0000000028188b42 0000000002000000
GPR08: 0000000000000000 0000000000000000 0000000080000070 0000000000000001
GPR12: 0000000000002000 c000000007a5d000 0000000000000000 c000203944461800
GPR16: 0000000000000000 0000000000000000 c0002039444618c8 c000203944461d1c
GPR20: 0000000000000000 0000000000000000 c000000001420a08 0000000000000000
GPR24: 0000000000000000 0000000000000000 c000003fd6a07a30 c0002039444618c8
GPR28: 0000000000000000 c000203944461890 0000000000000000 0000000000000001
[267244.952603] NIP [c000000000c68674] _raw_spin_lock_irqsave+0x44/0x100
[267244.952641] LR [c0000000001b5178] remove_wait_queue+0x38/0xc0
[267244.952675] Call Trace:
[267244.952691] [c000003fd6a07880] [c000003fd6a07a30] 0xc000003fd6a07a30 (unreliable)
[267244.952734] [c000003fd6a078c0] [c0000000001b5178] remove_wait_queue+0x38/0xc0
[267244.952789] [c000003fd6a07900] [c000000000a34134] sk_wait_data+0x1f4/0x310
[267244.952873] [c000003fd6a079a0] [c000000000b04b7c] tcp_recvmsg+0x67c/0xa70
[267244.952955] [c000003fd6a07b00] [c000000000b4a840] inet_recvmsg+0x80/0x120
[267244.953006] [c000003fd6a07b60] [c000000000a2561c] sock_recvmsg+0x7c/0xa0
[267244.953055] [c000003fd6a07ba0] [c00800000da09b58] cifs_readv_from_socket+0x78/0x2e0 [cifs]
[267244.953105] [c000003fd6a07c30] [c00800000da09e24] cifs_read_from_socket+0x64/0x80 [cifs]
[267244.953154] [c000003fd6a07ce0] [c00800000da0a42c] cifs_demultiplex_thread+0x17c/0xd40 [cifs]
[267244.953203] [c000003fd6a07dc0] [c000000000171ce8] kthread+0x168/0x1b0
[267244.953240] [c000003fd6a07e30] [c00000000000b628] ret_from_kernel_thread+0x5c/0xb4
[267244.953282] Instruction dump:
[267244.953304] fbe1fff8 f8010010 f821ffc1 7c7e1b78 60000000 60000000 39200000 8bed028a
[267244.953347] 992d028a 39400000 994d028c 814d0008 <7d20f029> 2c090000 40c20010 7d40f12d
[267244.953393] ---[ end trace da6e0ba5208827e0 ]---
[267245.962125]
[267245.962197] Sending IPI to other CPUs
[267245.981920] IPI complete
[267248.043801] kexec: Starting switchover sequence.
Environment
- Red Hat Enterprise Linux 7.5 (cifs client)
- seen on kernel-alt-4.14.0-49.6.1.el7a.ppc64le
- smb2.1 cifs client
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
