Kernel panic at dma_pool_alloc() function due to NULL pointer dereference upon reboot
Issue
- Kernel is getting panic while unloading the mlx5_core module by logging below messages.
crash> log
[..]
[ 151.697868] watchdog: watchdog0: watchdog did not stop!
[ 151.748255] dracut Warning: Killing all remaining processes
[ 151.866029] dracut Warning: Unmounted /oldroot.
[ 151.899148] dracut: Disassembling device-mapper devices
[ 152.104169] mlx5_core 0000:19:00.1: Shutdown was called
[ 152.205916] bond0: link status definitely down for interface eno2, disabling it
[ 152.906587] mlx5_core 0000:19:00.0: Shutdown was called
[ 153.011110] bond0: link status definitely down for interface eno1, disabling it
[ 153.018412] bond0: now running without any active interface!
[ 656.175800] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 656.183656] IP: [<ffffffff8880f13b>] dma_pool_alloc+0x1ab/0x280
[ 656.189596] PGD 0
[ 656.191638] Oops: 0000 [#1] SMP
[..]
[ 656.307084] CPU: 18 PID: 17160 Comm: kworker/18:2 Kdump: loaded Tainted: P OE ------------ 3.10.0-1062.12.1.el7.x86_64 #1
[ 656.319127] Hardware name: Dell Inc. PowerEdge R740/01YM03, BIOS 2.4.8 11/26/2019
[ 656.326592] Workqueue: events linkwatch_event
[ 656.330962] task: ffff9901ca8220e0 ti: ffff9901c87d8000 task.ti: ffff9901c87d8000
[ 656.338424] RIP: 0010:[<ffffffff8880f13b>] [<ffffffff8880f13b>] dma_pool_alloc+0x1ab/0x280
[ 656.346776] RSP: 0018:ffff9901c87db8c8 EFLAGS: 00010046
[ 656.352075] RAX: 0000000000000246 RBX: ffff990214c74280 RCX: 0000000000001000
[ 656.359191] RDX: 0000000000000000 RSI: 0000000000000246 RDI: ffff990214c74210
[ 656.366304] RBP: ffff9901c87db920 R08: 000000000001f080 R09: ffff98ebbfc07c00
[ 656.373421] R10: ffffffffc05640c4 R11: ffffffff887ddc4d R12: 00000000000080d0
[ 656.380535] R13: ffff990214c74210 R14: ffff990214c74200 R15: 0000000000000000
[ 656.387653] FS: 0000000000000000(0000) GS:ffff99021f640000(0000) knlGS:0000000000000000
[ 656.395719] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 656.401450] CR2: 0000000000000000 CR3: 00000017de402000 CR4: 00000000007607e0
[ 656.408565] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 656.415680] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 656.422796] PKRU: 00000000
[ 656.425500] Call Trace:
[ 656.427955] [<ffffffff8882516c>] ? kmem_cache_alloc_trace+0x3c/0x200
[ 656.434406] [<ffffffffc05640e3>] mlx5_alloc_cmd_msg+0xd3/0x2a0 [mlx5_core]
[ 656.441372] [<ffffffffc0566c92>] cmd_exec+0x112/0x860 [mlx5_core]
[ 656.447549] [<ffffffffc056742b>] mlx5_cmd_exec+0x2b/0x50 [mlx5_core]
[ 656.453996] [<ffffffffc0571354>] mlx5_core_access_reg+0xe4/0x130 [mlx5_core]
[ 656.461135] [<ffffffffc0571411>] mlx5_query_port_ptys+0x71/0x90 [mlx5_core]
[ 656.468189] [<ffffffffc0596035>] mlx5e_get_link_ksettings+0x75/0x500 [mlx5_core]
[ 656.475650] [<ffffffff88c56d06>] __ethtool_get_link_ksettings+0xa6/0x210
[ 656.482420] [<ffffffff88d80d17>] ? wait_for_completion+0x37/0x140
[ 656.488585] [<ffffffff887355bf>] ? queue_stop_cpus_work+0x8f/0xa0
[ 656.494752] [<ffffffffc06dd6fd>] bond_update_speed_duplex+0x3d/0x90 [bonding]
[ 656.501962] [<ffffffffc06e4d88>] bond_netdev_event+0x138/0x360 [bonding]
[ 656.508738] [<ffffffff88d88b6f>] notifier_call_chain+0x4f/0x70
[ 656.514642] [<ffffffff886cc3d6>] raw_notifier_call_chain+0x16/0x20
[ 656.520894] [<ffffffff88c4a6ed>] call_netdevice_notifiers_info+0x4d/0x80
[ 656.527661] [<ffffffff88c4ac31>] netdev_state_change+0x41/0x70
[ 656.533566] [<ffffffff88c6a6d0>] linkwatch_do_dev+0x40/0x60
[ 656.539212] [<ffffffff88c6a9e9>] __linkwatch_run_queue+0xf9/0x190
[ 656.545376] [<ffffffff88c6aaa5>] linkwatch_event+0x25/0x30
[ 656.550936] [<ffffffff886be21f>] process_one_work+0x17f/0x440
[ 656.556753] [<ffffffff886bf336>] worker_thread+0x126/0x3c0
[ 656.562313] [<ffffffff886bf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[ 656.568824] [<ffffffff886c61f1>] kthread+0xd1/0xe0
[ 656.573691] [<ffffffff886c6120>] ? insert_kthread_work+0x40/0x40
[ 656.579768] [<ffffffff88d8dd1d>] ret_from_fork_nospec_begin+0x7/0x21
[ 656.586191] [<ffffffff886c6120>] ? insert_kthread_work+0x40/0x40
[ 656.592268] Code: 4c 89 f6 48 89 df 48 89 45 b0 e8 81 20 19 00 8b 53 24 48 8b 45 b0 49 89 d7 4c 03 7b 10 83 43 20 01 48 03 53 18 48 89 c6 4c 89 ef <41> 8b 0f 89 4b 24 48 8b 4d b8 48 89 11 e8 e3 3e 57 00 41 81 e4
[ 656.612690] RIP [<ffffffff8880f13b>] dma_pool_alloc+0x1ab/0x280
[ 656.618706] RSP <ffff9901c87db8c8>
[ 656.622189] CR2: 0000000000000000
Environment
- Red Hat Enterprise Linux 7
- kernel-3.10.0-1062.12.1.el7
- mlx5_core, bonding
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.