Kernel panic with "general protection fault, probably for non-canonical address" due to involvement of third party module drbd

Solution Unverified - Updated -

Issue

  • Server panic due to "Kernel panic - not syncing: Fatal exception" / due to a general protection fault
  • Kernel panic with following call traces :
[ 1713.168705] drbd fnth_p2g_prod l08p10k02560001: peer( Secondary -> Primary )
[ 1784.338230] general protection fault, probably for non-canonical address 0x9a000000e958019c: 0000 [#1] SMP NOPTI
[ 1784.338231] CPU: 2 PID: 36240 Comm: grep Kdump: loaded Tainted: P           OE    --------- -  - 4.18.0-477.21.1.el8_8.x86_64 #1
[ 1784.338232] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008  12/07/2018
[ 1784.338232] RIP: 0010:string_nocheck+0x12/0x70
[ 1784.338232] Code: ef e8 e2 a7 00 00 4c 01 e3 e9 7a ff ff ff 66 2e 0f 1f 84 00 00 00 00 00 49 89 f2 48 89 ce 48 89 f8 48 c1 fe 30 66 85 f6 74 4f <44> 0f b6 0a 45 84 c9 74 46 83 ee 01 41 b8 01 00 00 00 48 8d 7c 37
[ 1784.338233] RSP: 0018:ffffa41883273b00 EFLAGS: 00010086
[ 1784.338234] RAX: ffffa41883273c20 RBX: ffffa41883273c10 RCX: ffff0a00ffffff04
[ 1784.338234] RDX: 9a000000e958019c RSI: ffffffffffffffff RDI: ffffa41883273c20
[ 1784.338235] RBP: 9a000000e958019c R08: 0000000000000011 R09: 0000000000000000
[ 1784.338235] R10: ffffa41883273c10 R11: ffff91dd8706a1d4 R12: ffff0a00ffffff04
[ 1784.338235] R13: ffffffffc0c1b4b6 R14: 0000000000000008 R15: ffffffffc0c1b4b6
[ 1784.338236] FS:  00007f25a7895740(0000) GS:ffff91e01fd00000(0000) knlGS:0000000000000000
[ 1784.338236] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1784.338236] CR2: 00005606c1d91bf8 CR3: 00000001378aa000 CR4: 0000000000350ee0
[ 1784.338237] Call Trace:
[ 1784.338237]  string+0x44/0x60
[ 1784.338237]  vsnprintf+0x340/0x520
[ 1784.338237]  vprintk_store+0xad/0x450
[ 1784.338237]  ? drbd_single_open+0xf9/0x130 [drbd]
[ 1784.338238]  vprintk_emit+0xff/0x250
[ 1784.338238]  printk+0x58/0x73
[ 1784.338238]  drbd_destroy_connection+0x10e/0x120 [drbd]
[ 1784.338238]  resource_in_flight_summary_show+0x73d/0x820 [drbd]
[ 1784.338239]  seq_read+0x163/0x420
[ 1784.338239]  full_proxy_read+0x53/0x80
[ 1784.338239]  vfs_read+0x91/0x150
[ 1784.338239]  ksys_read+0x4f/0xb0
[ 1784.338240]  unload_network_ops_symbols+0x85a3/0xb560 [falcon_lsm_pinned_16004]
[ 1784.338240]  ? do_syscall_64+0x5b/0x1b0
[ 1784.338240]  ? entry_SYSCALL_64_after_hwframe+0x61/0xc6
[ 1784.338241] Modules linked in: mptcp_diag xsk_diag raw_diag unix_diag af_packet_diag netlink_diag udp_diag tcp_diag inet_diag nft_counter xt_owner xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat drbd_transport_tcp(OE) drbd(OE) falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_16004(E) ext4 mbcache jbd2 nf_tables nfnetlink intel_rapl_msr intel_rapl_common amd_energy crct10dif_pclmul crc32_pclmul vfat fat ghash_clmulni_intel hv_balloon hv_utils hyperv_fb i2c_piix4 pcspkr joydev binfmt_misc auth_rpcgss sunrpc xfs libcrc32c ata_generic sd_mod t10_pi sg hv_netvsc hyperv_keyboard hid_hyperv hv_storvsc scsi_transport_fc ata_piix libata hv_vmbus crc32c_intel serio_raw dm_mirror dm_region_hash dm_log dm_mod
[ 1784.338254] ---[ end trace df8b01c41207b282 ]---
[ 1784.338255] RIP: 0010:string_nocheck+0x12/0x70
[ 1784.338255] Code: ef e8 e2 a7 00 00 4c 01 e3 e9 7a ff ff ff 66 2e 0f 1f 84 00 00 00 00 00 49 89 f2 48 89 ce 48 89 f8 48 c1 fe 30 66 85 f6 74 4f <44> 0f b6 0a 45 84 c9 74 46 83 ee 01 41 b8 01 00 00 00 48 8d 7c 37
[ 1784.338256] RSP: 0018:ffffa41883273b00 EFLAGS: 00010086
[ 1784.338256] RAX: ffffa41883273c20 RBX: ffffa41883273c10 RCX: ffff0a00ffffff04
[ 1784.338257] RDX: 9a000000e958019c RSI: ffffffffffffffff RDI: ffffa41883273c20
[ 1784.338257] RBP: 9a000000e958019c R08: 0000000000000011 R09: 0000000000000000
[ 1784.338257] R10: ffffa41883273c10 R11: ffff91dd8706a1d4 R12: ffff0a00ffffff04
[ 1784.338258] R13: ffffffffc0c1b4b6 R14: 0000000000000008 R15: ffffffffc0c1b4b6
[ 1784.338258] FS:  00007f25a7895740(0000) GS:ffff91e01fd00000(0000) knlGS:0000000000000000
[ 1784.338258] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1784.338259] CR2: 00005606c1d91bf8 CR3: 00000001378aa000 CR4: 0000000000350ee0
[ 1784.338259] Kernel panic - not syncing: Fatal exception
[ 1784.338259] Kernel Offset: 0x2b600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

Environment

  • Red Hat Enterprise Linux 8
  • Third party module [drbd]

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content