NULL pointer dereference in kvdo module code

Solution Verified - Updated -

Issue

System crashing and rebooting due to NULL pointer dereference in kvdo module code

        CPUS: 48
        DATE: Thu Jun 20 23:58:56 EDT 2024
      UPTIME: 17:33:42
LOAD AVERAGE: 10.16, 10.50, 10.62
       TASKS: 2366
    NODENAME: HOSTNAME
     RELEASE: 5.14.0-427.22.1.el9_4.x86_64
     VERSION: #1 SMP PREEMPT_DYNAMIC Mon Jun 10 09:23:36 EDT 2024
     MACHINE: x86_64  (2000 Mhz)
      MEMORY: 255.5 GB
       PANIC: "Oops: 0000 [#1] PREEMPT SMP NOPTI" (check log for details)
         PID: 1645
     COMMAND: "kvdo0:hashQ0"
        TASK: ff329c58cac92380  [THREAD_INFO: ff329c58cac92380]
         CPU: 38
       STATE: TASK_RUNNING (PANIC)

crash> mod -t
NAME  TAINTS
kvdo  O

crash> bt
PID: 1645     TASK: ff329c58cac92380  CPU: 38   COMMAND: "kvdo0:hashQ0"
 #0 [ff6df3744ea33bf0] machine_kexec at ffffffffa9e781e7
 #1 [ff6df3744ea33c48] __crash_kexec at ffffffffa9fef73a
 #2 [ff6df3744ea33d08] crash_kexec at ffffffffa9ff09c8
 #3 [ff6df3744ea33d10] oops_end at ffffffffa9e2f9bb
 #4 [ff6df3744ea33d30] page_fault_oops at ffffffffa9e8a5eb
 #5 [ff6df3744ea33d88] exc_page_fault at ffffffffaaa84ab2
 #6 [ff6df3744ea33db0] asm_exc_page_fault at ffffffffaac00bc2
    [exception RIP: finish_querying+202]
    RIP: ffffffffc144783a  RSP: ff6df3744ea33e60  RFLAGS: 00010246
    RAX: 0000000000000000  RBX: ff6df37460a5f9b8  RCX: 0000000000000017
    RDX: ff6df37460a4a150  RSI: 0000000000000004  RDI: ff6df37460dfc9b0
    RBP: ff6df37460dfc9b0   R8: ff6df37460a4a150   R9: ff6df37460a4a150
    R10: 000000000000002b  R11: ff6df37460dddda0  R12: ff6df37460b114a0
    R13: ff6df37460a01148  R14: ff6df37460a5fa50  R15: ff6df37460a011c0
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #7 [ff6df3744ea33e98] service_work_queue at ffffffffc148db43 [kvdo]
 #8 [ff6df3744ea33f00] work_queue_runner at ffffffffc148dd88 [kvdo]
 #9 [ff6df3744ea33f18] kthread at ffffffffa9f358cd
#10 [ff6df3744ea33f50] ret_from_fork at ffffffffa9e02c69


crash> log -T | tail -n 60
[Thu Jun 20 10:05:51 EDT 2024] perf: interrupt took too long (4999 > 4976), lowering kernel.perf_event_max_sample_rate to 40000
[Thu Jun 20 10:50:19 EDT 2024] kvdo0:reader: Page 35724 invalidated after read
[Thu Jun 20 10:50:19 EDT 2024] kvdo0:reader: Page 35725 invalidated after read
[Thu Jun 20 10:50:25 EDT 2024] kvdo0:reader: Page 35879 invalidated after read
[Thu Jun 20 11:05:27 EDT 2024] kvdo0:reader: Page 69372 invalidated after read
[Thu Jun 20 11:26:02 EDT 2024] perf: interrupt took too long (6268 > 6248), lowering kernel.perf_event_max_sample_rate to 31000
[Thu Jun 20 11:36:46 EDT 2024] kvdo0:reader: Page 48344 invalidated after read
[Thu Jun 20 12:48:37 EDT 2024] kvdo0:reader: Page 48117 invalidated after read
[Thu Jun 20 13:14:57 EDT 2024] kvdo0:reader: Page 25151 invalidated after read
[Thu Jun 20 13:47:29 EDT 2024] kvdo0:reader: Page 5556 invalidated after read
[Thu Jun 20 14:35:21 EDT 2024] perf: interrupt took too long (7848 > 7835), lowering kernel.perf_event_max_sample_rate to 25000
[Thu Jun 20 16:45:36 EDT 2024] kvdo0:reader: Page 17777 invalidated after read
[Thu Jun 20 17:14:34 EDT 2024] kvdo0:reader: Page 63710 invalidated after read
[Thu Jun 20 17:15:29 EDT 2024] kvdo0:reader: Page 65571 invalidated after read
[Thu Jun 20 17:37:54 EDT 2024] kvdo0:reader: Page 29651 invalidated after read
[Thu Jun 20 17:37:54 EDT 2024] kvdo0:reader: Page 29652 invalidated after read
[Thu Jun 20 22:17:04 EDT 2024] kvdo0:reader: Page 12093 invalidated after read
[Thu Jun 20 23:36:43 EDT 2024] kvdo0:reader: Page 53656 invalidated after read
[Thu Jun 20 23:58:55 EDT 2024] BUG: kernel NULL pointer dereference, address: 0000000000000001
[Thu Jun 20 23:58:55 EDT 2024] #PF: supervisor read access in kernel mode
[Thu Jun 20 23:58:55 EDT 2024] #PF: error_code(0x0000) - not-present page
[Thu Jun 20 23:58:55 EDT 2024] PGD 241ab3e067 P4D 2097597067 PUD 2097596067 PMD 0 
[Thu Jun 20 23:58:55 EDT 2024] Oops: 0000 [#1] PREEMPT SMP NOPTI
[Thu Jun 20 23:58:55 EDT 2024] CPU: 38 PID: 1645 Comm: kvdo0:hashQ0 Kdump: loaded Tainted: G           O      -------  ---  5.14.0-427.22.1.el9_4.x86_64 #1
[Thu Jun 20 23:58:55 EDT 2024] Hardware name: Dell Inc. PowerEdge R760xd2/0J3N69, BIOS 2.2.7 05/13/2024
[Thu Jun 20 23:58:55 EDT 2024] RIP: 0010:finish_querying+0xca/0x310 [kvdo]
[Thu Jun 20 23:58:55 EDT 2024] Code: 49 89 44 24 50 0f 85 d4 01 00 00 41 c7 44 24 34 04 00 00 00 48 8b 85 38 02 00 00 48 89 ef be 04 00 00 00 c6 85 90 01 00 00 04 <0f> b6 40 01 48 c7 45 10 40 63 44 c1 88 45 03 48 83 c4 08 5b 5d 41
[Thu Jun 20 23:58:55 EDT 2024] RSP: 0018:ff6df3744ea33e60 EFLAGS: 00010246
[Thu Jun 20 23:58:55 EDT 2024] RAX: 0000000000000000 RBX: ff6df37460a5f9b8 RCX: 0000000000000017
[Thu Jun 20 23:58:55 EDT 2024] RDX: ff6df37460a4a150 RSI: 0000000000000004 RDI: ff6df37460dfc9b0
[Thu Jun 20 23:58:55 EDT 2024] RBP: ff6df37460dfc9b0 R08: ff6df37460a4a150 R09: ff6df37460a4a150
[Thu Jun 20 23:58:55 EDT 2024] R10: 000000000000002b R11: ff6df37460dddda0 R12: ff6df37460b114a0
[Thu Jun 20 23:58:55 EDT 2024] R13: ff6df37460a01148 R14: ff6df37460a5fa50 R15: ff6df37460a011c0
[Thu Jun 20 23:58:55 EDT 2024] FS:  0000000000000000(0000) GS:ff329c77c00c0000(0000) knlGS:0000000000000000
[Thu Jun 20 23:58:55 EDT 2024] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Thu Jun 20 23:58:55 EDT 2024] CR2: 0000000000000001 CR3: 0000002f8c802002 CR4: 0000000000771ee0
[Thu Jun 20 23:58:55 EDT 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Thu Jun 20 23:58:55 EDT 2024] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[Thu Jun 20 23:58:55 EDT 2024] PKRU: 55555554
[Thu Jun 20 23:58:55 EDT 2024] Call Trace:
[Thu Jun 20 23:58:55 EDT 2024]  <TASK>
[Thu Jun 20 23:58:55 EDT 2024]  ? show_trace_log_lvl+0x1c4/0x2df
[Thu Jun 20 23:58:55 EDT 2024]  ? show_trace_log_lvl+0x1c4/0x2df
[Thu Jun 20 23:58:55 EDT 2024]  ? service_work_queue+0xb3/0x290 [kvdo]
[Thu Jun 20 23:58:55 EDT 2024]  ? __die_body.cold+0x8/0xd
[Thu Jun 20 23:58:55 EDT 2024]  ? page_fault_oops+0x134/0x170
[Thu Jun 20 23:58:55 EDT 2024]  ? exc_page_fault+0x62/0x150
[Thu Jun 20 23:58:55 EDT 2024]  ? asm_exc_page_fault+0x22/0x30
[Thu Jun 20 23:58:55 EDT 2024]  ? finish_querying+0xca/0x310 [kvdo]
[Thu Jun 20 23:58:55 EDT 2024]  service_work_queue+0xb3/0x290 [kvdo]
[Thu Jun 20 23:58:55 EDT 2024]  ? __pfx_autoremove_wake_function+0x10/0x10
[Thu Jun 20 23:58:55 EDT 2024]  ? __pfx_work_queue_runner+0x10/0x10 [kvdo]
[Thu Jun 20 23:58:55 EDT 2024]  work_queue_runner+0x58/0x70 [kvdo]
[Thu Jun 20 23:58:55 EDT 2024]  kthread+0xdd/0x100
[Thu Jun 20 23:58:55 EDT 2024]  ? __pfx_kthread+0x10/0x10
[Thu Jun 20 23:58:55 EDT 2024]  ret_from_fork+0x29/0x50
[Thu Jun 20 23:58:55 EDT 2024]  </TASK>
[Thu Jun 20 23:58:55 EDT 2024] Modules linked in: bonding tls nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 rfkill ip_set nf_tables nfnetlink vfat fat dm_multipath intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_ifs i10nm_edac nfit libnvdimm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel iTCO_wdt iTCO_vendor_support kvm irqbypass rapl kvdo(O) dm_bufio ipmi_ssif irdma intel_cstate pmt_telemetry intel_sdsi i40e pmt_class mgag200 i2c_algo_bit drm_shmem_helper dcdbas dell_smbios drm_kms_helper ib_uverbs ses enclosure dell_wmi_descriptor wmi_bmof intel_uncore pcspkr isst_if_mbox_pci syscopyarea ib_core acpi_ipmi mei_me isst_if_mmio sysfillrect isst_if_common intel_vsec i2c_i801 sysimgblt mei ipmi_si fb_sys_fops i2c_ismt i2c_smbus ipmi_devintf ipmi_msghandler acpi_power_meter joydev drm fuse xfs libcrc32c sd_mod sg ahci libahci
[Thu Jun 20 23:58:55 EDT 2024]  crct10dif_pclmul ice iaa_crypto crc32_pclmul mpt3sas libata nvme crc32c_intel nvme_core megaraid_sas ghash_clmulni_intel idxd tg3 raid_class scsi_transport_sas gnss idxd_bus nvme_common wmi t10_pi pinctrl_emmitsburg dm_mirror dm_region_hash dm_log dm_mod
[Thu Jun 20 23:58:55 EDT 2024] CR2: 0000000000000001
crash> 

Environment

  • Red Hat Enterprise Linux (RHEL) 9
    • kvdo

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content