RHEL7: kernel panic at kernel/trace/ftrace.c:1839

Solution In Progress - Updated -

Issue

  • Kernel panic with below logs:
[401273.017886] ------------[ cut here ]------------ [401273.017894] WARNING: CPU: 47 PID: 61359 at kernel/trace/ftrace.c:1839 ftrace_bug+0x223/0x270 [401273.017895] Modules linked in: nfnetlink_queue nfnetlink_log bluetooth rfkill mpt3sas mptctl mptbase udp_diag tcp_diag inet_diag dm_round_robin ib_iser(OE) iscsi_tcp libiscsi_tcp libiscsi dell_rbu nvme_rdma(OE) nvme_fabrics(OE) nvme(OE) nvme_core(OE) scsi_transport_iscsi bonding(OE) rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ib_ipoib(OE) ib_cm(OE) ip6t_REJECT nf_reject_ipv6 xt_conntrack ib_umad(OE) mlx5_fpga_tools(OE) ip_set nfnetlink mlx5_ib(OE) ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 mlx5_core(OE) nf_nat_ipv6 mlxfw(OE) ip6table_mangle vfio_mdev(OE) vfio_iommu_type1 vfio ip6table_security mdev(OE) ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack mlx4_en(OE) iptable_mangle [401273.017921]  iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc iTCO_wdt iTCO_vendor_support dell_smbios dcdbas sparse_keymap skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg pcspkr lpc_ich mei_me i2c_i801 mei joydev shpchp wmi ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad dm_multipath knem(OE) binfmt_misc ip_tables xfs libcrc32c mlx4_ib(OE) ib_uverbs(OE) ib_core(OE) crc32c_intel mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ixgbe mlx4_core(OE) igb drm devlink mdio mlx_compat(OE) ptp i2c_algo_bit pps_core dca i2c_core sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common [401273.017946]  ahci libahci libata mpt2sas raid_class scsi_transport_sas megaraid_sas dm_mirror dm_region_hash dm_log dm_mod
[401273.017951] CPU: 47 PID: 61359 Comm: cloudwalker-eng Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.11.6.el7.x86_64 #1
[401273.017953] Hardware name: Hangzhou WOQU Technology WOQUTECH R940/0GCTJ1, BIOS 2.9.4 11/06/2020 [401273.017954] Call Trace:
[401273.017958]  [<ffffffff8ad135d4>] dump_stack+0x19/0x1b [401273.017962]  [<ffffffff8a6945d8>] __warn+0xd8/0x100 [401273.017967]  [<ffffffff8ad29100>] ? __fentry__+0x10/0x10 [401273.017968]  [<ffffffff8a69471d>] warn_slowpath_null+0x1d/0x20 [401273.017970]  [<ffffffff8a7574c3>] ftrace_bug+0x223/0x270 [401273.017973]  [<ffffffff8a661e11>] ftrace_replace_code+0x2b1/0x420 [401273.017975]  [<ffffffff8a757baa>] ftrace_modify_all_code+0x6a/0xb0 [401273.017977]  [<ffffffff8a661f90>] arch_ftrace_update_code+0x10/0x20 [401273.017978]  [<ffffffff8a757c87>] ftrace_run_update_code+0x17/0x70 [401273.017980]  [<ffffffff8a757d19>] ftrace_startup_enable+0x39/0x50 [401273.017981]  [<ffffffff8a7581d4>] ftrace_startup+0x114/0x2e0 [401273.017983]  [<ffffffff8a7583e3>] register_ftrace_function+0x43/0x60
[401273.017985]  [<ffffffff8ad22187>] arm_kprobe+0x87/0xe0 [401273.017987]  [<ffffffff8ad2224b>] enable_kprobe+0x6b/0x80 [401273.017990]  [<ffffffff8a77a3c9>] enable_trace_probe+0xf9/0x150 [401273.017992]  [<ffffffff8ad24708>] kprobe_register+0x48/0x60 [401273.017994]  [<ffffffff8a771d85>] __ftrace_event_enable_disable+0x65/0x1a0
[401273.017995]  [<ffffffff8a77252d>] event_enable_write+0x9d/0xe0 [401273.017998]  [<ffffffff8a81f180>] vfs_write+0xc0/0x1f0 [401273.017999]  [<ffffffff8a81ffaf>] SyS_write+0x7f/0xf0 [401273.018001]  [<ffffffff8ad2579b>] system_call_fastpath+0x22/0x27 [401273.018002] ---[ end trace 522a1a9c731258d7 ]--- [401273.018003] ftrace faulted on modifying [<31ffffffc0772e70>] 0x31ffffffc0772e70 [401273.018007] Failed on adding breakpoints (39185):
[401273.020532] ------------[ cut here ]------------

Environment

  • Red Hat Enterprise Linux 7.5
  • CentOS package installed
    • kernel-debuginfo-3.10.0-862.11.6.el7.x86_64 kernel-debuginfo-common-x86_64-3.10.0-862.11.6.el7.x86_64
  • ftrace enabled by 3rd party application cloudwalker-engine

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content