System crashed with hard lockup due to lots of ftrace

Issue

• System crashed with the below log

...
[35027608.914636] NMI watchdog: Watchdog detected hard LOCKUP on cpu 50
[35027608.914722] Modules linked in:
[35027608.915981]  mfe_fileaccess_100700130(OE) mfe_aac_100700130(OE) ip6table_filter ip6_tables iptable_filter udp_diag tcp_diag inet_diag bonding sunrpc emcpdm(POE) emcpgpx(POE) emcpmpx(POE) emcp(POE) iTCO_wdt iTCO_vendor_support sb_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev ses enclosure lpc_ich scsi_transport_sas ipmi_ssif wmi acpi_power_meter ipmi_si ipmi_devintf ipmi_msghandler acpi_pad sg binfmt_misc ip_tables xfs libcrc32c sd_mod lpfc mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ixgbe drm nvmet_fc igb nvmet crc_t10dif crct10dif_generic crc32c_intel crct10dif_pclmul nvme_fc nvme_fabrics mdio ptp nvme_core megaraid_sas pps_core dca i2c_algo_bit
[35027608.916051]  scsi_transport_fc scsi_tgt drm_panel_orientation_quirks crct10dif_common dm_mirror dm_region_hash dm_log dm_mod [last unloaded: mfe_fileaccess_100700130]
[35027608.916062] CPU: 50 PID: 3160 Comm: xagt Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1127.el7.x86_64 #1
[35027608.916064] Hardware name: Cisco Systems Inc UCSC-C460-M4/UCSC-C460-M4, BIOS C460M4.3.0.3b.0.040420170254 04/04/2017
[35027608.916066] Call Trace:
[35027608.916069]  <NMI>  [<ffffffff8fd7ff85>] dump_stack+0x19/0x1b
[35027608.916088]  [<ffffffff8f74f765>] watchdog_overflow_callback+0x135/0x140
[35027608.916094]  [<ffffffff8f7a9037>] __perf_event_overflow+0x57/0x100
[35027608.916101]  [<ffffffff8f7b2834>] perf_event_overflow+0x14/0x20
[35027608.916105]  [<ffffffff8f60a9a0>] handle_pmi_common+0x1a0/0x250
[35027608.916120]  [<ffffffff8f987458>] ? ioremap_page_range+0x2e8/0x480
[35027608.916127]  [<ffffffff8f805284>] ? vunmap_page_range+0x234/0x470
[35027608.916133]  [<ffffffff8fa4e766>] ? ghes_copy_tofrom_phys+0x116/0x210
[35027608.916136]  [<ffffffff8f60ac7f>] intel_pmu_handle_irq+0xcf/0x1d0
[35027608.916145]  [<ffffffff8fd89031>] perf_event_nmi_handler+0x31/0x50
[35027608.916148]  [<ffffffff8fd8a93c>] nmi_handle.isra.0+0x8c/0x150
[35027608.916150]  [<ffffffff8fd8ab5d>] do_nmi+0x15d/0x460
[35027608.916153]  [<ffffffff8fd89d9c>] end_repeat_nmi+0x1e/0x81
[35027608.916156]  [<ffffffff8fd90635>] ? kprobes_inc_nmissed_count+0x55/0x80
[35027608.916159]  [<ffffffff8fd90635>] ? kprobes_inc_nmissed_count+0x55/0x80
[35027608.916161]  [<ffffffff8fd90635>] ? kprobes_inc_nmissed_count+0x55/0x80
[35027608.916162]  <EOE>  [<ffffffff8fd8becd>] kprobe_fault_handler+0x4d/0x130
[35027608.916166]  [<ffffffff8fd8d860>] __do_page_fault+0x420/0x500
[35027608.916168]  [<ffffffff8fd8d975>] do_page_fault+0x35/0x90
[35027608.916171]  [<ffffffff8fd89778>] page_fault+0x28/0x30
[35027608.916181]  [<ffffffff8f993cc9>] ? copy_user_enhanced_fast_string+0x9/0x20
[35027608.916184]  [<ffffffff8fd91118>] ? fetch_memory_u64+0x58/0xd0
[35027608.916186]  [<ffffffff8fd91f57>] ? fetch_deref_u64+0x77/0x80
[35027608.916188]  [<ffffffff8fd91f57>] fetch_deref_u64+0x77/0x80
[35027608.916191]  [<ffffffff8fd91951>] kprobe_dispatcher+0x1d1/0x5e0
[35027608.916194]  [<ffffffff8fd8dd64>] aggr_pre_handler+0x54/0xb0
[35027608.916204]  [<ffffffff8f855ea1>] ? setup_new_exec+0x1/0x1a0
[35027608.916206]  [<ffffffff8fd8ceda>] kprobe_ftrace_handler+0xba/0x120
[35027608.916209]  [<ffffffff8f855ea5>] ? setup_new_exec+0x5/0x1a0
[35027608.916211]  [<ffffffff8f855ea0>] ? set_dumpable+0x70/0x70
[35027608.916217]  [<ffffffff8f8b38a0>] ? load_elf_binary+0x390/0xdb0
[35027608.916221]  [<ffffffff8f762634>] ftrace_ops_list_func+0xf4/0x120
[35027608.916226]  [<ffffffff8fd96b14>] ftrace_regs_call+0x5/0x81
[35027608.916229]  [<ffffffff8f855ea1>] ? setup_new_exec+0x1/0x1a0
[35027608.916231]  [<ffffffff8f855ea5>] ? setup_new_exec+0x5/0x1a0
[35027608.916234]  [<ffffffff8f8b38a0>] ? load_elf_binary+0x390/0xdb0
[35027608.916239]  [<ffffffffc02be064>] ? load_misc_binary+0x64/0x460 [binfmt_misc]
[35027608.916247]  [<ffffffffc05a63a7>] ? mfe_aac_process_pre_events+0xb7/0x1a0 [mfe_aac_100700130]
[35027608.916250]  [<ffffffff8f85449a>] search_binary_handler+0x9a/0x1c0
[35027608.916253]  [<ffffffff8f855ba6>] do_execve_common.isra.24+0x616/0x880
[35027608.916255]  [<ffffffff8fd8b459>] kretprobe_trampoline_holder+0x9/0x9
[35027608.916262]  [<ffffffff8fd93478>] stub_execve+0x48/0x80
...