System was crashed after showing 'kernel: BUG: scheduling while atomic...' earlier

Solution In Progress - Updated -

Issue

  • System was crashed after below type of messages multiple times.
[  102.478951] BUG: scheduling while atomic: sidekiq/1466/0x10000100
[  102.478954] Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink intercepter(POE) km_services(POE) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc vmw_vmci parport i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c
[  102.478994]  sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect crc_t10dif sysimgblt crct10dif_generic fb_sys_fops ttm crct10dif_pclmul crct10dif_common nfit drm ahci libnvdimm libahci ata_piix crc32c_intel serio_raw libata vmw_pvscsi vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
[  102.479019] CPU: 2 PID: 1466 Comm: sidekiq Kdump: loaded Tainted: P           OE  ------------   3.10.0-1160.11.1.el7.x86_64 #1
[  102.479020] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020   
[  102.479021] Call Trace:
[  102.479023]  <IRQ>  [<ffffffff89780faa>] dump_stack+0x19/0x1b
[  102.479034]  [<ffffffff8977ab4f>] __schedule_bug+0x64/0x72
[  102.479037]  [<ffffffff89786d02>] __schedule+0x7c2/0x860
[  102.479042]  [<ffffffff890d7756>] __cond_resched+0x26/0x30
[  102.479044]  [<ffffffff8978706a>] _cond_resched+0x3a/0x50
[  102.479049]  [<ffffffff89228815>] kmem_cache_alloc+0x35/0x1f0
[  102.479055]  [<ffffffff8925fc0b>] ? getname_kernel+0x2b/0x120
[  102.479057]  [<ffffffff8925fc0b>] getname_kernel+0x2b/0x120
[  102.479058]  [<ffffffff89260ce7>] kern_path+0x37/0xb0
[  102.479106]  [<ffffffffc071a895>] ? alloc_for_index_all_cpus+0x67/0xca [intercepter]
[  102.479117]  [<ffffffffc071a954>] ? l_malloc+0x5c/0x17f [intercepter]
[  102.479131]  [<ffffffffc0724a52>] kern_path+0x32/0x34 [intercepter]
[  102.479154]  [<ffffffffc08321db>] getFileStructDataFromKernelPathnameInternal+0xa7/0x184 [intercepter]
[  102.479164]  [<ffffffffc0718d49>] ? ag_StrFree+0x49/0x4b [intercepter]
[  102.479180]  [<ffffffffc0834dd5>] linux_x64_createBackingFileEntity+0x2d/0x103 [intercepter]
[  102.479196]  [<ffffffffc08351fd>] linux_x64_createProcessEntity+0x352/0x53b [intercepter]
[  102.479206]  [<ffffffffc0718d49>] ? ag_StrFree+0x49/0x4b [intercepter]
[  102.479222]  [<ffffffffc07fbe96>] ? sd_AttributeSubtreeCreateDo+0x170/0x197 [intercepter]
[  102.479238]  [<ffffffffc0835495>] linux_x64_createThreadEntity+0xaf/0x333 [intercepter]
[  102.479253]  [<ffffffffc07fbeb7>] ? sd_AttributeSubtreeCreateDo+0x191/0x197 [intercepter]
[  102.479263]  [<ffffffffc071ab68>] ? __Malloc+0x67/0x6d [intercepter]
[  102.479279]  [<ffffffffc083578e>] linux_x64_createCurrentThreadEntity+0x75/0x11e [intercepter]
[  102.479294]  [<ffffffffc0835a9e>] linux_x64_createCurrentThreadScopeEventLnx+0x56/0x103 [intercepter]
[  102.479308]  [<ffffffffc083aae9>] linux_x64_createTcpEventLnx+0x5a/0x194 [intercepter]
[  102.479322]  [<ffffffffc083ba86>] linux_x64_hook_tcp_finish_connect+0x2b6/0x433 [intercepter]
[  102.479338]  [<ffffffffc07fa0a8>] ? sd_AttributesGetCounters+0x66/0x75 [intercepter]
[  102.479353]  [<ffffffffc080dd32>] ? MetricsGetAttributeMetrics+0x19/0x72 [intercepter]
[  102.479367]  [<ffffffffc080a056>] HookExitExecute+0x9c/0xe7 [intercepter]
[  102.479382]  [<ffffffffc080a109>] HookExitIterator+0x68/0x93 [intercepter]
[  102.479393]  [<ffffffffc071d64b>] HashTableLookup+0x249/0x340 [intercepter]
[  102.479407]  [<ffffffffc080a0a1>] ? HookExitExecute+0xe7/0xe7 [intercepter]
[  102.479418]  [<ffffffffc071d87b>] ag_HashTableIterateByKeyNoCopy+0x3d/0x3f [intercepter]
[  102.479433]  [<ffffffffc080a0a1>] ? HookExitExecute+0xe7/0xe7 [intercepter]
[  102.479444]  [<ffffffffc071ee49>] ag_CacheHandleIterateByKey+0x67/0x82 [intercepter]
[  102.479458]  [<ffffffffc080a0a1>] ? HookExitExecute+0xe7/0xe7 [intercepter]
[  102.479473]  [<ffffffffc080a188>] sd_HookExitExecute+0x54/0x6a [intercepter]
[  102.479486]  [<ffffffffc072b4a7>] HookExitHandler+0x63/0x76 [intercepter]
[  102.479499]  [<ffffffffc072e3d3>] ReturnCallPatch+0xa9/0x120 [intercepter]
[  102.479512]  [<ffffffffc072c0a4>] ReturnCallPatchHandler+0x6c/0xc4 [intercepter]
[  102.479514]  [<ffffffff890d3351>] ? __wake_up_common_lock+0x91/0xc0
[  102.479519]  [<ffffffff896ca18a>] ? tcp_get_metrics+0x1fa/0x2e0
[  102.479521]  [<ffffffff896baaff>] ? tcp_finish_connect+0xdf/0x130
[  102.479524]  [<ffffffffc070923d>] ? ReturnStub8+0x11/0x1c [km_services]
[  102.479526]  [<ffffffff896bb19a>] ? tcp_rcv_state_process+0x64a/0xf50
[  102.479527]  [<ffffffff896c545b>] tcp_v4_do_rcv+0x17b/0x350
[  102.479529]  [<ffffffff896c6c3c>] tcp_v4_rcv+0x7dc/0x9e0
[  102.479531]  [<ffffffffc066a2ce>] ? ipv4_confirm+0x4e/0x100 [nf_conntrack_ipv4]
[  102.479534]  [<ffffffff8969eead>] ip_local_deliver_finish+0xbd/0x200
[  102.479535]  [<ffffffff8969f1a0>] ip_local_deliver+0x60/0xe0
[  102.479537]  [<ffffffff8969edf0>] ? ip_rcv_finish+0x370/0x370
[  102.479538]  [<ffffffff8969eb10>] ip_rcv_finish+0x90/0x370
[  102.479539]  [<ffffffff8969f4e0>] ip_rcv+0x2c0/0x420
[  102.479541]  [<ffffffff8969ea80>] ? inet_del_offload+0x40/0x40
[  102.479544]  [<ffffffff89654829>] __netif_receive_skb_core+0x729/0xa10
[  102.479546]  [<ffffffff89654b28>] __netif_receive_skb+0x18/0x60
[  102.479547]  [<ffffffff89654bb0>] netif_receive_skb_internal+0x40/0xc0
[  102.479548]  [<ffffffff89655838>] napi_gro_receive+0xd8/0x100
[  102.479554]  [<ffffffffc019d6b4>] vmxnet3_rq_rx_complete+0x924/0xef0 [vmxnet3]
[  102.479557]  [<ffffffffc019e0e6>] vmxnet3_poll_rx_only+0x36/0xa0 [vmxnet3]
[  102.479558]  [<ffffffff896551cf>] net_rx_action+0x26f/0x390
[  102.479563]  [<ffffffff890a4b95>] __do_softirq+0xf5/0x280
[  102.479566]  [<ffffffff897974ec>] call_softirq+0x1c/0x30 
[  102.479572]  [<ffffffff8902f715>] do_softirq+0x65/0xa0
[  102.479573]  [<ffffffff890a4f15>] irq_exit+0x105/0x110
[  102.479576]  [<ffffffff89798936>] do_IRQ+0x56/0xf0
[  102.479578]  [<ffffffff8978a36a>] common_interrupt+0x16a/0x16a
[  102.479579]  <EOI> 
[  104.559463] ------------[ cut here ]------------

Environment

  • Red Hat Enterprise Linux 7
  • 3rd party module 'intercepter' loaded

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content