RHEL-5: BUG: soft lockup - CPU#8 stuck for 10s! [bond0:4990] due to bonding options.

Solution Verified - Updated -

Issue

  • BUG: Soft lockup messages found in /var/log/messages on Red Hat Enterprise Linux 5
  • Soft lockup messages appear as follows, and is exactly the same 24 hours later:
    kernel: BUG: soft lockup - CPU#8 stuck for 10s! [bond0:4990]
    kernel: CPU 8:
    kernel: Modules linked in: vfat fat usb_storage tun ipt_MASQUERADE iptable_nat ip_nat nfsd exportfs nfs_acl auth_rpcgss autofs4 hidp l2cap bluetooth lockd sunrpc
    scsi_dh_rdac cpufreq_ondemand acpi_cpufreq freq_table bridge bonding ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink iptable_filter ip_tables
    ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables x_tables be2iscsi ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio cxgb3i cxgb3
    8021q libiscsi_tcp libiscsi2 dm_round_robin dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec dell_wmi wmi button battery asus_acpi acpi_memhotplug
    ac ipv6 xfrm_nalgo crypto_api parport_pc lp parport ksm(U) kvm_intel(U) kvm(U) joydev i2c_i801 ide_cd cdc_ether cdrom i2c_core usbnet qla3xxx bnx2 sg e1000e pcspkr
    dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod ata_piix libata qla4xxx scsi_transport_iscsi2 scsi_transport_iscsi
    shpchp mptsas mptscsih mptbase scsi_transport_sas sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
    kernel: Pid: 4990, comm: bond0 Tainted: G      2.6.18-194.el5 #1
    kernel: RIP: 0010:[<ffffffff80065c23>]  [<ffffffff80065c23>] .text.lock.spinlock+0x29/0x30
    kernel: RSP: 0018:ffff81067e1f7d40  EFLAGS: 00000286
    kernel: RAX: ffff81067e1f7fd8 RBX: ffff81037fdbc000 RCX: ffff81037fdbc500
    kernel: RDX: 0000000000008948 RSI: ffff81067e1f7d90 RDI: ffff81037fdbd64c
    kernel: RBP: ffff81067e1f7e10 R08: ffff81067e1f7da0 R09: 0000000000000037
    kernel: R10: ffff810001027110 R11: 0000000010008040 R12: 0000000010008040
    kernel: R13: ffffffff8008ccb0 R14: ffff81067e1f7da0 R15: ffffffff80151248
    kernel: FS:  0000000000000000(0000) GS:ffff81010c4b7bc0(0000) knlGS:0000000000000000
    kernel: CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
    kernel: CR2: 00000000b7f33000 CR3: 0000000000201000 CR4: 00000000000026e0
    kernel:
    kernel: Call Trace:
    kernel:  [<ffffffff88277a75>] :bnx2:bnx2_ioctl+0x69/0xff
    kernel:  [<ffffffff886f9011>] :bonding:bond_check_dev_link+0xd9/0x1e4
    kernel:  [<ffffffff80063ff8>] thread_return+0x62/0xfe
    kernel:  [<ffffffff886fc748>] :bonding:bond_mii_monitor+0x90/0x496
    kernel:  [<ffffffff886fc6b8>] :bonding:bond_mii_monitor+0x0/0x496
    kernel:  [<ffffffff8004dc37>] run_workqueue+0x94/0xe4
    kernel:  [<ffffffff8004a472>] worker_thread+0x0/0x122
    kernel:  [<ffffffff800a198c>] keventd_create_kthread+0x0/0xc4
    kernel:  [<ffffffff8004a562>] worker_thread+0xf0/0x122
    kernel:  [<ffffffff8008e16d>] default_wake_function+0x0/0xe
    kernel:  [<ffffffff800a198c>] keventd_create_kthread+0x0/0xc4
    kernel:  [<ffffffff80032bdc>] kthread+0xfe/0x132
    kernel:  [<ffffffff8005efb1>] child_rip+0xa/0x11
    kernel:  [<ffffffff800a198c>] keventd_create_kthread+0x0/0xc4
    kernel:  [<ffffffff80032ade>] kthread+0x0/0x132
    kernel:  [<ffffffff8005efa7>] child_rip+0x0/0x11
    

Environment

  • Red Hat Enterprise Linux 5.5
    • kernel-2.6.18-194.el5
  • IBM System x3650 M2
  • Bonding mode 1
    • Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) (bnx2)
    • Bonding options:
      BONDING_OPTS="mode=1 miimon=100 use_carrier=0"
      
  • Virtual bridge and KVM guests

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.