slow ssh login with soft lockups in selinux code
Issue
- ssh login is delayed 30 seconds or more.
-
soft lockups appear in
sshdorselinuxdefconnear the functionsecurity_get_user_sids.
Example:Jul 26 18:08:53 host1 kernel: NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [sshd:14871]or
Oct 17 11:20:27 host1 kernel: NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [selinuxdefcon:14224] Oct 17 11:20:27 host1 kernel: Modules linked in: ip6table_filter ip6_tables sctp_diag sctp dccp_diag dccp tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag vxlan ip6_udp_tunnel udp_tunnel ip_set_hash_net xt_comment xt_ipvs xt_set ip_set nfnetlink ip_vs_wlc ip_vs dummy ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock xfs libcrc32c ppdev vmw_balloon coretemp joydev pcspkr sg vmw_vmci shpchp i2c_piix4 parport_pc parport binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod sd_mod cdrom crc_t10dif crct10dif_generic crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops mptspi ahci ttm Oct 17 11:20:27 host1 kernel: scsi_transport_spi libahci ata_piix mptscsih drm libata mptbase crc32c_intel serio_raw vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod Oct 17 11:20:27 host1 kernel: CPU: 2 PID: 14224 Comm: selinuxdefcon Tainted: G L ------------ T 3.10.0-693.21.1.el7.x86_64 #1 Oct 17 11:20:27 host1 kernel: Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015 Oct 17 11:20:27 host1 kernel: task: ffff88081c6d2f70 ti: ffff880110b68000 task.ti: ffff880110b68000 Oct 17 11:20:27 host1 kernel: RIP: 0010:[<ffffffff812c8103>] [<ffffffff812c8103>] sidtab_context_to_sid+0xb3/0x480 Oct 17 11:20:27 host1 kernel: RSP: 0018:ffff880110b6bcf8 EFLAGS: 00000286 Oct 17 11:20:27 host1 kernel: RAX: 0000000000000000 RBX: ffff880110b6bdb8 RCX: 0000000000000000 Oct 17 11:20:27 host1 kernel: RDX: ffff880110b6bda0 RSI: 0000000000000000 RDI: ffffffff81fddb40 Oct 17 11:20:27 host1 kernel: RBP: ffff880110b6bd40 R08: ffff880110b6bdb8 R09: 0000000000000300 Oct 17 11:20:27 host1 kernel: R10: ffffffffffffffff R11: ffffffffffffffff R12: ffffffffffffffff Oct 17 11:20:27 host1 kernel: R13: ffffffffffffffff R14: 0000000000000137 R15: ffff880110b6bce8 Oct 17 11:20:27 host1 kernel: FS: 00007f2f3d561800(0000) GS:ffff88083fc80000(0000) knlGS:0000000000000000 Oct 17 11:20:27 host1 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 17 11:20:27 host1 kernel: CR2: 0000000001d8acf8 CR3: 000000010fe92000 CR4: 00000000000007e0 Oct 17 11:20:27 host1 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Oct 17 11:20:27 host1 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Oct 17 11:20:27 host1 kernel: Call Trace: Oct 17 11:20:27 host1 kernel: [<ffffffff812d4ca5>] ? mls_setup_user_range+0x145/0x250 Oct 17 11:20:27 host1 kernel: [<ffffffff812d1407>] security_get_user_sids+0x3f7/0x550 Oct 17 11:20:27 host1 kernel: [<ffffffff812c59bb>] sel_write_user+0x12b/0x210 Oct 17 11:20:27 host1 kernel: [<ffffffff8118bb56>] ? get_zeroed_page+0x16/0x20 Oct 17 11:20:27 host1 kernel: [<ffffffff812c5890>] ? sel_write_member+0x200/0x200 Oct 17 11:20:27 host1 kernel: [<ffffffff812c40fa>] selinux_transaction_write+0x4a/0x80 Oct 17 11:20:27 host1 kernel: [<ffffffff81205680>] vfs_write+0xc0/0x1f0 Oct 17 11:20:27 host1 kernel: [<ffffffff816c0661>] ? system_call_after_swapgs+0xae/0x146 Oct 17 11:20:27 host1 kernel: [<ffffffff8120649f>] SyS_write+0x7f/0xe0 Oct 17 11:20:27 host1 kernel: [<ffffffff816c0715>] system_call_fastpath+0x1c/0x21 Oct 17 11:20:27 host1 kernel: [<ffffffff816c0661>] ? system_call_after_swapgs+0xae/0x146 Oct 17 11:20:27 host1 kernel: Code: 0f 1f 84 00 00 00 00 00 41 8b 50 0c 85 d2 74 08 39 d0 0f 84 80 02 00 00 4d 8b 64 24 50 4d 85 e4 0f 84 a2 02 00 00 41 8b 44 24 14 <85> c0 75 d9 41 8b 48 0c 85 c9 75 e1 49 8b 00 49 39 44 24 08 75
Environment
- Red Hat Enterprise Linux 7
- Docker
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
