Unable to login to the system (eg. with ssh) after rsyslog crashes, it looks frozen
Issue
- System hangs after sending a large amount of syslog messages via TCP
- I'm not able to login through ssh, but the server answers ping requests
- abrt-addon-ccpp user space deadlock when rsyslog daemon crashes
- rsyslogd crashes and no core dump is saved by abrt
- Below are the common call traces found on the logs when the problem happens:
INFO: task rsyslogd:13421 blocked for more than 120 seconds.
Not tainted 2.6.32-431.29.2.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
rsyslogd D 0000000000000003 0 13421 1 0x00000080
ffff88204eb15c98 0000000000000082 ffff882053935a28 ffff882053933500
ffff88204eb15c68 ffffffff810af000 ffff882053935a28 0000000000000000
ffff882050d105f8 ffff88204eb15fd8 000000000000fbc8 ffff882050d105f8
Call Trace:
[<ffffffff810af000>] ? exit_robust_list+0x90/0x160
[<ffffffff810767e5>] exit_mm+0x95/0x180
[<ffffffff81076c2f>] do_exit+0x15f/0x870
[<ffffffff81077398>] do_group_exit+0x58/0xd0
[<ffffffff8108cd46>] get_signal_to_deliver+0x1f6/0x460
[<ffffffff8100a265>] do_signal+0x75/0x800
[<ffffffff810149b9>] ? read_tsc+0x9/0x20
[<ffffffff811a04b8>] ? poll_select_copy_remaining+0xf8/0x150
[<ffffffff8100aa80>] do_notify_resume+0x90/0xc0
[<ffffffff8100b341>] int_signal+0x12/0x17
Environment
- Red Hat Enterprise Linux 6
- abrt-addon-ccpp package installed and abrt-ccpp service enabled
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.