Machine slowdown or lockup, massive rate of logging seen on console
Issue
- Slowdown of the machine is observed, accompanied by stack traces and massive logging on the console (
tty
). This can go as far as leading to a crash or softlockup. - Console logging has been known to produce soft lockups, hard lockups, NTP or time instability, and can cause various other similar issues.
- The behavior is triggered by any high rate logging to the
tty
console (which is framebuffer by default), a typical situation is usingiptables
LOG
target, and generating or encountering massive traffic that is matched by the-j LOG
rules. - One example of such traffic seen was being generated for security auditing, another was generated "naturally" in a Windows network by Samba NetBIOS broadcast.
- Hardlockup or soft lockup panic in framebuffer due to repeated kernel printk.
- Lockup in framebuffer console like:
Call Trace:
[<ffffffff80182784>] cfb_fillrect+0x1ed/0x2f6
[<ffffffff8017beb0>] fbcon_redraw+0x130/0x169
[<ffffffff8017f07e>] bit_putcs+0x42b/0x481
[<ffffffff80156347>] __next_cpu+0x19/0x28
[<ffffffff8017f564>] bit_cursor+0x490/0x4ac
[<ffffffff8017a2cb>] fbcon_putcs+0x20e/0x260
[<ffffffff8017ec53>] bit_putcs+0x0/0x481
[<ffffffff8017be53>] fbcon_redraw+0xd3/0x169
- Lockups detected like
NMI Watchdog detected LOCKUP on CPU #
Environment
- Red Hat Enterprise Linux
- Default
kernel.printk
configuration (console log level is 6) - Heavy kernel logging to console
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.