rsyslog crashes on segfault when disk-assisted queues are corrupted
Issue
-
rsyslog segfaults at startup, and systemd restarts it in a forever loop
# systemctl start rsyslog Job for rsyslog.service failed because a fatal signal was delivered to the control process. See "systemctl status rsyslog.service" and "journalctl -xe" for details. # journalctl | grep rsyslog Nov 11 13:15:08 server rsyslogd[928855]: action-0-builtin:omfwd queue[DA]: error dequeueing element - ignoring, but strange things may happen Nov 11 13:15:08 server rsyslogd[928855]: action-0-builtin:omfwd queue[DA]: qDeqDisk error happened at around offset 84191 Nov 11 13:15:08 server rsyslogd[928855]: action-0-builtin:omfwd queue[DA]: error dequeueing element - ignoring, but strange things may happen Nov 11 13:15:08 server rsyslogd[928855]: action-0-builtin:omfwd queue[DA]: qDeqDisk error happened at around offset 84344 Nov 11 13:15:08 server rsyslogd[928855]: action-0-builtin:omfwd queue[DA]: error dequeueing element - ignoring, but strange things may happen Nov 11 13:15:08 server kernel: traps: rs:action-0-bui[928866] general protection fault ip:55f977d51779 sp:7fb6251eb9b0 error:0 in rsyslogd[55f977d0c000+6b000] Nov 11 13:15:08 server systemd-coredump[928963]: Resource limits disable core dumping for process 928855 (rsyslogd). Nov 11 13:15:08 server systemd-coredump[928963]: Process 928855 (rsyslogd) of user 0 dumped core. Nov 11 13:15:08 server systemd[1]: rsyslog.service: Main process exited, code=dumped, status=11/SEGV Nov 11 13:15:08 server systemd[1]: rsyslog.service: Failed with result 'core-dump'. Nov 11 13:15:08 server systemd[1]: rsyslog.service: Scheduled restart job, restart counter is at 5. Nov 11 13:15:08 server systemd[1]: Stopped System Logging Service. Nov 11 13:15:08 server systemd[1]: rsyslog.service: Start request repeated too quickly. Nov 11 13:15:08 server systemd[1]: rsyslog.service: Failed with result 'core-dump'. Nov 11 13:15:08 server systemd[1]: Failed to start System Logging Service. -
rsyslog shows this message in debug mode on stderr
<Obj:1:msg:1: +iProtocolVersion:2:1:0: +iSeverity:2:1:5: +iFacility:2:2:16: +m' 3817.443895127:action 0 queue[DA]:Reg/w0: error property name: 'iFacilit<Obj' 3817.443898262:action 0 queue[DA]:Reg/w0: error var type: '1' Segmentation fault -
The following message is seen in
dmesgoutputkernel: rs:action 0 que[29724]: segfault at 5 ip 0000558247132cec sp 00007f75b3cbfb50 error 4 in rsyslogd[55824710c000+95000]
Environment
- Red Hat Enterprise Linux 7
- Red Hat Enterprise Linux 8
- Red Hat Enterprise Linux 9
- Red Hat Enterprise Linux 10
- rsyslog
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.