IO scheduler errors in system logs: cfq: forced dispatching is broken on Red Hat Enterprise Linux 5
Issue
- Seeing the following errors in /var/log/messages in production
kernel: cfq: forced dispatching is broken (nr_sorted=11), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=4294967285), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=28), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=4294967268), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=35), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=4294967261), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=47), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=4294967249), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=59), please report this
kernel: cfq: forced dispatching is broken (nr_sorted=4294967237), please report this
Environment
-
Red Hat Enterprise Linux 5 (RHEL5)
- Issue observed on RHEL5.5 (2.6.18-194.el5). Currently unknown if other versions are affected.
-
CFQ I/O scheduler
- No other schedulers are affected.
- Issue occurs only when I/O's are inserted at the tail of the queue (not when inserting at the beginning or merging I/O's).
-
device-mapper-multipath
path_grouping_policy multibusandrr_min_io 1is set (one I/O sent to each path before switching to the next).path_checkeris set totur, which triggers the error message (usually at intervals of 4xpolling_interval)
-
EMC SYMMETRIX SAN
- SRDF is disabled
- No write pacing.
- Delays (up to 10ms) in flushing and allocating cache are seen on the SAN performance graphs.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.
