Ceph volumes hang or stall when iptables/nftables is running
Issue
- Whenever we start virtual machines backed by a Ceph volume, disk I/O on these VMs will stall soon after the initial boot. dmesg of affected machines contains the following messages:
[ 615.074762] Not tainted 4.18.0-147.el8.x86_64 #1
[ 615.075418] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 615.076434] xfsaild/vda1 D 0 416 2 0x80000000
[ 615.077173] Call Trace:
[ 615.077512] ? __schedule+0x253/0x830
[ 615.077997] schedule+0x28/0x70
[ 615.078563] xfs_log_force+0x160/0x2e0 [xfs]
[ 615.079155] ? wake_up_q+0x70/0x70
[ 615.079628] xfsaild+0x17b/0x7f0 [xfs]
[ 615.080161] ? xfs_trans_ail_cursor_first+0x80/0x80 [xfs]
[ 615.080864] kthread+0x112/0x130
[ 615.081286] ? kthread_flush_work_fn+0x10/0x10
[ 615.081880] ret_from_fork+0x35/0x40
- Another footprint is that Ceph Volume creation can hang or take a long time.
Environment
- Red hat Enterprise Linux 8.1
- kernel-4.18.0-147.5.1.el8
- OSP16
- iptables using nftables backend
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.