How to configure max write bandwidth to disk for service or user with cgroups

Issue

• The objective is to limit the disk bandwidth of an application that sometimes saturates the disk and then impact other services.

• The use of BlockIOWriteBandwidth in a systemd service will create a cgroup with a limited bandwidth:

[Service]
Type=forking
# Max storage write throughput on disk1
BlockIOWriteBandwidth=/dev/mapper/disk1 10M

• We can see the limitation in the cgroup:

$ cat /sys/fs/cgroup/blkio/service1.slice/service1.service/blkio.throttle.write_bps_device
253:0 10000000

• But the throughput remains the same on the disk:

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdc               0.00     0.00    0.00  478.00     0.00 52368.00   512.00   143.14  299.25    0.00  299.25   2.09 100.00

• The same result can be obtained with the tool fio:

 cgexec -g blkio:test_blkio fio  --randrepeat=1 --ioengine=libaio  --gtod_reduce=1 --name=test --filename=/root/file --bs=128k --iodepth=64 --size=1G --readwrite=randwrite
test: (g=0): rw=randwrite, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=libaio, iodepth=64
fio-3.1
bs: 1 (f=1): [w(1)][87.5%][r=0KiB/s,w=51.4MiB/s][r=0,w=411 IOPS][eta 00m:02s]

=> The bandwidth is still over 10MB/s