Half of the paths to the same LUN show lesser throughput as compared to other half.

Solution Verified - Updated -

Issue

  • IO throughput is not balanced across all the paths to the LUN.
  • Half of the paths to the same LUN show lesser throughput as compared to other half.
  • IO Performance degradation in Enterprise array

Out of 4 paths to the LUN, 2 paths are showing one third throughput. IO is balanced unevenly.

01/31/2017 08:43:02 PM
  Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
  sdc               0.00     0.00    0.00    3.00     0.00    12.00     8.00     0.00    0.67    0.00    0.67   0.67   0.20
  sdd               0.00     0.00    0.00    4.00     0.00    36.00    18.00     0.00    0.00    0.00    0.00   0.00   0.00
  sda               0.00     0.00    0.00   20.00     0.00 61440.00  6144.00     0.49   23.75    0.00   23.75  24.75  49.50
  sde               0.00     0.00    0.00   21.00     0.00 64512.00  6144.00     0.49   23.52    0.00   23.52  23.38  49.10
  sdb               0.00     0.00    0.00   21.00     0.00 21504.00  2048.00     0.15    7.52    0.00    7.52   7.38  15.50
  sdf               0.00     0.00    0.00   21.00     0.00 21504.00  2048.00     0.18    8.67    0.00    8.67   8.67  18.20
  dm-0              0.00    82.00    0.00   83.00     0.00 168960.00  4071.33     1.32   15.78    0.00   15.78  11.94  99.10
  dm-15             0.00     0.00    0.00    6.00     0.00    24.00     8.00     0.00    0.17    0.00    0.17   0.17   0.10
  01/31/2017 08:43:03 PM
  Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
  sdc               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
  sdd               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
  sda               0.00     0.00    0.00   22.00     0.00 67584.00  6144.00     0.48   22.86    0.00   22.86  21.95  48.30
  sde               0.00     0.00    0.00   21.00     0.00 64512.00  6144.00     0.50   23.00    0.00   23.00  23.95  50.30
  sdb               0.00     0.00    0.00   22.00     0.00 22528.00  2048.00     0.17    7.50    0.00    7.50   7.50  16.50
  sdf               0.00     0.00    0.00   21.00     0.00 21504.00  2048.00     0.16    7.81    0.00    7.81   7.81  16.40
  dm-0              0.00    86.00    0.00   86.00     0.00 176128.00  4096.00     1.31   15.29    0.00   15.29  11.50  98.90
  dm-15             0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
  01/31/2017 08:43:04 PM
  Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
  sdc               0.00     0.00    0.00   14.00     0.00    68.00     9.71     0.02    1.43    0.00    1.43   0.43   0.60
  sdd               0.00     0.00    0.00   13.00     0.00    64.00     9.85     0.02    1.31    0.00    1.31   0.38   0.50
  sda               0.00     0.00    0.00   21.00     0.00 64512.00  6144.00     0.50   23.81    0.00   23.81  23.86  50.10
  sde               0.00     0.00    0.00   22.00     0.00 67584.00  6144.00     0.48   22.82    0.00   22.82  21.91  48.20
  sdb               0.00     0.00    0.00   21.00     0.00 21504.00  2048.00     0.15    7.33    0.00    7.33   7.33  15.40
  sdf               0.00     0.00    0.00   21.00     0.00 21504.00  2048.00     0.16    7.67    0.00    7.67   7.71  16.20
  dm-0              0.00    86.00    0.00   85.00     0.00 175104.00  4120.09     1.30   15.49    0.00   15.49  11.56  98.30
  dm-15             0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
  01/31/2017 08:43:05 PM
  Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
  sdc               0.00     0.00    0.00    1.00     0.00     4.00     8.00     0.00    1.00    0.00    1.00   1.00   0.10
  sdd               0.00     0.00    0.00    1.00     0.00     8.00    16.00     0.00    0.00    0.00    0.00   0.00   0.00
  sda               0.00     0.00    0.00   21.00     0.00 64512.00  6144.00     0.49   23.48    0.00   23.48  23.48  49.30
  sde               0.00     0.00    0.00   21.00     0.00 64512.00  6144.00     0.49   23.19    0.00   23.19  23.19  48.70
  sdb               0.00     0.00    0.00   21.00     0.00 21504.00  2048.00     0.15    7.19    0.00    7.19   7.19  15.10
  sdf               0.00     0.00    0.00   21.00     0.00 21504.00  2048.00     0.16    7.38    0.00    7.38   7.43  15.60
  dm-0              0.00    84.00    0.00   84.00     0.00 172032.00  4096.00     1.29   15.36    0.00   15.36  11.75  98.70
  dm-15             0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00

READ as well as WRITE throughput was less for an enterprise array

READ
dd if=/dev/mapper/mpathb of=/dev/null bs=4096k iflag=direct count=1953
1953+0 records in
1953+0 records out
8191475712 bytes (8.2 GB) copied, 36.284 s, 226 MB/s

WRITE
dd if=/dev/zero of=/dev/mapper/mpathb bs=4096k oflag=direct count=1953
1953+0 records in
1953+0 records out
8191475712 bytes (8.2 GB) copied, 53.8566 s, 152 MB/s

Environment

  • Red Hat Enterprise Linux 6

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.