NVMe performance degradation RHEL 7.2 to 7.3

Latest response

Simple read tests show a significant performance degradation on RHEL 7.3

RHEL/CentOS 7.2

dd if=/dev/nvme0n1 of=/dev/null bs=256k iflag=direct
102020939776 bytes (102 GB) copied, 47,6167 s, 2,1 GB/s

RHEL/CentOS 7.3

dd if=/dev/nvme0n1 of=/dev/null bs=256k iflag=direct
204656082944 bytes (205 GB) copied, 162,362 s, 1,3 GB/s

System

Supermicro X10DRH
NVMe: Intel P3700 400GB PCIe card

The Intel P3700 400GB NVMe card should be capable of 2,7 GB/s reads and 1,2 GB/s writes - what can I do to reach the limits of the hardware ?

Responses

Are the RHEL 7.2 and RHEL 7.3 instances running on the same hardware spec? Performance varies depending on capacity and form factor.

You might struggle to reach limitations of the P3700 due to factors external to the device. More on it's capabilities and performance. Storage Review - Intel SSD DC P3700 2.5" NVMe SSD

Yes same motherboard - Supermicro X10DRH, Intel P3700 400GB PCIe card - not an external drive. http://www.intel.com/content/www/us/en/solid-state-drives/solid-state-drives-dc-p3700-series.html

We tried to use two of those cards as striped dm_cache for a big RAID6 volume but the sequential read performance from the cached LV was significant slower than reading direct from the RAID6 - I then checked the parts with the "dd" test above.

I will upgrade the 7.2 machine to 7.3 to verify the performance degradation.

Interesting issue, please make sure you post back your results.. very interested in what you find!

Interesting. Looking forward to hearing about the performance after upgrading to 7.3.

Has nothing todo with the 7.3 upgrade - seems to depend on the hardware ! Performance on this test is nearly identical beetween 7.2 and 7.3

Motherboard           Result
Supermicro X10DRH     1,3 GB/s (tested on three different machines)
Supermicro X10DRL-i   2,1 GB/s (only one machine available)

I've contacted our Supermicro reseller on this issue - will report back if this problem can be resolved.

Thanks for sharing. Quite a difference between the two boards.

Please keep us updated on the response from SuperMicro.

I suspect that placement of the card matters. Some PCIe slots are more local to the CPUs than others on Supermicro designs and it's worth consulting the circuit block diagram to see which slot is connected to which CPU (or bridge)

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.