What is DIF/DIX (also known as PI)? Does Red Hat support it?

Resolution

DIF is a new feature recently added to the SCSI Standard. It adds 8 bytes to the end of each sector on disk.

        bit→
↓byte       7        6         5         4         3         2         1         0
       +--------+---------+---------+---------+---------+---------+---------+---------+
  0    |                                                                              |
       +---                                                                        ---+
       :                                  User Data                                   :
       +---                                                                        ---+
 n-1   |                                                                              | where n=512|4096
       +--------+---------+---------+---------+---------+---------+---------+---------+
       +--------+---------+---------+---------+---------+---------+---------+---------+
  n    | (msb)                                                                        |
       +--------          Logical Block Guard (CRC16 e.g.)                   ---------+
 n+1   |                                                                        (lsb) |
       +--------+---------+---------+---------+---------+---------+---------+---------+
 n+2   | (msb)                                                                        |
       +--------          Logical Block Application Tag                      ---------+
 n+3   |                                                                        (lsb) |
       +--------+---------+---------+---------+---------+---------+---------+---------+
 n+4   |                                                                              |
       +---                                                                        ---+
       :                  Logical Block Reference Tag                                 :
       +---                                                                        ---+
 n+7   |                                                                              | 
       +--------+---------+---------+---------+---------+---------+---------+---------+

It increases the size of the commonly-used 512-byte disk block from 512 to 520 bytes. The extra bytes comprise the Data Integrity Field (DIF). The basic idea is that the HBA will calculate a checksum value for the data block on writes, and store it in the DIF. The storage device will confirm the checksum on receive, and store the data plus checksum. On a read, the checksum will be checked by the storage device and by the receiving HBA.

The Data Integrity Extension (DIX) allows this check to move up the stack: the application calculates the checksum and passes it to the HBA, to be appended to the 512 byte data block. This provides a full end-to-end data integrity check.

DIF/DIX and RHEL6

Some vendors have adopted the name Protection Information (PI) to refer to the DIF/DIX functionality. There is one difficulty associated with DIF/DIX on Linux - the memory management system may change the data buffer while it is queued for a write. If it does this, then the memory management system must remember to keep that page marked dirty after the I/O succeeds. If the memory management system changes the data in the buffer after the checksum is calculated, but before the write is done, then the checksum test will fail, the write will fail, and the filesystem will go read-only, or some similar failure will occur.

Because of this, users of Red Hat Enterprise Linux 6 should note the following: The DIF/DIX hardware checksum feature must only be used with applications that exclusively issue O_DIRECT I/O. These applications may use the raw block device, or the XFS file system in O_DIRECT mode. (XFS is the only filesystem that does not fall back to buffered IO when doing certain allocation operations). Only applications designed for use with O_DIRECT I/O and DIF/DIX hardware should enable this feature.

DIF/DIX is a Tech Preview in RHEL 6.0 and later. There are currently just two driver/hba combinations that have this support: Emulex lpfc and LSI mpt2sas. There are just a few storage vendors who support it: the Netapp Engenio FC RAID array, FUJITSU ETERNUS Storage system, and certain Hitachi SAS disks. We expect additional storage vendors to support this feature in the future.

DIF/DIX and RHEL7/8

DIF/DIX is fully supported in RHEL 7.6 and later, provided that the hardware vendor has qualified it and provides full support for the particular HBA and storage array configuration it is connected to on RHEL. Contact your storage vendor for determination if the HBA/storage combination has been tested and fully supported on RHEL 7.6 or later. DIF/DIX is not supported on other configurations. It is not supported for use on the boot device, and it is not supported on virtualized guests. Red Hat does not support using ASMLib when DIF/DIX is enabled. DIF/DIX is enabled/disabled at the storage device, which involves various layers up to (and including) the application. The method for activating DIF on storage devices is device-dependent.