What is DIF/DIX (also known as PI)? Does Red Hat support it?

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Linux 7.2 or later

Issue

  • What is DIF/DIX (also known as PI)? Does Red Hat support it?

Resolution

DIF is a new feature recently added to the SCSI Standard. It increases the size of the commonly-used 512-byte disk block from 512 to 520 bytes. The extra bytes comprise the Data Integrity Field (DIF). The basic idea is that the HBA will calculate a checksum value for the data block on writes, and store it in the DIF. The storage device will confirm the checksum on receive, and store the data plus checksum. On a read, the checksum will be checked by the storage device and by the receiving HBA.

The Data Integrity Extension (DIX) allows this check to move up the stack: the application calculates the checksum and passes it to the HBA, to be appended to the 512 byte data block. This provides a full end-to-end data integrity check.

Some vendors have adopted the name Protection Information (PI) to refer to the DIF/DIX functionality. There is one difficulty associated with DIF/DIX on Linux - the memory management system may change the data buffer while it is queued for a write. If it does this, then the memory management system must remember to keep that page marked dirty after the I/O succeeds. If the memory management system changes the data in the buffer after the checksum is calculated, but before the write is done, then the checksum test will fail, the write will fail, and the filesystem will go read-only, or some similar failure will occur.

Because of this, users of Red Hat Enterprise Linux 6 should note the following: The DIF/DIX hardware checksum feature must only be used with applications that exclusively issue O_DIRECT I/O. These applications may use the raw block device, or the XFS file system in O_DIRECT mode. (XFS is the only filesystem that does not fall back to buffered IO when doing certain allocation operations). Only applications designed for use with O_DIRECT I/O and DIF/DIX hardware should enable this feature.

DIF/DIX is a Tech Preview in RHEL 6.0. There are currently just two driver/hba combinations that have this support: Emulex lpfc and LSI mpt2sas. There are just a few storage vendors who support it: the Netapp Engenio FC RAID array, FUJITSU ETERNUS Storage system, and certain Hitachi SAS disks. We expect additional storage vendors to support this feature in the future.

Note that in RHEL7, ASMLib should not be used with enabled DIF/DIX (ref. KB#315643).

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

2 Comments

In a case of RHEL6 if we need to disable DIF/DIX support what should we need to do?

You should disable the DIF/DIX attribute for the LUN on the storage device. This would be storage array dependent.

There is a sysfs parameter "protection_type" for the scsi_disk object, which if set to 0 (e.g. echo "0" > {sysfs node}) will disable the use of the READ_32/WRITE_32 commands with the protection information, but this does not stop the block layer code from computing the checksums, and I'm not sure the storage device would even accept non-DIF/DIX commands on a DIF/DIX LUN. If it did, write operations would not update the tags properly. If the device was later used with DIF/DIX enabled on the host, errors would likely result (unless the exact same sectors were overwritten).

When the storage device is configured as non-DIF/DIX, the LUN attributes reported to the SCSI layer cause everything to be configured properly at device scan time.