Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

28.3. Configuring Persistent Memory for File System Direct Access

File system direct access requires the namespace to be configured to the fsdax mode. This mode allows for the direct access programming model. When a device is configured in the fsdax mode, a file system can be created on top of it, and then mounted with the -o fsdax mount option. Then, any application that performs an mmap() operation on a file on this file system gets direct access to its storage. See the following example:
# ndctl create-namespace --force --reconfig=namespace0.0 --mode=fsdax --map=mem
{
   "dev":"namespace0.0",
   "mode":"fsdax",
   "size":17177772032,
   "uuid":"e6944638-46aa-4e06-a722-0b3f16a5acbf",
   "blockdev":"pmem0"
}
In the example, namespace0.0 is converted to namespace fsdax mode. With the --map=mem argument, ndctl puts operating system data structures used for Direct Memory Access (DMA) in system DRAM.
To perform DMA, the kernel requires a data structure for each page in the memory region. The overhead of this data structure is 64 bytes per 4-KiB page. For small devices, the amount of overhead is small enough to fit in DRAM with no problems. For example, the 16-GiB namespace only requires 256MiB for page structures. Because NVDIMM devices are usually small and expensive, storing the kernel’s page tracking data structures in DRAM is preferable, as indicated by the --map=mem parameter.
In the future, NVDIMM devices might be terabytes in size. For such devices, the amount of memory required to store the page tracking data structures might exceed the amount of DRAM in the system. One TiB of persistent memory requires 16 GiB just for page structures. As a result, specifying the --map=dev parameter to store the data structures in the persistent memory itself is preferable in such cases.
After configuring the namespace in the fsdax mode, the namespace is ready for a file system. Starting with Red Hat Enterprise Linux 7.3, both the Ext4 and XFS file system enable using persistent memory as a Technology Preview. File system creation requires no special arguments. To get the DAX functionality, mount the file system with the dax mount option. For example:
# mkfs -t xfs /dev/pmem0
# mount -o dax /dev/pmem0 /mnt/pmem/
Then, applications can use persistent memory and create files in the /mnt/pmem/ directory, open the files, and use the mmap operation to map the files for direct access.
When creating partitions on a pmem device to be used for direct access, partitions must be aligned on page boundaries. On the Intel 64 and AMD64 architecture, at least 4KiB alignment for the start and end of the partition, but 2MiB is the preferred alignment. By default, the parted tool aligns partitions on 1MiB boundaries. For the first partition, specify 2MiB as the start of the partition. If the size of the partition is a multiple of 2MiB, all other partitions are also aligned.