Chapter 3. Configure Page Memory

3.1. About Page Memory

A memory page is a fixed size, continuous block of memory and is used when transferring data from one storage medium to another, and to allocate memory. In some architectures, larger sized pages are available for improved memory allocation. These pages are known as large (or huge) pages.

The default memory page size in most operating systems is 4 kilobytes (kb). For a 32-bit operating system the maximum amount of memory is 4 GB, which equates to 1,048,576 memory pages. A 64-bit operating system can address 18 Exabytes of memory (in theory), resulting in a very large number of memory pages. The overhead of managing such a large number of memory pages is significant, regardless of the operating system.

Large memory pages are pages of memory which are significantly larger than 4 kb (usually 2 Mb). In some instances it is configurable from 2 MB to 2 GB, depending on the CPU architecture.

Large memory pages are locked in memory, and cannot be swapped to disk like normal memory pages. The advantage for this is if the heap is using large page memory it can not be paged or swapped to disk so it is always readily available. For Linux, the disadvantage is that applications must attach to it using the correct flag for the shmget() system call. Additionally, the proper security permissions are required for the memlock() system call. For any application that does not have the ability to use large page memory, the server behaves as if the large page memory does not exist, which can be a problem.

For additional information on page size refer to the Red Hat Enterprise Linux Configuring Hugetlb Huge Pages.

3.2. Configure Page Memory

Page memory configuration to optimize Red Hat JBoss Data Grid’s performance must be implement at the operating system level and at the JVM level. The provided instructions are for the Red Hat Enterprise Linux operating system. Use both the operating system level and JVM level instructions for optimal performance.

Configure Page Memory for Red Hat Enterprise Linux

  1. Set the Shared Memory Segment Size

    As root, set the maximum size of a shared memory segment in bytes; below we define this to be 32 GB:

    # echo "kernel.shmmax = 34359738368" >> /etc/sysctl.conf
  2. Set the Huge Pages

    The number of huge pages is set to the total amount of memory the JVM will consume (heap, meta space, thread stacks, native code) divided by the Hugepagesize. In Red Hat Enterprise Linux systems Hugepagesize is set to 2048 KB.

    1. The number of huge pages required can be determined by the following formula:

      Heap + Meta space + Native JVM Memory + (Number of Threads * Thread Stack Size)
    2. Assuming a JVM with a 32 GB Heap, 2 GB of Meta space, a 512 MB native footprint, and 500 threads, each with a default size of 1 MB per thread, we have the following equation.

      32*(1024*1024*1024) + 2*(1024*1024*1024) + 512*(1024*1024) + (500 * 1024*1024)
    3. The resulting value can now be converted to hugepages. Since there are 2048 KB in a single hugepage we perform the following:

      37568380928 / (2*1024*1024)
    4. As root, set the number of huge pages determined from the previous steps to be allocated to the operating system:

      # echo "vm.nr_hugepages = 17914" >> /etc/sysctl.conf
  3. Assigned Shared Memory Permissions

    As root, set the ID of the user group that is allowed to create shared memory segments using the hugetlb_shm_group file. This value should match the group id of the user running the JVM:

    # echo "vm.hugetlb_shm_group = 500" >> /etc/sysctl.conf
  4. Update the Resource Limits

    To allow a user to lock the required amount of memory, update the resource limits in the /etc/security/limits.conf file by adding the following:

    jboss      soft   memlock      unlimited
    jboss      hard   memlock      unlimited

    This change allows the user jboss to lock the system’s available memory.

  5. Configure Authentication using PAM

    Linux’s PAM handles authentication for applications and services. Ensure that the configured system resource limits apply when using su and sudo as follows:

    • Configure PAM for su

      Add the following line to the /etc/pam.d/su file:

      session    required   pam_limits.so
    • Configure PAM for sudo

      Add the following line to the /etc/pam.d/sudo file:

      session    required   pam_limits.so
  6. Reboot the system for the changes to take effect. Since Huge Pages allocate a contiguous block of memory these must be allocated at system boot; attempts to claim these dynamically while the system is running may result in system hangs if the memory is unable to be reclaimed.

Procedure: Configure Page Memory for the JVM

  1. Set the Heap Size

    Use the -Xms and -Xmx parameters to set the minumum and maximum heap sizes for your JVM, as discussed in Java Virtual Machine Settings.

  2. Enable Large Pages

    Enabled large pages for the JVM by adding the following parameter, as discussed in Java Virtual Machine Settings:

    -XX:+UseLargePages