Chapter 3. Configure Page Memory
3.1. About Page Memory
A memory page is a fixed size, continuous block of memory and is used when transferring data from one storage medium to another, and to allocate memory. In some architectures, larger sized pages are available for improved memory allocation. These pages are known as large (or huge) pages.
The default memory page size in most operating systems is 4 kilobytes (kb). For a 32-bit operating system the maximum amount of memory is 4 GB, which equates to 1,048,576 memory pages. A 64-bit operating system can address 18 Exabytes of memory (in theory), resulting in a very large number of memory pages. The overhead of managing such a large number of memory pages is significant, regardless of the operating system.
Large memory pages are pages of memory which are significantly larger than 4 kb (usually 2 Mb). In some instances it is configurable from 2 MB to 2 GB, depending on the CPU architecture.
Large memory pages are locked in memory, and cannot be swapped to disk like normal memory pages. The advantage for this is if the heap is using large page memory it can not be paged or swapped to disk so it is always readily available. For Linux, the disadvantage is that applications must attach to it using the correct flag for the
shmget() system call. Additionally, the proper security permissions are required for the
memlock() system call. For any application that does not have the ability to use large page memory, the server behaves as if the large page memory does not exist, which can be a problem.
For additional information on page size refer to the Red Hat Enterprise Linux Configuring Hugetlb Huge Pages.
3.2. Configure Page Memory
Page memory configuration to optimize Red Hat JBoss Data Grid’s performance must be implement at the operating system level and at the JVM level. The provided instructions are for the Red Hat Enterprise Linux operating system. Use both the operating system level and JVM level instructions for optimal performance.
Configure Page Memory for Red Hat Enterprise Linux
Set the Shared Memory Segment Size
As root, set the maximum size of a shared memory segment in bytes; below we define this to be 32 GB:
# echo "kernel.shmmax = 34359738368" >> /etc/sysctl.conf
Set the Huge Pages
The number of huge pages is set to the total amount of memory the JVM will consume (heap, meta space, thread stacks, native code) divided by the
Hugepagesize. In Red Hat Enterprise Linux systems
Hugepagesizeis set to 2048 KB.
The number of huge pages required can be determined by the following formula:
Heap + Meta space + Native JVM Memory + (Number of Threads * Thread Stack Size)
Assuming a JVM with a 32 GB Heap, 2 GB of Meta space, a 512 MB native footprint, and 500 threads, each with a default size of 1 MB per thread, we have the following equation.
32*(1024*1024*1024) + 2*(1024*1024*1024) + 512*(1024*1024) + (500 * 1024*1024)
The resulting value can now be converted to hugepages. Since there are 2048 KB in a single hugepage we perform the following:
37568380928 / (2*1024*1024)
As root, set the number of huge pages determined from the previous steps to be allocated to the operating system:
# echo "vm.nr_hugepages = 17914" >> /etc/sysctl.conf
Assigned Shared Memory Permissions
As root, set the ID of the user group that is allowed to create shared memory segments using the hugetlb_shm_group file. This value should match the group id of the user running the JVM:
# echo "vm.hugetlb_shm_group = 500" >> /etc/sysctl.conf
Update the Resource Limits
To allow a user to lock the required amount of memory, update the resource limits in the /etc/security/limits.conf file by adding the following:
jboss soft memlock unlimited jboss hard memlock unlimited
This change allows the user
jbossto lock the system’s available memory.
Configure Authentication using PAM
Linux’s PAM handles authentication for applications and services. Ensure that the configured system resource limits apply when using
Configure PAM for su
Add the following line to the /etc/pam.d/su file:
session required pam_limits.so
Configure PAM for sudo
Add the following line to the /etc/pam.d/sudo file:
session required pam_limits.so
- Reboot the system for the changes to take effect. Since Huge Pages allocate a contiguous block of memory these must be allocated at system boot; attempts to claim these dynamically while the system is running may result in system hangs if the memory is unable to be reclaimed.
Procedure: Configure Page Memory for the JVM
Set the Heap Size
-Xmxparameters to set the minumum and maximum heap sizes for your JVM, as discussed in Java Virtual Machine Settings.
Enable Large Pages
Enabled large pages for the JVM by adding the following parameter, as discussed in Java Virtual Machine Settings: