Hugepage amount configuration on RHEL7 with Hugepagesize 1GB

Latest response

I am quite new to this topic so please excuse me if I am asking quite obvious things. My current RHEL7 comes with an Hugepagesize = 2048 kB and zero HugePages like this:

HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

The system allows me freely to modify the amount of HugePages to any value xx on the fly with either of these two options:

echo xx > /proc/sys/vm/nr_hugepages
sysctl -w vm.nr_hugepages=xx

Now I added the following kernel boot parameters to grub to boot the system with 32 HugePages of 1GB:

cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-123.el7.x86_64 root=UUID=38b95071-ece4-49c3-b062-bd8a188e95cd ro crashkernel=auto rd.lvm.lv=vg00/root rd.lvm.lv=vg00/swap crashkernel=auto vconsole.font=latarcyrheb-sun16 vconsole.keymap=fi rhgb quiet default_hugepagesz=1GB hugepagesz=1GB hugepages=32

AnonHugePages: 4687872 kB
HugePages_Total: 32
HugePages_Free: 32
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 1048576 kB

After that I can not use echo nor sysctl anymore to modify the amount of HugePages:

sysctl -w vm.nr_hugepages=34
sysctl: setting key "vm.nr_hugepages": Invalid argument
vm.nr_hugepages = 34

echo 34 > /proc/sys/vm/nr_hugepages
-bash: echo: write error: Invalid argument

Does this happen because the amount of HugePages has to be set during boot in case of 1GB HugePages caused by the possible memory fragmentation issue? The issue is a bit annoying for me as in this case any modification of the amount of HugePages would require a server reboot. Anybody here got an answer to this?

Responses

Hello,

We would recommend you to disable the transparent huge pages. Then try entering the desired hugepages value manually in /etc/sysctl file and execute "sysctl -p" or reboot the server to check if configured huge pages are reflecting or not.

This occurs by design. See the note on allocation in the kernel documentation:

/usr/share/doc/kernel-doc-3.10.0/Documentation/kernel-parameters.txt

        hugepages=      [HW,X86-32,IA-64] HugeTLB pages to allocate at boot.
        hugepagesz=     [HW,IA-64,PPC,X86-64] The size of the HugeTLB pages.
                        On x86-64 and powerpc, this option can be specified
                        multiple times interleaved with hugepages= to reserve
                        huge pages of different sizes. Valid pages sizes on
                        x86-64 are 2M (when the CPU supports "pse") and 1G
                        (when the CPU supports the "pdpe1gb" cpuinfo flag)
                        Note that 1GB pages can only be allocated at boot time
                        using hugepages= and not freed afterwards.

If you require to be able to re-allocate HugeTLB mid-boot, then use 2M pages.

How i can change the "AnonHugePages=0" in /etc/sysctl.conf ?.can some one help me on this

[root@localhost ~]# grep Huge /proc/meminfo AnonHugePages: 10240 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB