Redhat Enterprise 5.6 memory upgrade not recognized.

Latest response

I recently upgraded the RAM in a Dell T610 Server from 32GB to 96GB and when I run a freemem or /cat/meminfo is still shows the original RAM amount. I already:
1. installed all Redhat system and kernel updates.
2. Updated the Grub.conf file to reflect the new memory size.
3. Applied all firmware and bios updates.
4. The BIOS does recognize and passes the memory test.
5. The memory type is Dell original RAM and compatible.
6. Redhat 5.6 supposed to recognize upto 1TB.

What's Next!

Nick

Responses

I tried to upload the sosreport but it too large.

Did you also try to edit the Grub settings directly from the boot screen by typing e on the selected kernel and add mem=xxm? Link below talked about that.

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Installation_Guide/s2-trouble-ram.html

yes, I tried that! that was #2 on my list and pretty much the only recommendation I can find on dealing with this problem. I tried several different amounts and it still did not work.

Hey Nick, can you please provide us with
uname -a
cat /etc/redhat-release
cat /proc/cmdline
dmidecode -s bios-version
dmidecode | grep "Memory Device" | wc -l
-- see if that last line matches the number of DIMMS you just installed. Or, better yet...
dmidecode | grep -A5 "Memory Device" | grep Size

Then, do the following and see if any of the modules are detected, but not online:
grep line /sys/devices/system/memory/*/state

Thank You for responding!

UNAME: Linux localhost.localdomain 2.6.18-371.1.2.el5xen #1 SMP Mon Oct 7 16:38:30 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
Release: Red Hat Enterprise Linux Server release 5.10 (Tikanga)
CMDLINE: ro root=/dev/VolGroup00/LogVol00 rhgb quiet mem=92000M
BIOS ver> 6.3.0
Memory Device: 12 24 168
I had not result on State but included all mem info:
Handle 0x1000, DMI type 16, 15 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: Multi-bit ECC
Maximum Capacity: 192 GB
Error Information Handle: Not Provided
Number Of Devices: 12

Handle 0x1100, DMI type 17, 28 bytes
Memory Device
Array Handle: 0x1000
Error Information Handle: Not Provided
Total Width: 72 bits
Data Width: 64 bits
Size: 8192 MB
Form Factor: DIMM
Set: 1
Locator: DIMM_A1
Bank Locator: Not Specified
Type: DDR3
Type Detail: Synchronous Registered (Buffered)
Speed: 1333 MHz
Manufacturer: 00AD04B380AD
Serial Number: 27542E26
Asset Tag: 01131221
Part Number: HMT31GR7CFR4A-H9
Rank: 2

DMICODE Memory: Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB

I should add to above comment that the /sys/devices/system/memory/ folder does not exist.

Thanks
Nick

Nick - I have to admit, I'm a bit stumped. To be fair, I have never used a "xen" kernel (I'm a KVM/RHEV guy myself). The XEN kernel may alter the OS in a way that my troubleshooting might not be the best method. I apologize we haven't come to a solution/conclusion yet.

I just checked my R610 with the same memory 12x8GB (which your OS sees from the dmidecode output). However, my bios is still a 2.x release.

The fact that /sys/devices/system/memory/memory*/state is missing is bizarre to me.

Do you have 2 processors installed in that system? I would be surprised that dmidecode could "see" the memory, but not access it - if it was due to only a single processor configuration. Even still - I would expect top or free to show 48GB, not 32GB.

Another thing to consider - I know you mentioned you tried a few different values - have you tried using mem=98304M - which is 8192x12 (I'm grasping at straws here) or remove it altogether. I did not need to specify my memory by hand. If you are able, I would try to boot to the non-xen kernel (just to see if it recognizes the memory).

[root@rhnsat01 ~]# uname -n; cat /etc/redhat-release; dmidecode -s bios-version; dmidecode | grep "Memory Device" | wc -l; dmidecode | grep -A5 "Memory Device" | grep Size
uname -n: Linux rhnsat01.company.com 2.6.18-348.12.1.el5 #1 SMP Mon Jul 1 17:54:12 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
redhat-release: Red Hat Enterprise Linux Server release 5.9 (Tikanga)
bios-version: 2.0.13
mem number: 12
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB
Size: 8192 MB

I think this is important - note how many "memory devices" I have
[root@rhnsat01 ~]# grep online /sys/devices/system/memory/memory*/state | wc -l
774

This is an answer I received in another post(Can anyone verify it?);

No, you are out of luck, at least without upgrading to RHEL 6:
Linux localhost.localdomain 2.6.18-371.1.2.el5xen

The XEN kernel in RHEL5 does only support up to 32G RAM
http://xen.1045712.n5.nabble.com/memory-quesion-td2609396.html

Note, this is a limitation of XEN, not RHEL

Hey Nick - I hate to hijack a thread as I know you are certainly looking for an answer. Do you actually need Xen? Red Hat selected KVM and it's fully supported by them. I believe converting your XEN Virtual Machines to KVM should be fairly straight-forward. I do not know your requirements, but I thought I would offer that as a possible solution. As I admitted previously, I do not know XEN, but I suspected that was the issue.

Your system likely has the non-XEN kernels installed and you can try to boot to them by selecting the other kernel at boot.
You can see if the non-XEN kernels are a.) installed and b.) addressed in your grub
rpm -qa kernel
grep kern /boot/grub/grub.conf |awk '{ print $2 }'

I did chat with Redhat support in the past and they verified my system should support upto 1TB of Memory. According to other users the xen can never handle that much.
what is the right answer?

Hello,

According to Red Hat's knowledgebase article, https://access.redhat.com/site/articles/3976, On RHEL5, the maximum support memory is 512G. The minimum requirement of memory is 512MB. This particular article doesn't mention the xen kernel though. However, you should be able to install the non-xen kernel and then check to see if the extra memory does show up.

sosreport can be uploaded to dropbox.redhat.com in the directory /incoming
add the file-name to the Red Hat case.

Does the DELL post show the new amount of memory?
Does the server have enough CPU's to support this new number of DIMMs?
For if you use AMD processors you have distributed memory management.

Kind regards,

Jan Gerrit Kootstra

James thanks for the information on changing the kernel. You can hijack my thread anytime. The big question is does KVM support more memory than XEN and how much?

Thank
Nick

In almost any case, KVM is the preferable option. To answer your question: 1TB for RHEL 5.

http://www.redhat.com/resourcelibrary/articles/virtualization-limits-rhel-hypervisors

The different hypervisor vendors seem to go back-and-forth regarding which is the best, but you will often find that KVM (and RHEV) supports more memory, more CPUs, more vCPUs, has better benchmarks, etc...

So - a better question: If your physical node is simply a Hypervisor, could you run RHEL 6 instead?