Virtualization limits for Red Hat Enterprise Linux with KVM

Updated -

The following limits apply to Red Hat Enterprise Linux Server with KVM virtualization. Red Hat also offers Red Hat Virtualization, which offers greater scalability and more advanced features. To see limits for Red Hat Virtualization, please see Supported limits for Red Hat Virtualization.

Intel 64 and AMD64:

  Red Hat Enterprise Linux 5 Server with KVM Red Hat Enterprise Linux 6 Server with KVM Red Hat Enterprise Linux 7 Server with KVM Red Hat Enterprise Linux 8 Server with KVM Red Hat Enterprise Linux 9 Server with KVM
Maximum number of virtual CPUs in virtualized guest 16 240 240 710 710
Maximum memory in virtualized guest 512 GB1 4 TB1,2,5 6 TB5 16 TB4,5 16 TB4,5
Minimum memory in virtualized guest 512 MB3 512 MB3 512 MB 512 MB 512 MB

IBM Z:

  Red Hat Enterprise Linux 7 Server with KVM Red Hat Enterprise Linux 8 Server with KVM Red Hat Enterprise Linux 9 Server with KVM
Maximum number of virtual CPUs in virtualized guest 240 2486 2486
Maximum memory in virtualized guest 2 TB5 10 TB5 10 TB5
Minimum memory in virtualized guest 1 GB 1 GB 1 GB

IBM POWER 8 and POWER 9:

  Red Hat Enterprise Linux 7 Server with KVM Red Hat Enterprise Linux 8 Server with KVM Red Hat Enterprise Linux 9 Server with KVM
Maximum number of virtual CPUs in virtualized guest 192 384 Unsupported
Maximum memory in virtualized guest 10 TB5 10 TB5 Unsupported
Minimum memory in virtualized guest 2 GB 2 GB Unsupported

Other support considerations:

  • Supported limits reflect the current state of system testing by Red Hat and its partners. Systems exceeding these supported limits may be included in the Hardware Catalog after joint testing between Red Hat and its partners. If they exceed the supported limits posted here, entries in the Hardware Catalog will be fully supported.
  • In addition to supported limits reflecting hardware capability, there may be additional limits under the Red Hat Enterprise Linux subscription terms. Supported limits are subject to change based on ongoing testing activities.
  • For limits using the Xen hypervisor included with Red Hat Enterprise Linux 5, see Virtualization limits for Red Hat Enterprise Linux with Xen
  • Until 2021 there was a support limit of 4 active guests per RHEL hypervisor. This limit has been removed so more than 4 guests are supported.

Footnotes:

  1. Supports the maximum supported memory in the host, all of which may be allocated to the virtualized guest. 32-bit guests with Physical Address Extension (PAE) support will only be able to access 64 GB. This is a virtual hardware limitation.
  2. For Red Hat Enterprise Linux 6.2 and earlier, the maximum memory in a virtual machine is 512 GB. For Red Hat Enterprise Linux 6.3, the maximum memory in a virtual machine is 2000 GB.
  3. This is the recommended limit in text mode. The graphical mode requires 640 MB.
  4. Note that virtualized guests larger than 8 TB currently require explicit virtual NUMA configuration, because the maximum virtual NUMA node size is 8 TB.
  5. Live migrating guests with more than 1 TB of memory may have certain limitations. For details see the RHEL 8 or RHEL 9 documentation, based on your version of RHEL.
  6. If the VM is running in secure execution mode, the vCPU limit on IBM Z is only 247.
  • Component
  • kvm

19 Comments

Maximum number of concurrently running virtual guests => does this imply that a RHEL7 system with libvirt/kvm/qemu installed will only run 4 VMs? What does concurrent imply here?

It means that at a given point in time you can only have 4 VMs in running state in RHEL KVM. But you can have as many as you want defined and down. This limitation does not apply to RHV, RHOSP. Also note this is a supportability constraint, the software allows you to run more.

"This guest limit does not apply to Red Hat Enterprise Linux with Unlimited Guests. There is no guest limit for Red Hat Enterprise Virtualization, Red Hat Enterprise Linux OpenStack Platform, and the Red Hat Enterprise Linux with Smart Virtualization bundle."

What about Red Hat Enterprise Linux for Virtual Datacenters + Red Hat Enterprise Linux subscriptions in a server bare metal? Is there no guest limit too?

I am a bit confused about the limit for concurrent guests for RHEL 8+KVM.

This was not properly announced, and did not exist for RHEL7+KVM.

In this article it is not mentioned, I only found it in the reports of Insights

Hi Jan.

The 4 guest limit was present for a long time, including RHEL 7. It was mentioned on this article.

But this being now removed, this article was updated a few days ago removing it. The limitation no longer exists. Insights and RHEL8 docs should be updated soon as well, we are tracking both internally.

Thanks

Hi Germano,

If I am not mistaken, it is a limit enforced by the SKU you buy and not by the operating system.

The 100 guests per hosts limit of VDC has been removed in the last few years too.

Regards,

Jan Gerrit

Hi Jan,

Indeed, the software itself did not enforce the limit at any point.

Thanks

Hi Germano,

With this comment, my interpretation is that customers are free to have more than 4 concurrent KVM guests running at the same time. The documentation will be updated in due course. Is that correct?

Yes, there is no limit anymore. We should have all docs updated by now, where are you seeing the 4 limit referenced?

Thanks, Germano. Truth be told, I am unable to find it in our documentations. This makes it very clear on the other hand.

Thanks Sean, I was afraid we may have left some note behind. There were notes in the documentation, but they were removed when this change. Thanks for confirming!

Hi, one of our customers now in benchmark and looking to choose alternatives between KVM and VMware. They have Superdome server with 448 Cores. They wants to be able to use all of them and now they achieved max of 384. Can we provide some solution or customer going to chose VMware as Hypervisor

Anoel, for that we need basically 2 things. a) An understanding if that technically can work/is supported from Linux/RHEL, I think that's the case. b) We need HPE to do EET testing with our departments. That means HPE gives us access to a config with 448 cores (or more), and with appropriate amount of memory, and then Red Hat engineering tests the system and can declare 448 cores on that hardware supported. I do not see that currently for HPE superdome in our hardware certification page (but maybe I just searched incorrectly). The customer should approach HPE about this.

@christian - alreay done. at 1792 CUs and 48TB RAM for RHEL 7 and 8 and the limits have been raised to that level in RHEL 9. see https://access.redhat.com/articles/rhel-limits

Monte, good to know, thanks! I just performed my search a second time, but still do not see the result of the EET.. I access https://catalog.redhat.com/hardware/search?p=1&c_catalog_channel=Server, then search for 'superdome'. I open each of the 6 found detail pages, but none seems to have a kcs attached which notifies about the EET.

Hi , I see that in article "Maximum number of virtual CPUs in virtualized guest" in Intel based Machines increased from 384 up to 710. Is it means that we as Red Hat checked and updated the support to max 710 vCPU? Can someone confirm? and if yes, are some special definition should be done to enable 448 vCPU or it will be done by default? And additional question - are customer need to upgrade to specific 8.x version or it is only configuration/definition? Thanks

You could lookup the name of the colleague who modified this in 'version history' and contact if you are in doubt for some reason. There are no footnotes on technical extra requirements, so 710 should be supported without further steps. Especially, with this change no EET is required to support the 710vcpus. We recommend though to use hardware which passed hardware certification, looking up that certification page for the system in question can make sense - sometimes kbases with extra hints are attached. Question if a minor version is required is a good one: I would assume that the 710 applies to rhel8.6GA. I suspect we did not rise that limit and tested with 8.0 or 8.2 kernels. We should clarify this in a footnote.

hi, is there no limitation on the storage per VM?

There's no limit on storage, block device sizes which are supported when hooked to the host are also expected to not cause issues when tied to guests.