Supported Limits for Red Hat Virtualization
Table of Contents
These maximums and minimums are based on performance testing of Red Hat Virtualization (RHV) 4.x and Red Hat Enterprise Virtualization (RHEV) 3.x. as a large-scale environment.
To operate Red Hat Virtualization as a large-scale environment, apply the recommendations in Guidelines for Large Deployments of Red Hat Virtualization.
Red Hat Virtualization 4.x Limits
The following limits apply to Red Hat Virtualization (RHV) 4.x
Virtual Machine Maximums:
- Maximum concurrently running virtual machines: Unlimited
- Maximum virtual CPUs per virtual machine:
- 288 for RHV 4.0-4.1 (288 for Q35 machines, 240 for PC machines)
- 384 for RHV 4.2-4.3 (384 for Q35 machines, 240 for PC machines)
- 710 for RHV 4.4 (710 for Q35 machines, 240 for PC machines)
- Maximum memory per virtual machine:
- 4 TB1, 2 for RHV 4.3
- 16 TB1, 2,3 for RHV 4.4, including virtual machines with cluster level 4.3 in RHV 4.4.
- Minimum memory per virtual machine: N/A
- Maximum single disk size per virtual machine: 8 TB
1. Supports the maximum memory per VM in the host, all of which you can allocate to the virtual machine. 32-bit guests with Physical Address Extension (PAE) support are only able to access 64 GB. This is a virtual hardware limitation.
2. Subject to the limits specified in Virtualization limits for Red Hat Enterprise Linux with KVM.
3. Note that virtualized guests larger than 8 TB currently require explicit virtual NUMA configuration, because the maximum virtual NUMA node size is 8 TB.
The following maximums apply to the Red Hat Virtualization Host (RHVH) and Red Hat Enterprise Linux host.
- Logical CPU cores or threads: 768
- RAM: 12 TB
- Simultaneous live migrations: 2 incoming, 2 outgoing (more)
- Live migration bandwidth: Default to 52 MiB (~436 Mb) per migration when using the legacy migration policy. Other policies use adaptive throughput values based on the speed of the physical device. QoS policies can limit migration bandwidth.
Self-hosted Engine (SHE) Maximums:
- Number of nodes: 7
RHV Manager Logical Entity Maximums:
- Data center
- Maximum concurrently-running virtual machines on a single RHV Manager with multiple datacenters: 4000 supported
- Maximum data center count: 400
- Maximum host count: 250 per data center
- Maximum VM count: 4000 supported
- Maximum cluster count: 400 (e.g. 400 clusters in one data center, or one cluster on each of 400 data centers)
- Maximum VM count: 4000 supported
- Maximum 200 networks per single host
- Maximum 300 networks per cluster
- Maximum total number of networks: 100 networks on 150 hosts, or 15000 total networks (or 60 networks per cluster with 250 hosts)
- Maximum domains per data center: 50 supported. However, each additional storage domain can cause some performance degradation. So use as many storage domains as you need, up to 50, but avoid using any more than you need.
- Hosts per domain: No limit
- Logical volumes per block domain (more): 1500
- Maximum number of LUNs (more): 300
- Maximum disk size: 500 TiB (limited to 8 TiB by default)
- Hosts: 400
- Hosts per data center: 250
- Virtual Machines (VMs)
- VMs: 4000
- The total number of VMs depends on the host hardware and resource needs
- Storage Pool Manager (SPM) concurrent activities, such as metadata operations and storage live migration: 10
- Storage data operations run on every host, up to 10 operations per host. Example operations are copying or moving disks from one storage domain to another storage domain.
- In RHV 4.3, some data operations still run on the SPM.
- In RHV 4.4, we plan to move all data operations to run on any host.
Red Hat Enterprise Virtualization 3.x
NOTE More detailed maximums for RHEV 3.x are not available.
- Maximum number of concurrently running virtual machines: Unlimited
- Maximum number of virtual CPUs per virtual machine: 160
- Maximum memory per virtual machine: 2 TB1
- Minimum memory per virtual machine: 512 MB
1. Supports the maximum memory per VM in the host, all of you can allocate to the virtual machine. 32-bit guests with Physical Address Extension (PAE) support are only able to access 64 GB. This is a virtual hardware limitation.
The following notes apply to all versions of Red Hat Virtualization.
- Supported limits reflect the current state of system testing by Red Hat and its partners. Systems exceeding these supported limits may be included in the Hardware Catalog after joint testing between Red Hat and its partners. If they exceed the supported limits posted here, entries in the Hardware Catalog are fully supported. In addition to supported limits reflecting hardware capability, there may be additional limits under the Red Hat Enterprise Linux subscription terms. Supported limits are subject to change based on ongoing testing activities.
- These limits do not apply to Red Hat Enterprise Linux (RHEL) with KVM virtualization, which offers virtualization for low-density environments. Instead, see "Virtualization limits for RHEL with KVM".
- Guest operating systems have different minimum memory requirements. Virtual machine memory can be allocated as small as required.
- More specific limitations can be found in the "Requirements" topic of the RHV "Planning and Prerequisites Guide: Requirements."
Related articles and documentation
- Recommended sizes and technical limitations of RHEV storage domains (For RHV 4.1 and earlier)
- Understanding live migration "migration_max_bandwidth" and "max_outgoing_migrations" parameters in vdsm.conf (For RHV 4.1 and earlier to RHEV 3.x)
- RHV Planning and Prerequisites Guide: Requirements
- Red Hat Virtualization
- Article Type
Can this article be improved (or another article created) to provide documentation around the physical maximums? For example, how many hosts per cluster are supported? How many clusters per datacenter? How many CPU / host? How much RAM / host? How many LUNS (FC/iSCSI) per host?
Many of those limits are reported in the Technical Guide, for example, for 3.5 you can find it at https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.5/html-single/Technical_Guide/index.html#Minimum_requirements_and_supported_limits
Where it lists limits of ram, storage, etc.
I think this is not correct documentation --> "A maximum of 2 TB of RAM is supported."
Please update this KB with the RHEV 4 Limits.
Please update this KB with the RHEV 4 Limits.
There is a new document for RHV 4.1 with comprehensive information: Red Hat Virtualization Manager Limitations At the moment when I am writing it is still not finished but RH is conducting very thorough performance/limitation tests. When they finish, it will be the most accurate and precise document.
max vCPUs is not accurate. RHV 4.2 has a limit of 288, and 4.2-4.3 has 384
There is a typo: RHV 4.1 has a limit of 288, and RHV 4.2-4.3 has 384, right?
For the RHV 4.x Manager Storage Limits: I would guess the limits of storage domain are per datacenter (otherwise it would not make sense to have 400 datacenters). Same goes for the Maximum number of LUNs, I guess that's per storage domain and not a global limit as stated currently.
Additional question: how do direct LUNs behave? Is there a limit on those?
Can I have 400 DC and each having 4000 VMs? I there any clear documentation on these limits I can refer to?
I believe this document require a revisit from Red Hat team to reword it better for the community to understand. RHV- Manager maximums Data center Maximum Cluster Maximum Host maximum
The way I interpret the document is
400 data center with 400 cluster/DC with 400 hosts/Cluster with 4000 VM/cluster
Total of 1,60,000 - clusters 6,40,00,000 - Hosts 2,56,00,00,00,000 - VMs
Just for the correct spacing:
l'm guessing, but running 256 billion VMs can't be the right interpretation :)
They changed the wording @Sivakumar Velusamy
"*Maximum concurrently-running virtual machines on a single RHV Manager with multiple datacenters: 4000 supported, 5000 tested"
Maximum host count: 256 per data center should be 250 not 256.
Thanks, Mordechai. I've updated the text accordingly.