Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

Virtualization Security Guide

Red Hat Enterprise Linux 6

Securing your virtual environment

Jiri Herrmann

Red Hat Customer Content Services

Yehuda Zimmerman

Red Hat Customer Content Services

Scott Radvan

Red Hat Customer Content Services

Tahlia Richardson

Red Hat Customer Content Services

Thanks go to the following people for enabling the creation of this guide:

Paul Moore

Red Hat Engineering

Kurt Seifried

Red Hat Engineering

David Jorm

Red Hat Engineering

Abstract

This guide provides an overview of virtualization security technologies provided by Red Hat. It also provides recommendations for securing hosts, guests, and shared infrastructure and resources in virtualized environments.

Chapter 1. Introduction

1.1. Virtualized and Non-Virtualized Environments

A virtualized environment presents opportunities for both the discovery of new attack vectors and the refinement of existing exploits that may not previously have presented value to an attacker. It is therefore important to take steps to ensure the security of both the physical hosts and the guests running on them when creating and maintaining virtual machines.
Non-Virtualized Environment

In a non-virtualized environment, hosts are separated from each other physically and each host has a self-contained environment, consisting of services such as a web server, or a DNS server. These services communicate directly to their own user space, host kernel and physical host, offering their services directly to the network. The following image represents a non-virtualized environment:

Non-Virtualized Environment

Figure 1.1. Non-Virtualized Environment

Virtualized Environment

In a virtualized environment, several operating systems can be housed (as "guests") within a single host kernel and physical host. The following image represents a virtualized environment:

Virtualized Environment

Figure 1.2. Virtualized Environment

When services are not virtualized, machines are physically separated. Any exploit is therefore usually contained to the affected machine, with the obvious exception of network attacks. When services are grouped together in a virtualized environment, extra vulnerabilities emerge in the system. If there is a security flaw in the hypervisor that can be exploited by a guest instance, this guest may be able to not only attack the host, but also other guests running on that host. This is not theoretical; attacks already exist on hypervisors. These attacks can extend beyond the guest instance and could expose other guests to attack.

1.2. Why Virtualization Security Matters

Deploying virtualization in your infrastructure provides many benefits but can also introduce new risks. Virtualized resources and services should be deployed with the following security considerations:
  • The host/hypervisor become prime targets; they are often a single point of failure for guests and data.
  • Virtual machines can interfere with each other in undesirable ways. Assuming no access controls were in place to help prevent this, one malicious guest could bypass a vulnerable hypervisor and directly access other resources on the host system, such as the storage of other guests.
  • Resources and services can become difficult to track and maintain; with rapid deployment of virtualized systems comes an increased need for management of resources, including sufficient patching, monitoring and maintenance.
  • Technical staff may lack knowledge, have gaps in skill sets, and have minimal experience in virtual environments. This is often a gateway to vulnerabilities.
  • Resources such as storage can be spread across, and dependent upon, several machines. This can lead to overly complex environments, and poorly-managed and maintained systems.
  • Virtualization does not remove any of the traditional security risks present in your environment; the entire solution stack, not just the virtualization layer, must be secured.
This guide aims to assist you in mitigating your security risks by offering a number of virtualization recommended practices for Red Hat Enterprise Linux that will help you secure your virtualized infrastructure.

1.3. Leveraging SELinux with sVirt

sVirt integrates virtualization into the existing security framework provided by SELinux (Security-Enhanced Linux), applying Mandatory Access Control (MAC) to virtual machines. The main objective of sVirt is to protect hosts and guests from attacks via security vulnerabilities in the hypervisor. SELinux secures a system by applying access policy across different processes. sVirt extends this capability to hosts and guests by treating each guest as a process, allowing administrators to apply similar policies designed to prevent malicious guests from accessing restricted resources. For more information on sVirt, refer to Chapter 4, sVirt.

Chapter 2. Host Security

2.1. Why Host Security Matters

When deploying virtualization technologies, you must ensure that the host physical machine and its operating system cannot be compromised. In this case the host is a Red Hat Enterprise Linux system that manages the system, devices, memory and networks as well as all guest virtual machines. If the host physical machine is insecure, all guest virtual machines in the system are vulnerable. There are several ways to enhance security on systems using virtualization. You or your organization should create a Deployment Plan. This plan needs to contain the following:
  • Operating specifications
  • Specifies which services are needed on your guest virtual machines
  • Specifies the host physical servers as well as what support is required for these services
Here are a few security issues to consider while developing a deployment plan:
  • Run only necessary services on host physical machines. The fewer processes and services running on the host physical machine, the higher the level of security and performance.
  • Enable SELinux on the hypervisor. Read Section 2.1.2, “SELinux and Virtualization” for more information on using SELinux and virtualization.
  • Use a firewall to restrict traffic to the host physical machine. You can setup a firewall with default-reject rules that will help secure the host physical machine from attacks. It is also important to limit network-facing services.
  • Do not allow normal users to access the host operating system. If the host operating system is privileged, granting access to unprivileged accounts may compromise the level of security.

2.1.1. Security Concerns when Adding Block Devices to a Guest

When using host block devices, partitions, and logical volumes (LVMs) it is important to follow these guidelines:
  • The host physical machine should not use filesystem labels to identify file systems in the fstab file, the initrd file or on the kernel command line. Doing so presents a security risk if guest virtual machines have write access to whole partitions or LVM volumes, because a guest virtual machine could potentially write a filesystem label belonging to the host physical machine, to its own block device storage. Upon reboot of the host physical machine, the host physical machine could then mistakenly use the guest virtual machine's disk as a system disk, which would compromise the host physical machine system.
    It is preferable to use the UUID of a device to identify it in the fstab file, the initrd file or on the kernel command line. While using UUIDs is still not completely secure on certain file systems, a similar compromise with UUID is significantly less feasible.
  • Guest virtual machines should not be given write access to whole disks or block devices (for example, /dev/sdb). Guest virtual machines with access to whole block devices may be able to modify volume labels, which can be used to compromise the host physical machine system. Use partitions (for example, /dev/sdb1) or LVM volumes to prevent this problem.
    If you are using raw access to partitions, for example /dev/sdb1 or raw disks such as /dev/sdb, you should configure LVM to only scan disks that are safe, using the global_filter setting.

Note

When the guest virtual machine only has access to image files, these issues are not relevant.

2.1.2. SELinux and Virtualization

Security Enhanced Linux was developed by the NSA with assistance from the Linux community to provide stronger security for Linux. SELinux limits an attacker's abilities and works to prevent many common security exploits such as buffer overflow attacks and privilege escalation. It is because of these benefits that all Red Hat Enterprise Linux systems should run with SELinux enabled and in enforcing mode.

Procedure 2.1. Creating and mounting a logical volume on a guest virtual machine with SELinux enabled

  1. Create a logical volume. This example creates a 5 gigabyte logical volume named NewVolumeName on the volume group named volumegroup. This example also assumes that there is enough disk space. You may have to create additional storage on a network device and give the guest access to it. This information is discussed in more detail in the Red Hat Enterprise Linux Virtualization Administration Guide.
    # lvcreate -n NewVolumeName -L 5G volumegroup
  2. Format the NewVolumeName logical volume with a file system that supports extended attributes, such as ext3.
    # mke2fs -j /dev/volumegroup/NewVolumeName
  3. Create a new directory for mounting the new logical volume. This directory can be anywhere on your file system. It is advised not to put it in important system directories (/etc, /var, /sys) or in home directories (/home or /root). This example uses a directory called /virtstorage
    # mkdir /virtstorage
  4. Mount the logical volume.
    # mount /dev/volumegroup/NewVolumeName /virtstorage
  5. Set the SELinux type for the folder you just created.
    # semanage fcontext -a -t virt_image_t "/virtstorage(/.*)?"
    
    If the targeted policy is used (targeted is the default policy) the command appends a line to the /etc/selinux/targeted/contexts/files/file_contexts.local file which makes the change persistent. The appended line may resemble this:
    /virtstorage(/.*)?    system_u:object_r:virt_image_t:s0
    
  6. Run the command to change the type of the mount point (/virtstorage) and all files under it to virt_image_t (the restorecon and setfiles commands read the files in /etc/selinux/targeted/contexts/files/).
    # restorecon -R -v /virtstorage
    

Note

Create a new file (using the touch command) on the file system.
# touch /virtstorage/newfile
Verify the file has been relabeled using the following command:
# sudo ls -Z /virtstorage
-rw-------. root root system_u:object_r:virt_image_t:s0 newfile
The output shows that the new file has the correct attribute, virt_image_t.

2.1.3. SELinux

This section contains topics to consider when using SELinux with your virtualization deployment. When you deploy system changes or add devices, you must update your SELinux policy accordingly. To configure an LVM volume for a guest virtual machine, you must modify the SELinux context for the respective underlying block device and volume group. Make sure that you have installed the policycoreutils-python package (yum install policycoreutils-python) before running the command.
# semanage fcontext -a -t virt_image_t -f -b /dev/sda2
# restorecon /dev/sda2
KVM and SELinux

The following table shows the SELinux Booleans which affect KVM when launched by libvirt.

KVM SELinux Booleans
SELinux BooleanDescription
virt_use_commAllow virt to use serial/parallel communication ports.
virt_use_fusefsAllow virt to read fuse files.
virt_use_nfsAllow virt to manage NFS files.
virt_use_sambaAllow virt to manage CIFS files.
virt_use_sanlockAllow sanlock to manage virt lib files.
virt_use_sysfsAllow virt to manage device configuration (PCI).
virt_use_xserverAllow virtual machine to interact with the xserver.
virt_use_usbAllow virt to use USB devices.

2.1.4. Virtualization Firewall Information

Various ports are used for communication between guest virtual machines and corresponding management utilities.

Note

Any network service on a guest virtual machine must have the applicable ports open on the guest virtual machine to allow external access. If a network service on a guest virtual machine is firewalled it will be inaccessible. Always verify the guest virtual machine's network configuration first.
  • ICMP requests must be accepted. ICMP packets are used for network testing. You cannot ping guest virtual machines if the ICMP packets are blocked.
  • Port 22 should be open for SSH access and the initial installation.
  • Ports 80 or 443 (depending on the security settings on the RHEV Manager) are used by the vdsm-reg service to communicate information about the host physical machine.
  • Ports 5634 to 6166 are used for guest virtual machine console access with the SPICE protocol.
  • Ports 49152 to 49216 are used for migrations with KVM. Migration may use any port in this range depending on the number of concurrent migrations occurring.
  • Enabling IP forwarding (net.ipv4.ip_forward = 1) is also required for shared bridges and the default bridge. Note that installing libvirt enables this variable so it will be enabled when the virtualization packages are installed unless it was manually disabled.

Note

Note that enabling IP forwarding is not required for physical bridge devices. When a guest virtual machine is connected through a physical bridge, traffic only operates at a level that does not require IP configuration such as IP forwarding.

Chapter 3. Guest Security

3.1. Why Guest Security Matters

While the security of the host system is critical in ensuring the security of the guests running on the host, it does not remove the need for properly securing the individual guest machines. All of the security risks associated with a conventional, non-virtualized system still exist when the system is run as a virtualized guest. Any resources accessible to the guest system, such as critical business data or sensitive customer information, could be made vulnerable if the guest system were to be compromised.

Chapter 4. sVirt

4.1. Introduction

Since virtual machines under KVM are implemented as Linux processes, KVM leverages the standard Linux security model to provide isolation and resource controls. The Linux kernel includes SELinux (Security-Enhanced Linux), a project developed by the US National Security Agency to add mandatory access control (MAC), multi-level security (MLS) and multi-category security (MCS) through a flexible and customizable security policy. SELinux provides strict resource isolation and confinement for processes running on top of the Linux kernel, including virtual machine processes. The sVirt project builds upon SELinux to further facilitate virtual machine isolation and controlled sharing. For example, fine-grained permissions could be applied to group virtual machines together to share resources.
From a security point of view, the hypervisor is a tempting target for attackers, as a compromised hypervisor could lead to the compromise of all virtual machines running on the host system. Integrating SELinux into virtualization technologies helps improve hypervisor security against malicious virtual machines trying to gain access to the host system or other virtual machines.
Refer to the following image which represents isolated guests, limiting the ability for a compromised hypervisor (or guest) to launch further attacks, or to extend to another instance:
Attack path isolated by SELinux

Figure 4.1. Attack path isolated by SELinux

Note

For more information on SELinux, refer to Red Hat Enterprise Linux Security-Enhanced Linux.

4.2. SELinux and Mandatory Access Control (MAC)

Security-Enhanced Linux (SELinux) is an implementation of MAC in the Linux kernel, checking for allowed operations after standard discretionary access controls (DAC) are checked. SELinux can enforce a user-customizable security policy on running processes and their actions, including attempts to access file system objects. Enabled by default in Red Hat Enterprise Linux, SELinux limits the scope of potential damage that can result from the exploitation of vulnerabilities in applications and system services, such as the hypervisor.
sVirt integrates with libvirt, a virtualization management abstraction layer, to provide a MAC framework for virtual machines. This architecture allows all virtualization platforms supported by libvirt and all MAC implementations supported by sVirt to interoperate.

4.3. sVirt Configuration

SELinux Booleans are variables that can be toggled on or off, quickly enabling or disabling features or other special conditions. Booleans can be toggled by running either setsebool boolean_name {on|off} for a temporary change, or setsebool -P boolean_name {on|off} to make the change persistent across reboots.
The following table shows the SELinux Boolean values that affect KVM when launched by libvirt. The current state of these booleans (on or off) can be found by running the command getsebool -a|grep virt.

Table 4.1. KVM SELinux Booleans

SELinux BooleanDescription
staff_use_svirtAllow staff user to create and transition to sVirt domains.
unprivuser_use_svirtAllow unprivileged user to create and transition to sVirt domains.
virt_sandbox_use_auditAllow sandbox containers to send audit messages.
virt_sandbox_use_netlinkAllow sandbox containers to use netlink system calls.
virt_sandbox_use_sys_adminAllow sandbox containers to use sys_admin system calls, such as mount.
virt_transition_userdomainAllow virtual processes to run as user domains.
virt_use_commAllow virt to use serial/parallel communication ports.
virt_use_execmemAllow confined virtual guests to use executable memory and executable stack.
virt_use_fusefsAllow virt to read FUSE mounted files.
virt_use_nfsAllow virt to manage NFS mounted files.
virt_use_rawipAllow virt to interact with rawip sockets.
virt_use_sambaAllow virt to manage CIFS mounted files.
virt_use_sanlockAllow confined virtual guests to interact with the sanlock.
virt_use_usbAllow virt to use USB devices.
virt_use_xserverAllow virtual machine to interact with the X Window System.

Note

For more information on SELinux Booleans, refer to Red Hat Enterprise Linux Security Enhanced Linux.

4.4. sVirt Labeling

Like other services under the protection of SELinux, sVirt uses process based mechanisms, labels and restrictions to provide extra security and control over guest instances. Labels are applied automatically to resources on the system based on the currently running virtual machines (dynamic), but can also be manually specified by the administrator (static), to meet any specific requirements that may exist.

4.4.1. Types of sVirt Labels

The following table outlines the different sVirt labels that can be assigned to resources such as virtual machine processes, image files and shared content:

Table 4.2. sVirt Labels

TypeSELinux ContextDescription/Effect
Virtual Machine Processessystem_u:system_r:svirt_t:MCS1MCS1 is a randomly selected field. Currently approximately 500,000 labels are supported.
Virtual Machine Imagesystem_u:object_r:svirt_image_t:MCS1Only svirt_t processes with the same MCS1 fields are able to read/write these image files and devices.
Virtual Machine Shared Read/Write Contentsystem_u:object_r:svirt_image_t:s0All svirt_t processes are allowed to write to the svirt_image_t:s0 files and devices.
Virtual Machine Shared Shared Read Only contentsystem_u:object_r:svirt_content_t:s0All svirt_t processes are able to read files/devices with this label.
Virtual Machine Imagesystem_u:object_r:virt_content_t:s0System default label used when an image exits. No svirt_t virtual processes are allowed to read files/devices with this label.

4.4.2. Dynamic Configuration

Dynamic label configuration is the default labeling option when using sVirt with SELinux. Refer to the following example which demonstrates dynamic labeling:
# ps -eZ | grep qemu-kvm

system_u:system_r:svirt_t:s0:c87,c520 27950 ?  00:00:17 qemu-kvm
In this example, the qemu-kvm process has a base label of system_u:system_r:svirt_t:s0. The libvirt system has generated a unique MCS label of c87,c520 for this process. The base label and the MCS label are combined to form the complete security label for the process. Likewise, libvirt takes the same MCS label and base label to form the image label. This image label is then automatically applied to all host files that the VM is required to access, such as disk images, disk devices, PCI devices, USB devices, and kernel/initrd files. Each process is isolated from other virtual machines with different labels.
The following example shows the virtual machine's unique security label (with a corresponding MCS label of c87,c520 in this case) as applied to the guest disk image file in /var/lib/libvirt/images:
# ls -lZ /var/lib/libvirt/images/*

  system_u:object_r:svirt_image_t:s0:c87,c520   image1
The following example shows dynamic labeling in the XML configuration for the guest:
<seclabel type='dynamic' model='selinux' relabel='yes'>
  <label>system_u:system_r:svirt_t:s0:c87,c520</label>
  <imagelabel>system_u:object_r:svirt_image_t:s0:c87,c520</imagelabel>
</seclabel>

4.4.3. Dynamic Configuration with Base Labeling

To override the default base security label in dynamic mode, the <baselabel> option can be configured manually in the XML guest configuration, as shown in this example:
<seclabel type='dynamic' model='selinux' relabel='yes'>
  <baselabel>system_u:system_r:svirt_custom_t:s0</baselabel>
  <label>system_u:system_r:svirt_custom_t:s0:c87,c520</label>
  <imagelabel>system_u:object_r:svirt_image_t:s0:c87,c520</imagelabel>
</seclabel>

4.4.4. Static Configuration with Dynamic Resource Labeling

Some applications require full control over the generation of security labels but still require libvirt to take care of resource labeling. The following guest XML configuration demonstrates an example of static configuration with dynamic resource labeling:
<seclabel type='static' model='selinux' relabel='yes'>
  <label>system_u:system_r:svirt_custom_t:s0:c87,c520</label>
</seclabel>

4.4.5. Static Configuration without Resource Labeling

Primarily used in MLS (multi-level security) or otherwise strictly controlled environments, static configuration without resource relabeling is possible. Static labels allow the administrator to select a specific label, including the MCS/MLS field, for a virtual machine. Administrators who run statically-labeled virtual machines are responsible for setting the correct label on the image files. The virtual machine will always be started with that label, and the sVirt system will never modify the label of a statically-labelled virtual machine's content. The following guest XML configuration demonstrates an example of this scenario:
<seclabel type='static' model='selinux' relabel='no'>
  <label>system_u:system_r:svirt_custom_t:s0:c87,c520</label>
</seclabel>

Chapter 5. Network Security in a Virtualized Environment

5.1. Network Security Overview

In almost all situations, the network is the only way to access systems, applications and management interfaces. As networking plays such a critical role in the management of virtualized systems and the availability of their hosted applications, it is very important to ensure that the network channels both to and from the virtualized systems are secure.
Securing the network allows administrators to control access and protect sensitive data from information leaks and tampering.

Appendix A. Further Information

A.1. SELinux and sVirt

Further information on SELinux and sVirt:

A.2. Virtualization Security

Further information on virtualization security:

Appendix B. Revision History

Revision History
Revision 0.4-33Wed Mar 08 2017Jiri Herrmann
Prepared the book for the 6.9 GA release
Revision 0.4-31Mon Dec 20 2016Jiri Herrmann
Prepared the book for the 6.9 Beta release
Revision 0.4-30Mon May 10 2016Jiri Herrmann
Prepared the book for the 6.8 GA release
Revision 0.4-23Tue Mar 01 2016Jiri Herrmann
Prepared the book for the 6.8 beta release
Revision 0.4-22Thu Oct 08 2015Jiri Herrmann
Cleaned up the Revision History
Revision 0.4-21Fri Oct 10 2014Scott Radvan
Version for 6.6 GA release.

Legal Notice

Copyright © 2016 Red Hat, Inc.
This document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.