Chapter 11. Provisioning Virtual Machines in VMware vSphere

VMware vSphere is an enterprise-level virtualization platform from VMware. Red Hat Satellite 6 can interact with the vSphere platform, including creating new virtual machines and controlling their power management states.

11.1. Prerequisites for VMware vSphere Provisioning

The requirements for VMware vSphere provisioning include:

  • A Capsule Server managing a network on the vSphere environment. Ensure no other DHCP services run on this network to avoid conflicts with the Capsule Server. For more information, see Chapter 3, Configuring Networking.
  • An existing VMware template if you want to use image-based provisioning.
  • Synchronized content repositories for Red Hat Enterprise Linux. For more information, see Synchronizing Red Hat Repositories in the Content Management Guide.
  • An activation key for host registration. For more information, see Creating An Activation Key in the Content Management guide.

11.2. Creating a VMware vSphere User

The VMware vSphere server requires an administration-like user for Satellite Server communication. For security reasons, do not use the administrator user for such communication. Instead, create a user with the following permissions:

For VMware vCenter Server version 6.7, set the following permissions:

  • All Privileges → Datastore → Allocate Space, Browse datastore, Update Virtual Machine files, Low level file operations
  • All Privileges → Network → Assign Network
  • All Privileges → Resource → Assign virtual machine to resource pool
  • All Privileges → Virtual Machine → Change Config (All)
  • All Privileges → Virtual Machine → Interaction (All)
  • All Privileges → Virtual Machine → Edit Inventory (All)
  • All Privileges → Virtual Machine → Provisioning (All)

For VMware vCenter Server version 6.5, set the following permissions:

  • All Privileges → Datastore → Allocate Space, Browse datastore, Update Virtual Machine files, Low level file operations
  • All Privileges → Network → Assign Network
  • All Privileges → Resource → Assign virtual machine to resource pool
  • All Privileges → Virtual Machine → Configuration (All)
  • All Privileges → Virtual Machine → Interaction (All)
  • All Privileges → Virtual Machine → Inventory (All)
  • All Privileges → Virtual Machine → Provisioning (All)

11.3. Adding a VMware vSphere Connection to Satellite Server

Use this procedure to add a VMware vSphere connection in Satellite Server’s compute resources.

Ensure that the host and network-based firewalls are configured to allow Satellite to vCenter communication on TCP port 443. Verify that Satellite is able to resolve the host name of vCenter and vCenter is able to resolve Satellite Server’s host name.

Procedure

To add a connection, complete the following steps:

  1. In the Satellite web UI, navigate to Infrastructure > Compute Resources, and in the Compute Resources window, click Create Compute Resource.
  2. In the Name field, enter a name for the resource.
  3. From the Provider list, select VMware.
  4. In the Description field, enter a description for the resource.
  5. In the VCenter/Server field, enter the IP address or host name of the vCenter server.
  6. In the User field, enter the user name with permission to access the vCenter’s resources.
  7. In the Password field, enter the password for the user.
  8. Click Load Datacenters to populate the list of data centers from your VMware vSphere environment.
  9. From the Datacenter list, select a specific data center to manage from this list.
  10. In the Fingerprint field, ensure that this field is populated with the fingerprint from the data center.
  11. From the Display Type list, select a console type, for example, VNC or VMRC. Note that VNC consoles are unsupported on VMware ESXi 6.5 and later.
  12. Optional: In the VNC Console Passwords field, select the Set a randomly generated password on the display connection check box to secure console access for new hosts with a randomly generated password. You can retrieve the password for the VNC console to access guest virtual machine console from the libvirtd host from the output of the following command:

    # virsh edit your_VM_name
    <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0' passwd='your_randomly_generated_password'>

    The password randomly generates every time the console for the virtual machine opens, for example, with virt-manager.

  13. From the Enable Caching list, you can select whether to enable caching of compute resources. For more information, see Section 11.8, “Caching of Compute Resources”.
  14. Click the Locations and Organizations tabs and verify that the values are automatically set to your current context. You can also add additional contexts.
  15. Click Submit to save the connection.

For CLI Users

Create the connection with the hammer compute-resource create command. Select Vmware as the --provider and set the instance UUID of the data center as the --uuid:

# hammer compute-resource create --name "My_vSphere" \
--provider "Vmware" \
--description "vSphere server at vsphere.example.com" \
--server "vsphere.example.com" --user "My_User" \
--password "My_Password" --locations "My_Location" --organizations "My_Organization" \
--datacenter "My_Datacenter"

11.4. Adding VMware vSphere Images to Satellite Server

VMware vSphere uses templates as images for creating new virtual machines. If using image-based provisioning to create new hosts, you need to add VMware template details to your Satellite Server. This includes access details and the template name.

Procedure

To add an image, complete the following steps:

  1. In the Satellite web UI, navigate to Infrastructure > Compute Resources and in the Compute Resources window, click the VMware vSphere connection.
  2. In the Name field, enter a name for the image.
  3. From the Operatingsystem list, select the image’s base operating system.
  4. From the Architecture list, select the operating system architecture.
  5. In the User field, enter the SSH user name for image access. This is normally the root user.
  6. In the Password field, enter the SSH password for image access.
  7. From the User data list, select whether you want the images to support user data input, such as cloud-init data.
  8. In the Image field, enter the relative path and name of the template on the vSphere environment. Do not include the data center in the relative path.
  9. Click Submit to save the image details.

For CLI Users

Create the image with the hammer compute-resource image create command. Use the --uuid field to store the relative template path on the vSphere environment.

# hammer compute-resource image create --name "Test_vSphere_Image" \
--operatingsystem "RedHat 7.2" --architecture "x86_64" \
--username root --uuid "Templates/RHEL72" \
--compute-resource "My_vSphere"

11.5. Adding VMware vSphere Details to a Compute Profile

You can predefine certain hardware settings for virtual machines on VMware vSphere. You achieve this through adding these hardware settings to a compute profile.

Procedure

To add VMware vSphere details to a compute profile, complete the following steps:

  1. In the Satellite web UI, navigate to Infrastructure > Compute Profiles and, in the Compute Profiles window, click the name of the compute profile, and then click the vSphere connection.
  2. In the CPUs field, enter the number of CPUs to allocate to the new host.
  3. In the Cores per socket field, enter the number of cores to allocate to each CPU.
  4. In the Memory field, enter the amount of memory to allocate to the new host.
  5. In the Cluster field, enter the name of the target host cluster on the VMware environment.
  6. From the Resource pool list, select an available resource allocations for the host.
  7. In the Folder field, enter the folder to organize the host.
  8. From the Guest OS list, select the operating system you want to use in VMware vSphere.
  9. From the SCSI controller list, select the disk access method for the host.
  10. From the Virtual H/W version list, select the underlying VMware hardware abstraction to use for virtual machines.
  11. You can select the Memory hot add or CPU hot add check boxes if you want to add more resources while the virtual machine is powered.
  12. From the Image list, select the image to use if performing image-based provisioning.
  13. From the Network Interfaces list, select the network parameters for the host’s network interface. You can create multiple network interfaces. However, at least one interface must point to a Capsule-managed network.
  14. Select the Eager zero check box if you want to use eager zero thick provisioning. If unchecked, the disk uses lazy zero thick provisioning.
  15. Click Submit to save the compute profile.

For CLI Users

  1. To create the compute profile, enter the following command:

    # hammer compute-profile create --name "VMWare CP"
  2. To add the values for the compute profile, enter the following command:

    # hammer compute-profile values create --compute-profile "VMWare CP" \
    --compute-resource "My_vSphere" \
    --interface "compute_type=VirtualE1000,compute_network=mynetwork \
    --volume "size_gb=20G,datastore=Data,name=myharddisk,thin=true" \
    --compute-attributes "cpus=1,corespersocket=2,memory_mb=1024,cluster=MyCluster,path=MyVMs,start=true"

11.6. Creating Hosts on a VMware vSphere Server

The VMware vSphere provisioning process provides the option to create hosts over a network connection or using an existing image.

For network-based provisioning, you must create a host to access either Satellite Server’s integrated Capsule or an external Capsule Server on a VMware vSphere virtual network, so that the host has access to PXE provisioning services. The new host entry triggers the VMware vSphere server to create the virtual machine. If the virtual machine detects the defined Capsule Server through the virtual network, the virtual machine boots to PXE and begins to install the chosen operating system.

DHCP Conflicts

If you use a virtual network on the VMware vSphere server for provisioning, ensure that you select a virtual network that does not provide DHCP assignments. This causes DHCP conflicts with Satellite Server when booting new hosts.

For image-based provisioning, use the pre-existing image as a basis for the new volume.

Procedure

To create a host for a VMware vSphere server, complete the following steps:

  1. In the Satellite web UI, navigate to Hosts > Create Host.
  2. In the Name field, enter the name that you want to become the provisioned system’s host name.
  3. Click the Organization and Location tabs to ensure that the provisioning context is automatically set to the current context.
  4. From the Host Group list, select the host group that you want to use to populate the form.
  5. From the Deploy on list, select the VMware vSphere connection.
  6. From the Compute Profile list, select a profile to use to automatically populate virtual machine-based settings.
  7. Click the Interface tab and click Edit on the host’s interface.
  8. Verify that the fields are automatically populated with values. Note in particular:

    • The Name from the Host tab becomes the DNS name.
    • The Satellite Server automatically assigns an IP address for the new host.
  9. Ensure that the MAC address field is blank. The VMware vSphere server assigns one to the host.
  10. Verify that the Managed, Primary, and Provision options are automatically selected for the first interface on the host. If not, select them.
  11. In the interface window, review the VMware vSphere-specific fields that are populated with settings from our compute profile. Modify these settings to suit your needs.
  12. Click the Operating System tab, and confirm that all fields automatically contain values.
  13. Select the Provisioning Method that you want:

    • For network-based provisioning, click Network Based.
    • For image-based provisioning, click Image Based.
    • For boot-disk provisioning, click Boot disk based.
  14. Click Resolve in Provisioning templates to check the new host can identify the right provisioning templates to use.
  15. Click the Virtual Machine tab and confirm that these settings are populated with details from the host group and compute profile. Modify these settings to suit your requirements.
  16. Click the Parameters tab and ensure that a parameter exists that provides an activation key. If not, add an activation key.
  17. Click Submit to save the host entry.

For CLI Users

Create the host from a network with the hammer host create command and include --provision-method build to use network-based provisioning.

# hammer host create --name "vmware-test1" --organization "My_Organization" \
--location "New York" --hostgroup "Base" \
--compute-resource "My_vSphere" --provision-method build \
--build true --enabled true --managed true \
--interface "managed=true,primary=true,provision=true,compute_type=VirtualE1000,compute_network=mynetwork" \
--compute-attributes="cpus=1,corespersocket=2,memory_mb=1024,cluster=MyCluster,path=MyVMs,start=true" \
--volume="size_gb=20G,datastore=Data,name=myharddisk,thin=true"

For more information about additional host creation parameters for this compute resource, enter the hammer host create --help command.

For CLI Users

Create the host from an image with the hammer host create command and include --provision-method image to use image-based provisioning.

# hammer host create --name "vmware-test2" --organization "My_Organization" \
--location "New York" --hostgroup "Base" \
--compute-resource "My_VMware" --provision-method image \
--image "Test VMware Image" --enabled true --managed true \
--interface "managed=true,primary=true,provision=true,compute_type=VirtualE1000,compute_network=mynetwork" \
--compute-attributes="cpus=1,corespersocket=2,memory_mb=1024,cluster=MyCluster,path=MyVMs,start=true" \
--volume="size_gb=20G,datastore=Data,name=myharddisk,thin=true"

For more information about additional host creation parameters for this compute resource, enter the hammer host create --help command.

11.7. Using the VMware vSphere Cloud-init and Userdata Templates for Provisioning

You can use VMware with the Cloud-init and Userdata templates to insert user data into the new virtual machine, to make further VMware customization, and to enable the VMware-hosted virtual machine to call back to Satellite.

You can use the same procedures to set up a VMware compute resource within Satellite, with a few modifications to the work flow.

VMware cloud-init Provisioning Overview

When you set up the compute resource and images for VMware provisioning in Satellite, the following sequence of provisioning events occur:

  • The user provisions one or more virtual machines using the Satellite web UI, API, or hammer
  • Satellite calls the VMware vCenter to clone the virtual machine template
  • Satellite userdata provisioning template adds customized identity information
  • When provisioning completes, the Cloud-init provisioning template instructs the virtual machine to call back to Capsule when cloud-init runs
  • VMware vCenter clones the template to the virtual machine
  • VMware vCenter applies customization for the virtual machine’s identity, including the host name, IP, and DNS
  • The virtual machine builds, cloud-init is invoked and calls back Satellite on port 80, which then redirects to 443

Port and Firewall Requirements

Because of the cloud-init service, the virtual machine always calls back to Satellite even if you register the virtual machine to Capsule. Ensure that you configure port and firewall settings to open any necessary connections.

For more information about port and firewall requirements, see Port and Firewall Requirements in the Installing Satellite and Ports and Firewalls Requirements in Installing Capsule Server.

Associating the userdata and Cloud-init Templates with the Operating System

  1. In the Satellite web UI, navigate to Hosts > Operating Systems, and select the operating system that you want to use for provisioning.
  2. Click the Template tab.
  3. From the Cloud-init template list, select Cloudinit default.
  4. From the User data template list, select UserData open-vm-tools.
  5. Click Submit to save the changes.

Preparing an Image to use the cloud-init Template

To prepare an image, you must first configure the settings that you require on a virtual machine that you can then save as an image to use in Satellite.

To use the cloud-init template for provisioning, you must configure a virtual machine so that cloud-init is installed, enabled, and configured to call back to Satellite Server.

For security purposes, you must install a CA certificate to use HTTPs for all communication. This procedure includes steps to clean the virtual machine so that no unwanted information transfers to the image you use for provisioning.

If you have an image with cloud-init, you must still follow this procedure to enable cloud-init to communicate with Satellite because cloud-init is disabled by default.

  1. On the virtual machine that you use to create the image, install cloud-init, open-vm-tools, and perl:

    # yum -y install cloud-init open-vm-tools perl
  2. Create a configuration file for cloud-init:

    # vi /etc/cloud/cloud.cfg.d/example_cloud-init_config.cfg
  3. Add the following information to the example_cloud_init_config.cfg file:

    datasource_list: [NoCloud]
    datasource:
      NoCloud:
        seedfrom: https://satellite.example.com/userdata/
    EOF
  4. Enable the CA certificates for the image:

    # update-ca-trust enable
  5. Download the katello-server-ca.crt file from Satellite Server:

    # wget -O /etc/pki/ca-trust/source/anchors/cloud-init-ca.crt http://satellite.example.com/pub/katello-server-ca.crt
  6. To update the record of certificates, enter the following command:

    # update-ca-trust extract
  7. Use the following commands to clean the image:

    # systemctl stop rsyslog
    # systemctl stop auditd
    # package-cleanup --oldkernels --count=1
    # yum clean all
  8. Use the following commands to reduce logspace, remove old logs, and truncate logs:

    # logrotate -f /etc/logrotate.conf
    # rm -f /var/log/*-???????? /var/log/*.gz
    # rm -f /var/log/dmesg.old
    # rm -rf /var/log/anaconda
    # cat /dev/null > /var/log/audit/audit.log
    # cat /dev/null > /var/log/wtmp
    # cat /dev/null > /var/log/lastlog
    # cat /dev/null > /var/log/grubby
  9. Remove udev hardware rules:

    # rm -f /etc/udev/rules.d/70*
  10. Remove the uuid from ifcfg scripts:

    # cat > /etc/sysconfig/network-scripts/ifcfg-ens192 <<EOM
    DEVICE=ens192
    ONBOOT=yes
    EOM
  11. Remove the SSH host keys:

    # rm -f /etc/ssh/SSH_keys
  12. Remove root user’s shell history:

    # rm -f ~root/.bash_history
    # unset HISTFILE
  13. Remove root user’s SSH history:

    # rm -rf ~root/.ssh/known_hosts

You can now create an image from this virtual machine.

You can use the Section 11.4, “Adding VMware vSphere Images to Satellite Server” section to add the image to Satellite.

Configuring Capsule to Forward the user data Template

If you deploy Satellite with the Capsule templates feature, you must configure Satellite to recognize hosts' IP addresses forwarded over the X-Forwarded-For HTTP header to serve correct template payload.

For security reasons, Satellite recognizes this HTTP header only from localhost. For each individual Capsule, you must configure a regular expression to recognize hosts' IP addresses. From the web UI, you can do this by navigating to Administer > Settings > Provisioning, and changing the Remote address setting. From the CLI, you can do this by entering the following command:

# hammer settings set --name remote_addr --value '(localhost(4|6|4to6)?|192.168.122.(1|2|3))'

11.8. Caching of Compute Resources

Caching of compute resources speeds up rendering of VMware information.

11.8.1. Enabling Caching of Compute Resources

To enable or disable caching of compute resources:

  1. In the Satellite web UI, navigate to Infrastructure > Compute Resources.
  2. Click the Edit button to the right of the VMware server you want to update.
  3. Select the Enable caching check box.

11.8.2. Refreshing the Compute Resources Cache

To refresh the cache of compute resources to update compute resources information:

Procedure

  1. In the Satellite web UI, navigate to Infrastructure > Compute Resources.
  2. Select a VMware server you want to refresh the compute resources cache for and click the Refresh Cache button.

For CLI Users

Use this API call to refresh the compute resources cache:

# curl -H "Accept:application/json,version=2" \
-H "Content-Type:application/json" -X PUT \
-u username:password -k \
https://satellite.example.com/api/compute_resources/compute_resource_id/refresh_cache

Use the hammer compute-resource list command to determine the ID of the VMware server you want to refresh the compute resources cache for.