Chapter 7. Troubleshooting Container Development Kit Problems

There are a number of provisioning steps performed when Vagrant launches a virtual machine. These may come from plugins, such as the Vagrant Registration plugin for Red Hat, or from steps that are included in the Vagrantfile for provisioning the box on startup. Vagrant runs these commands by using SSH.

In many cases, errors during the provisioning step are not fatal, so the Vagrant box will still be running. Use the vagrant status command to see what the state of your box is. You can stop the Vagrant box with the vagrant halt command. If the machine is running, you should be able to log into it using the vagrant ssh command to examine the VM and see what went wrong.

While the box is running, you can rerun the provisioning steps by running the vagrant provision command.

Note that you need to be in the same directory where your Vagrantfile is. If you have lost track of where your Vagrantfile is, use the vagrant global-status command to list the boxes that you have started and the directory where the Vagrantfile and state is stored.

Any system running the CDK (Windows, Mac, or RHEL), requires not only that the computer it is running on have Virtualization Support, but also that the support be enabled in the BIOS. On a Red Hat Enterprise Linux system, if Virtualization Support is not enabled when you run vagrant up, you will see a message similar to the following:

Error while creating domain: Error saving the server: Call to virDomainDefineXML failed: invalid argument: could not find capabilities for domaintype=kvm

Enabling Virtualization Support in the computer’s BIOS should fix the problem.

On Red Hat Enterprise Linux, run the virt-host-validate command to check whether or not virtualization has been enabled on the computer.

Due to internal configuration problems of the Container Development Kit box, OpenShift start may time out. To repeat the provisioning steps, and thus start OpenShift in the Container Development Environment, run:

$ vagrant provision

You can run the subscription-manager tool to log in, register or unregister the box, check your subscription status, or enable available software repositories as you would on any other Red Hat Enterprise Linux installation. However, you need to use the sudo -i command to be root because the root password for the Red Hat Enterprise Linux Vagrant boxes is not distributed. The sudo utility has been setup for the vagrant user to run any commands.

For more information, see Red Hat Subscription Management.

The Vagrant Registration plugin automatically detaches the box from the Red Hat subscription when you shut down the box using the vagrant halt or vagrant destroy commands. If the box is not shut down by Vagrant, this does not happen. If you still have the box set up, you should be able to bring the box up again using the vagrant up command and then shut it down correctly with vagrant halt or vagrant destroy, which will unregister the box from Red Hat Subscription Management.

Alternatively, you can use the subscription management on the Red Hat Customer Portal to find and delete the virtual system that is no longer being used.

One of the steps in section Section 5.2.1.2, “Installing and Initializing Virtualization Software” installs a PolicyKit rule in the /etc/polkit-1/rules.d/ directory that allows non-root users to run Vagrant and perform libvirt operations that are normally restricted to root users as long as the user is a member of the vagrant group. The error message is:

Error while connecting to libvirt: Error making a connection to libvirt URI qemu:///system:
Call to virConnectOpen failed: authentication failed: no agent is available to authenticate

To resolve this, make sure your user ID is a member of the vagrant group. You can do this by running the id command. Review the installation steps listed above to verify that the PolicyKit rule is installed in /etc/polkit-1/rules.d/. Restart the PolicyKit and libvirtd services for the changes to take effect:

# systemctl restart libvirtd
# systemctl restart polkit

Normally, Vagrant takes care of all interaction with libvirt, creating and destroying resources as necessary. Running the vagrant destroy command should free up any resources that were allocated to an environment. However, if the Vagrant state directory, .vagrant is deleted, or if a Vagrant operation is interrupted, before it can clean up, it is possible that libvirt resources are left around that cannot be removed using vagrant destroy. If that happens, you need to use the virsh command-line utility or virt-manager, a graphical tool, to clean up.

Note that these tools must be run as root (or with the sudo command) to see all of the libvirt resources on the system. If you do not run them as root, it appears that there are no allocated resources.

If there is no volume of such name (check by running virsh vol-list default), you can undefine this name in libvirt:

# virsh undefine rhel-ose_default

For Container Development Kit with libvirt, old Vagrant boxes are not automatically removed when the new one is installed. As a result, even after adding a new box, the old one is used in place of the new box.

To add a new Vagrant box for an updated Container Development Kit, you need to destroy the running boxes, delete the .vagrant/ directory in each directory where you ran vagrant up and remove the image created by each box.

If you ran both the OpenShift and Kubernetes Vagrantfiles, delete the respective virtual machines completely before adding the updated box. First go to each directory to stop the running box and remove the .vagrant/ directory.

$ cd ~/cdk/components/rhel/misc/rhel-k8s-singlenode-setup
$ vagrant destroy && rm -rf .vagrant/
$ cd ~/cdk/components/rhel/rhel-ose
$ vagrant destroy && rm -rf .vagrant/

Next remove the images representing the Vagrant boxes in the libvirt images directory and restart libvirt.

# rm -rf /var/lib/libvirt/images/*_0.img
# systemctl restart libvirtd

You can now proceed to adding the new Vagrant box.

There are a number of provisioning steps performed when Vagrant launches a virtual machine. These may come from plugins, such as the vagrant-registration plugin, or from steps that are included in the Vagrantfile for provisioning the box on startup. Vagrant runs these commands by using SSH.

In many cases, errors during the provisioning step are not fatal, so the Vagrant box is still running. Use the vagrant status command to see what the state of your box is. You can stop the Vagrant box with the vagrant halt command. If the machine is running, you should be able to log into it using the vagrant ssh command to examine the VM and see what went wrong.

While the box is running, you can rerun the provisioning steps by running the vagrant provision command.

Note that you need to be in the same directory where your Vagrant file is. If you have lost track of where your Vagrantfile is, use the vagrant global-status command to list all the boxes that you have started and the directory where the Vagrantfile and state is stored.