Chapter 7. Troubleshooting Container Development Kit Problems

Use the information in this chapter to troubleshoot Vagrant Container Development Kit in general. See the Red Hat Container Development Kit 2.3 Release Notes and Known Issues to learn about the main features of the product and about problems that you may encounter when using this version.

7.1. Troubleshooting General Problems

How do I start Vagrant?
Vagrant is strictly command-line oriented. There are no menu entries. To run Vagrant, launch a command prompt or a terminal session on your machine. Vagrant should be automatically added to your path, so you only need to type vagrant.
Something failed during vagrant up or it complained that an SSH command failed. If I try vagrant up again, it says the VM is already running.

There are a number of provisioning steps performed when Vagrant launches a virtual machine. These may come from plugins, such as the Vagrant Registration plugin for Red Hat, or from steps that are included in the Vagrantfile for provisioning the box on startup. Vagrant runs these commands by using SSH.

In many cases, errors during the provisioning step are not fatal, so the Vagrant box will still be running. Use the vagrant status command to see what the state of your box is. You can stop the Vagrant box with the vagrant halt command. If the machine is running, you should be able to log into it using the vagrant ssh command to examine the VM and see what went wrong.

While the box is running, you can rerun the provisioning steps by running the vagrant provision command.

Note that you need to be in the same directory where your Vagrantfile is. If you have lost track of where your Vagrantfile is, use the vagrant global-status command to list the boxes that you have started and the directory where the Vagrantfile and state is stored.

The vagrant up command fails with error message could not find capabilities for domaintype=kvm.

Any system running the CDK (Windows, Mac, or RHEL), requires not only that the computer it is running on have Virtualization Support, but also that the support be enabled in the BIOS. On a Red Hat Enterprise Linux system, if Virtualization Support is not enabled when you run vagrant up, you will see a message similar to the following:

Error while creating domain: Error saving the server: Call to virDomainDefineXML failed: invalid argument: could not find capabilities for domaintype=kvm

Enabling Virtualization Support in the computer’s BIOS should fix the problem.

On Red Hat Enterprise Linux, run the virt-host-validate command to check whether or not virtualization has been enabled on the computer.

The output of vagrant up indicates that OpenShift failed to start when using the rhel-ose Vagrantfile.

Due to internal configuration problems of the Container Development Kit box, OpenShift start may time out. To repeat the provisioning steps, and thus start OpenShift in the Container Development Environment, run:

$ vagrant provision
Red Hat registration failed, or I entered the wrong username and password. How can I manually register the box or check my subscription status?

You can run the subscription-manager tool to log in, register or unregister the box, check your subscription status, or enable available software repositories as you would on any other Red Hat Enterprise Linux installation. However, you need to use the sudo -i command to be root because the root password for the Red Hat Enterprise Linux Vagrant boxes is not distributed. The sudo utility has been setup for the vagrant user to run any commands.

For more information, see Red Hat Subscription Management.

How do I free up my Red Hat subscription used by the Vagrant box?

The Vagrant Registration plugin automatically detaches the box from the Red Hat subscription when you shut down the box using the vagrant halt or vagrant destroy commands. If the box is not shut down by Vagrant, this does not happen. If you still have the box set up, you should be able to bring the box up again using the vagrant up command and then shut it down correctly with vagrant halt or vagrant destroy, which will unregister the box from Red Hat Subscription Management.

Alternatively, you can use the subscription management on the Red Hat Customer Portal to find and delete the virtual system that is no longer being used.

I need to make changes as root on the Vagrant boxes, what is the root password?
The sudo utility has been set up for the vagrant user to run any commands as root. You can use the sudo command to run a single command as root, or use sudo -i to start an interactive root shell.
Running vagrant global-status gives me a permission denied error.
Run the id command to check whether your regular user ID is a member of the vagrant group. If you skipped this step before, log out and log back in for the changes to take effect. Adding the user to the vagrant group works in conjuction with the PolicyKit rule that was added during the steps run under root to allow non-root users to work with libvirt.

7.2. Troubleshooting Problems with libvirt

When I try to run Vagrant, I get an error message about connecting to libvirt. Do I need to be root to run Vagrant?

One of the steps in section Section 5.2.1.2, “Installing and Initializing Virtualization Software” installs a PolicyKit rule in the /etc/polkit-1/rules.d/ directory that allows non-root users to run Vagrant and perform libvirt operations that are normally restricted to root users as long as the user is a member of the vagrant group. The error message is:

Error while connecting to libvirt: Error making a connection to libvirt URI qemu:///system:
Call to virConnectOpen failed: authentication failed: no agent is available to authenticate

To resolve this, make sure your user ID is a member of the vagrant group. You can do this by running the id command. Review the installation steps listed above to verify that the PolicyKit rule is installed in /etc/polkit-1/rules.d/. Restart the PolicyKit and libvirtd services for the changes to take effect:

# systemctl restart libvirtd
# systemctl restart polkit
The vagrant command is failing because libvirt says the resource it is trying to create already exists.

Normally, Vagrant takes care of all interaction with libvirt, creating and destroying resources as necessary. Running the vagrant destroy command should free up any resources that were allocated to an environment. However, if the Vagrant state directory, .vagrant is deleted, or if a Vagrant operation is interrupted, before it can clean up, it is possible that libvirt resources are left around that cannot be removed using vagrant destroy. If that happens, you need to use the virsh command-line utility or virt-manager, a graphical tool, to clean up.

Note that these tools must be run as root (or with the sudo command) to see all of the libvirt resources on the system. If you do not run them as root, it appears that there are no allocated resources.

The vagrant up command fails with Name rhel-ose_default of domain about to create is already taken.

If there is no volume of such name (check by running virsh vol-list default), you can undefine this name in libvirt:

# virsh undefine rhel-ose_default
I want to install an updated Vagrant box, but when I run it, I keep seeing the older box still being used.

For Container Development Kit with libvirt, old Vagrant boxes are not automatically removed when the new one is installed. As a result, even after adding a new box, the old one is used in place of the new box.

To add a new Vagrant box for an updated Container Development Kit, you need to destroy the running boxes, delete the .vagrant/ directory in each directory where you ran vagrant up and remove the image created by each box.

If you ran both the OpenShift and Kubernetes Vagrantfiles, delete the respective virtual machines completely before adding the updated box. First go to each directory to stop the running box and remove the .vagrant/ directory.

Warning

The rm -rf command is destructive. Be absolutely sure you identify the proper directory to delete before running this command.

$ cd ~/cdk/components/rhel/misc/rhel-k8s-singlenode-setup
$ vagrant destroy && rm -rf .vagrant/
$ cd ~/cdk/components/rhel/rhel-ose
$ vagrant destroy && rm -rf .vagrant/

Next remove the images representing the Vagrant boxes in the libvirt images directory and restart libvirt.

# rm -rf /var/lib/libvirt/images/*_0.img
# systemctl restart libvirtd

You can now proceed to adding the new Vagrant box.

7.3. Troubleshooting Problems on Microsoft Windows

The vagrant ssh command is not working, it cannot find ssh.exe.
In order to use vagrant ssh on a Microsoft Windows system, you need to have ssh.exe in your path. It is recommended to install Cygwin and ensure that ssh.exe is installed and in your path.
The vagrant up command refused to run because rsync is not in the path.
If rsync is being used for Vagrant synchronized folders on a Microsoft Windows system because it was explicitly selected, Vagrant does not start a Vagrant box unless rsync.exe is available in the path. It is recommended to install Cygwin and ensure that rsync.exe is in your path.
Something failed during vagrant up or it complained that an SSH command failed. If I try vagrant up again, it says the VM is already running.

There are a number of provisioning steps performed when Vagrant launches a virtual machine. These may come from plugins, such as the vagrant-registration plugin, or from steps that are included in the Vagrantfile for provisioning the box on startup. Vagrant runs these commands by using SSH.

In many cases, errors during the provisioning step are not fatal, so the Vagrant box is still running. Use the vagrant status command to see what the state of your box is. You can stop the Vagrant box with the vagrant halt command. If the machine is running, you should be able to log into it using the vagrant ssh command to examine the VM and see what went wrong.

While the box is running, you can rerun the provisioning steps by running the vagrant provision command.

Note that you need to be in the same directory where your Vagrant file is. If you have lost track of where your Vagrantfile is, use the vagrant global-status command to list all the boxes that you have started and the directory where the Vagrantfile and state is stored.

Errors upon upgrading Vagrant from 1.7.4 to 1.8.1.
The C:/Users/cdk/.vagrant.d/ folder needs to be removed when upgrading from Vagrant 1.7.4 to Vagrant 1.8.1. After uninstalling the 1.7.4 version, remove the folder and the install the 1.8.1 version.

7.4. Additional Resources