12.3. Installing the Para-virtualized Drivers

The following sections describe how to install and configure your fully virtualized guests to run on Red Hat Enterprise Linux 5.1 or above with para-virtualized drivers.

Important

Para-virtualized drivers are only supported on certain hardware and version combinations. Verify your hardware and operating system requirements are met before proceeding to install para-virtualized drivers.

Note

If you are installing a new guest system, in order to gain maximal benefit from the para-virtualized block device drivers, you should create the guest with at least two disks.
Using the para-virtualized drivers for the disk that contains the MBR and the boot loader (GRUB), and for the /boot partition. This partition can be very small, as it only needs to have enough capacity to hold the /boot partition.
Use the second disk and any additional disks for all other partitions (for example, /, /usr) or logical volumes.
Using this installation method, when the para-virtualized block device drivers are later installed after completing the install of the guest, only booting the guest and accessing the /boot partition will use the virtualized block device drivers.

12.3.1. Common installation steps

The list below covers the high level steps common across all guest operating system versions.
  1. Copy the RPMs for your hardware architecture to a suitable location in your guest operating system. Your home directory is sufficient. If you do not know which RPM you require verify against the table at Section 12.2, “Para-virtualization Restrictions and Support”.
  2. Use the rpm command or the yum command to install the packages. The rpm utility will install the following four new kernel modules into /lib/modules/[%kversion][%kvariant]/extra/xenpv/%release:
    • the PCI infrastructure module, xen_platform_pci.ko,
    • the ballooning module, xen_balloon.ko,
    • the virtual block device module, xen_vbd.ko,
    • and the virtual network device module, xen_vnif.ko.
  3. If the guest operating does not support automatically loading the para-virtualized drivers (for example, Red Hat Enterprise Linux 3) perform the required post-install steps to copy the drivers into the operating system specific locations.
  4. Shut down your guest operating system.
  5. Reconfigure the guest operating system's configuration file on the host to use the installed para-virtualized drivers.
  6. Remove the “type=ioemu” entry for the network device.
  7. Add any additional disk partitions, volumes or LUNs to the guest so that they can be accessed via the para-virtualized (xen-vbd) disk driver.
  8. For each physical device, LUN, partition or volume you want to use the para-virtualized drivers you must edit the disk entry for that device in the libvirt configuration file.
  9. A typical disk entry resembles the following:
    <disk type='file' device='disk'>
      <driver name='file'/>
      <source file='/dev/hda6'/>
      <target dev='hda'/>
    </disk>
    
    Modify each disk entry, as desired, to use the para-virtualized by changing the driver elements as shown below.
    <disk type='file' device='disk'>
      <driver name='tap' type='aio'/>
      <source file='/dev/hda6'/>
      <target dev='xvda'/>
    </disk>
    
  10. Add any additional storage entities you want to use for the para-virtualized block device driver.
  11. Restart your guest:
    # xm start YourGuestName
    Where YourGuestName is the name of the configuration file or the guest operating system's name as defined in its configuration file in the name = "os_name" parameter.
  12. Reconfigure the guest network.

12.3.2. Installation and Configuration of Para-virtualized Drivers on Red Hat Enterprise Linux 3

This section contains detailed instructions for the para-virtualized drivers in a Red Hat Enterprise 3 guest operating system.

Note

These packages do not support booting from a para-virtualized disk. Booting the guest operating system kernel still requires the use of the emulated IDE driver, while any other (non-system) user-space applications and data can use the para-virtualized block device drivers.
Driver Installation

The list below covers the steps to install a Red Hat Enterprise Linux 3 guest with para-virtualized drivers.

  1. Install the latest kernel version. The para-virtualized drivers require at least Red Hat Enterprise Linux 3.9 kernel version kernel-2.4.21-60.EL for all the required headers.
  2. Copy the kmod-xenpv rpm for your hardware architecture and kernel variant to your guest operating system.
  3. Use the rpm utility to install the RPM packages. Ensure you have correctly identified which package you need for your guest operating system variant and architecture.
    [root@rhel3]# rpm -ivh kmod-xenpv*
    
  4. Use the commands below load the para-virtualized driver modules. %kvariant is the kernel variant the para-virtualized drivers have been build against and %release corresponds to the release version of the para-virtualized drivers.
    [root@rhel3]# mkdir -p /lib/modules/'uname -r'/extra/xenpv
    [root@rhel3]# cp -R /lib/modules/2.4.21-52.EL[%kvariant]/extra/xenpv/%release \
    /lib/modules/'uname -r'/extra/xenpv
    [root@rhel3]# depmod -ae
    [root@rhel3]# modprobe xen-vbd
    [root@rhel3]# modprobe xen-vnif
    

    Note

    Warnings will be generated by insmod when installing the binary driver modules due to Red Hat Enterprise Linux 3 having MODVERSIONS enabled. These warnings can be ignored.
  5. Verify /etc/modules.conf and make sure you have an alias for eth0 like the one below. If you are planning to configure multiple interfaces add an additional line for each interface.
    alias eth0 xen-vnif
    
    Edit /etc/rc.local and add the line:
    insmod /lib/modules/'uname -r'/extra/xenpv/%release/xen-vbd.o
    

    Note

    Substitute “%release” with the actual release version (for example 0.1-5.el) for the para-virtualized drivers. If you update the para-virtualized driver RPM package make sure you update the release version to the appropriate version.
  6. Shutdown the virtual machine (use “#shutdown -h now” inside the guest).
  7. Edit the guest configuration file in /etc/xen/YourGuestName with a text editor, performing the following changes:
    • Remove the “type=ioemu” entry from the “vif=” entry.
    • Add any additional disk partitions, volumes or LUNs to the guest so that they can be accessed via the para-virtualized (xen-vbd) disk driver.
    • For each physical device, LUN, partition or volume you want to use the para-virtualized drivers you must edit the disk entry for that device in the libvirt configuration file.
    • A typical disk entry resembles the following:
      <disk type='file' device='disk'>
        <driver name='file'/>
        <source file='/dev/hda6'/>
        <target dev='hda'/>
      </disk>
      
      Modify each disk entry, as desired, to use the para-virtualized by changing the driver elements as shown below.
      <disk type='file' device='disk'>
        <driver name='tap' type='aio'/>
        <source file='/dev/hda6'/>
        <target dev='xvda'/>
      </disk>
      
      
    • Once complete, save the modified configuration file and restart the guest.
  8. Boot the virtual machine:
    # xm start YourGuestName
    Where YourGuestName is the name of the configuration file or the guest operating system's name as defined in its configuration file in the name = "os_name" parameter.

Warning

The para-virtualized drivers are not automatically added and loaded to the system because weak-modules and modversions support is not provided in Red Hat Enterprise Linux 3. To insert the module execute the command below.
insmod xen_vbd.ko
Red Hat Enterprise Linux 3 requires the manual creation of the special files for the block devices which use xen-vbd. The steps below will cover how to create and register para-virtualized block devices.
Use the following script to create the special files after the para-virtualized block device driver is loaded.
#!/bin/sh
module="xvd"
mode="664"
major=`awk "\\$2==\"$module\" {print \\$1}" /proc/devices`
# < mknod for as many or few partitions on xvd disk attached to FV guest >
# change/add xvda to xvdb, xvbd, etc. for 2nd, 3rd, etc., disk added in
# in xen config file, respectively.
mknod /dev/xvdb b $major 16
mknod /dev/xvdb1 b $major 17
mknod /dev/xvdb2 b $major 18
chgrp disk /dev/xvd*
chmod 0660 /dev/xvd*
For each additional virtual disk, increment the minor number by 16. In the example below an additional device, minor number 16, is created.
# mknod /dev/xvdc b $major 16
# mknod /dev/xvdc1 b $major 17
This would make the next device 32 which can be created by:
# mknod /dev/xvdd b $major 32
# mknod /dev/xvdd1 b $major 33
Now you should verify the partitions which you have created are available.
[root@rhel3]# cat /proc/partitions
major   minor     #blocks   name

  3        0      10485760  hda
  3        1        104391  hda1
  3        2      10377990  hda2
202        16         64000  xvdb
202        17         32000  xvdb1
202        18        32000  xvdb2
253        0       8257536  dm-0
253        1       2031616  dm-1
In the above output, you can observe that the partitioned device “xvdb” is available to the system.
The procedure below adds the new device to the guest and makes it persistent after rebooting. All these commands are executed on the guest.
  1. Create directories to mount the block device image in.
    [root@rhel3]# mkdir /mnt/pvdisk_p1
    [root@rhel3]# mkdir /mnt/pvdisk_p2
    
  2. Mount the devices to the new folders.
    [root@rhel3]# mount /dev/xvdb1 /mnt/pvdisk_p1
    [root@rhel3]# mount /dev/xvdb2 /mnt/pvdisk_p2
    
  3. Verify the devices are mounted correctly.
    [root@rhel3]# df /mnt/pvdisk_p1
    Filesystem           1K-blocks      Used   Available Use%  Mounted on
    /dev/xvdb1               32000        15       31985   1%  /mnt/pvdisk_p1
    
  4. Update the /etc/fstab file inside the guest to mount the devices during the boot sequence. Add the following lines:
    /dev/xvdb1   /mnt/pvdisk_p1   ext3    defaults        1 2
    /dev/xvdb2   /mnt/pvdisk_p2   ext3    defaults        1 2
    

Note

Using a Red Hat Enterprise Linux 5.1 host (dom0), the "noapic" parameter should be added to the kernel boot line in your virtual guest's /boot/grub/grub.conf entry as seen below. Keep in mind your architecture and kernel version may be different.
kernel /vmlinuz-2.6.9-67.EL ro root=/dev/VolGroup00/rhel4_x86_64 rhgb noapic
A Red Hat Enterprise Linux 5.2 dom0 will not need this kernel parameter for the guest.

Important

The Itanium (ia64) binary RPM packages and builds are not presently available.

12.3.3. Installation and Configuration of Para-virtualized Drivers on Red Hat Enterprise Linux 4

This section contains detailed instructions for the para-virtualized drivers in a Red Hat Enterprise 4 guest operating system.

Note

These packages do not support booting from a para-virtualized disk. Booting the guest operating system kernel still requires the use of the emulated IDE driver, while any other (non-system) user-space applications and data can use the para-virtualized block device drivers.
Driver Installation

The list below covers the steps to install a Red Hat Enterprise Linux 4 guest with para-virtualized drivers.

  1. Copy the kmod-xenpv, modules-init-tools and modversions RPMs for your hardware architecture and kernel variant to your guest operating system.
  2. Use the rpm utility to install the RPM packages. Make sure you have correctly identified which package you need for your guest operating system variant and architecture. An updated module-init-tools is required for this package, it is available with the Red Hat Enterprise Linux 4-6-z kernel or newer.
    [root@rhel4]# rpm -ivh modversions
    [root@rhel4]# rpm -Uvh module-init-tools
    [root@rhel4]# rpm -ivh kmod-xenpv*
    

    Note

    There are different packages for UP, SMP, Hugemem and architectures so make sure you have the right RPMs for your kernel.
  3. Execute cat /etc/modprobe.conf to verify you have an alias for eth0 like the one below. If you are planning to configure multiple interfaces add an additional line for each interface. If it does not look like the entry below change it.
    alias eth0 xen-vnif
    
  4. Shutdown the virtual machine (use “#shutdown -h now” inside the guest).
  5. Edit the guest configuration file in /etc/xen/YourGuestsName in the following ways:
    • Remove the “type=ioemu” entry from the “vif=” entry.
    • Add any additional disk partitions, volumes or LUNs to the guest so that they can be accessed via the para-virtualized (xen-vbd) disk driver.
    • For each additional physical device, LUN, partition or volume add an entry similar to the one shown below to the “disk=” section in the guest configuration file. The original “disk=” entry might also look like the entry below.
      disk = [ "file:/var/lib/libvirt/images/rhel4_64_fv.dsk,hda,w"]
      
    • Once you have added additional physical devices, LUNs, partitions or volumes; the para-virtualized driver entry in your XML configuration file should resemble the entry shown below.
      disk = [ "file:/var/lib/libvirt/images/rhel3_64_fv.dsk,hda,w",
      "tap:aio:/var/lib/libvirt/images/UserStorage.dsk,xvda,w" ]
      

      Note

      Use “tap:aio” for the para-virtualized device if a file-based image is used.
  6. Boot the virtual machine using the virsh command:
    # virsh start YourGuestName
On the first reboot of the virtual guest, kudzu will ask you to "Keep or Delete the Realtek Network device" and "Configure the xen-bridge device". You should configure the xen-bridge and delete the Realtek network device.

Note

Using a Red Hat Enterprise Linux 5.1 host (dom0), the "noapic" parameter should be added to the kernel boot line in your virtual guest's /boot/grub/grub.conf entry as seen below. Keep in mind your architecture and kernel version may be different.
kernel /vmlinuz-2.6.9-67.EL ro root=/dev/VolGroup00/rhel4_x86_64 rhgb noapic
A Red Hat Enterprise Linux 5.2 dom0 will not need this kernel parameter for the guest.
Now, verify the partitions which you have created are available.
[root@rhel4]# cat /proc/partitions
major    minor   #blocks   name

   3        0    10485760  hda
   3        1      104391  hda1
   3        2    10377990  hda2
 202        0       64000  xvdb
 202        1       32000  xvdb1
 202        2       32000  xvdb2
 253        0     8257536  dm-0
 253        1     2031616  dm-1
In the above output, you can see the partitioned device “xvdb” is available to the system.
The procedure below adds the new device to the guest and makes it persistent after rebooting. All these commands are executed on the guest.
  1. Create directories to mount the block device image in.
    [root@rhel4]# mkdir /mnt/pvdisk_p1
    [root@rhel4]# mkdir /mnt/pvdisk_p2
    
  2. Mount the devices to the new folders.
    [root@rhel4]# mount /dev/xvdb1 /mnt/pvdisk_p1
    [root@rhel4]# mount /dev/xvdb2 /mnt/pvdisk_p2
    
  3. Verify the devices are mounted correctly.
    [root@rhel4]# df /mnt/pvdisk_p1
    Filesystem           1K-blocks      Used   Available Use%  Mounted on
    /dev/xvdb1               32000        15       31985   1%  /mnt/pvdisk_p1
    
  4. Update the /etc/fstab file inside the guest to mount the devices during the boot sequence. Add the following lines:
    /dev/xvdb1   /mnt/pvdisk_p1   ext3    defaults        1 2
    /dev/xvdb2   /mnt/pvdisk_p2   ext3    defaults        1 2
    

Note

This package is not supported for Red Hat Enterprise Linux 4-GA through Red Hat Enterprise Linux 4 update 2 systems and kernels.

Important

IA64 binary RPM packages and builds are not presently available.

Note

The xen-vbd driver may not automatically load. Execute the following command on the guest, substituting %release with the correct release version for the para-virtualized drivers.
# insmod /lib/modules/'uname -r'/weak-updates/xenpv/%release/xen_vbd.ko

12.3.4. Xen Para-virtualized Drivers on Red Hat Enterprise Linux 5

This section contains detailed instructions for the para-virtualized drivers in a Red Hat Enterprise Linux 5 guest operating system.

Note

These packages do not support booting from a para-virtualized disk. Booting the guest operating system kernel still requires the use of the emulated IDE driver, while any other (non-system) user-space applications and data can use the para-virtualized block device drivers.
The procedure below covers the steps to enable the para-virtualized drivers for a Red Hat Enterprise Linux 5 guest.

Procedure 12.1. Enable para-virtualized drivers for a Red Hat Enterprise Linux Guest

  1. Shutdown the virtual machine (use “#shutdown -h now” inside the guest).
  2. Edit the guest configuration file in /etc/xen/<Your GuestsName> in the following ways:
    1. Remove the “type=ioemu” entry from the “vif=” entry.
    2. Add any additional disk partitions, volumes or LUNs to the guest so that they can be accessed via the para-virtualized (xen-vbd) disk driver.
    3. For each additional physical device, LUN, partition or volume add an entry similar to the one shown below to the “disk=” section in the guest configuration file. The original “disk=” entry might also look like the entry below.
      disk = [ "file:/var/lib/libvirt/images/rhel4_64_fv.dsk,hda,w"]
      
    4. Once you have added additional physical devices, LUNs, partitions or volumes; the para-virtualized driver entry in your XML configuration file should resemble the entry shown below.
      disk = [ "file:/var/lib/libvirt/images/rhel3_64_fv.dsk,hda,w",
      "tap:aio:/var/lib/libvirt/images/UserStorage.dsk,xvda,w" ]
      

      Note

      Use “tap:aio” for the para-virtualized device if a file-based image is used.
  3. Boot the virtual machine using the virsh command:
    # virsh start YourGuestName
To verify the network interface has come up after installing the para-virtualized drivers issue the following command on the guest. It should display the interface information including an assigned IP address
[root@rhel5]# ifconfig eth0
Now, verify the partitions which you have created are available.
[root@rhel5]# cat /proc/partitions
major minor  #blocks    name
   3     0   10485760   hda
   3     1     104391   hda1
   3     2   10377990   hda2
 202     0      64000   xvdb
 202     1      32000   xvdb1
 202     2      32000   xvdb2
 253     0    8257536   dm-0
 253     1    2031616   dm-1
In the above output, you can see the partitioned device “xvdb” is available to the system.
The procedure below adds the new device to the guest and makes it persistent after rebooting. All these commands are executed on the guest.
  1. Create directories to mount the block device image in.
    [root@rhel5]# mkdir /mnt/pvdisk_p1
    [root@rhel5]# mkdir /mnt/pvdisk_p2
    
  2. Mount the devices to the new folders.
    [root@rhel5]# mount /dev/xvdb1 /mnt/pvdisk_p1
    [root@rhel5]# mount /dev/xvdb2 /mnt/pvdisk_p2
    
  3. Verify the devices are mounted correctly.
    [root@rhel5]# df /mnt/pvdisk_p1
    Filesystem           1K-blocks      Used   Available Use%  Mounted on
    /dev/xvdb1               32000        15       31985   1%  /mnt/pvdisk_p1
    
  4. Update the /etc/fstab file inside the guest to mount the devices during the boot sequence. Add the following lines:
    /dev/xvdb1   /mnt/pvdisk_p1   ext3    defaults        1 2
    /dev/xvdb2   /mnt/pvdisk_p2   ext3    defaults        1 2
    

Note

Using a Red Hat Enterprise Linux 5.1 host (dom0), the "noapic" parameter should be added to the kernel boot line in your virtual guest's /boot/grub/grub.conf entry as seen below. Keep in mind your architecture and kernel version may be different.
kernel /vmlinuz-2.6.9-67.EL ro root=/dev/VolGroup00/rhel4_x86_64 rhgb noapic
A Red Hat Enterprise Linux 5.2 dom0 will not need this kernel parameter for the guest.
Hiding fake interfaces

Sometimes, activating the para-virtualized drivers does not delete the old virtualized network interfaces. To remove these interfaces from guests use the following procedure.

  1. Add the following lines to the /etc/modprobe.d/blacklist file. Blacklist 8139cp and 8139too for the RealTek 8139 and e1000 for the virtualized Intel e1000 NIC.
    8139cp
    8139too
    e1000
  2. Remove the old network scripts from the /etc/sysconfig/network-scripts directory.
  3. Reboot the guest. The default network interface should now use the para-virtualized drivers.

12.3.5. Xen Para-virtualized Drivers on Red Hat Enterprise Linux 6

This section describes the use of para-virtualized drivers in a Red Hat Enterprise Linux 6 guest.
The para-virtualized drivers are enabled by default for a Red Hat Enterprise Linux 6 guest. The guest will automatically unplug any emulated devices that are presented to it, and will use the para-virtualized drivers instead.
Although the para-virtualized drivers are enabled by default, they can be disabled. Add the following to the guest kernel command line upon boot to disable the para-virtualized drivers:
xen_emul_unplug=never