Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

2.2. perf kvm

You can use the perf command with the kvm option to collect guest operating system statistics from the host.
In Red Hat Enterprise Linux, the perf package provides the perf command. Run rpm -q perf to see if the perf package is installed. If it is not installed, and you want to install it to collect and analyze guest operating system statistics, run the following command as the root user: 
yum install perf
In order to use perf kvm in the host, you must have access to the /proc/modules and /proc/kallsyms files from the guest. There are two methods to achieve this. Refer to the following procedure, Procedure 2.1, “Copying /proc files from guest to host” to transfer the files into the host and run reports on the files. Alternatively, refer to Procedure 2.2, “Alternative: using sshfs to directly access files” to directly mount the guest and access the files.

Procedure 2.1. Copying /proc files from guest to host

Important

If you directly copy the required files (for instance, via scp) you will only copy files of zero length. This procedure describes how to first save the files in the guest to a temporary location (with the cat command), and then copy them to the host for use by perf kvm.

  1. Log in to the guest and save files

    Log in to the guest and save /proc/modules and /proc/kallsyms to a temporary location, /tmp: 
    # cat /proc/modules > /tmp/modules
# cat /proc/kallsyms > /tmp/kallsyms

  2. Copy the temporary files to the host

    Once you have logged off from the guest, run the following example scp commands to copy the saved files to the host. You should substitute your host name and TCP port if they are different: 
    # scp root@GuestMachine:/tmp/kallsyms guest-kallsyms
# scp root@GuestMachine:/tmp/modules guest-modules
    You now have two files from the guest (guest-kallsyms and guest-modules) on the host, ready for use by perf kvm.

  3. Recording and reporting events with perf kvm

    Using the files obtained in the previous steps, recording and reporting of events in the guest, the host, or both is now possible.
    Run the following example command: 
    # perf kvm --host --guest --guestkallsyms=guest-kallsyms \
--guestmodules=guest-modules record -a -o perf.data

    Note

    If both --host and --guest are used in the command, output will be stored in perf.data.kvm. If only --host is used, the file will be named perf.data.host. Similarly, if only --guest is used, the file will be named perf.data.guest.
    Pressing Ctrl-C stops recording.

  4. Reporting events

    The following example command uses the file obtained by the recording process, and redirects the output into a new file, analyze. 
    perf kvm --host --guest --guestmodules=guest-modules report -i perf.data.kvm \
--force > analyze
    View the contents of the analyze file to examine the recorded events: 
    # cat analyze


# Events: 7K cycles
#
# Overhead       Command  Shared Object      Symbol
# ........  ............  .................  .........................
#
    95.06%            vi  vi                 [.] 0x48287
     0.61%          init  [kernel.kallsyms]  [k] intel_idle
     0.36%            vi  libc-2.12.so       [.] _wordcopy_fwd_aligned
     0.32%            vi  libc-2.12.so       [.] __strlen_sse42
     0.14%       swapper  [kernel.kallsyms]  [k] intel_idle
     0.13%          init  [kernel.kallsyms]  [k] uhci_irq
     0.11%          perf  [kernel.kallsyms]  [k] generic_exec_single
     0.11%          init  [kernel.kallsyms]  [k] tg_shares_up
     0.10%      qemu-kvm  [kernel.kallsyms]  [k] tg_shares_up

[output truncated...]

Procedure 2.2. Alternative: using sshfs to directly access files

  • Important

    This is provided as an example only. You will need to substitute values according to your environment. 
    # Get the PID of the qemu process for the guest:
PID=`ps -eo pid,cmd | grep "qemu.*-name GuestMachine" \
| grep -v grep | awk '{print $1}'`

# Create mount point and mount guest
mkdir -p /tmp/guestmount/$PID
sshfs -o allow_other,direct_io GuestMachine:/ /tmp/guestmount/$PID

# Begin recording
perf kvm --host --guest --guestmount=/tmp/guestmount \
record -a -o perf.data

# Ctrl-C interrupts recording. Run report:
perf kvm --host --guest --guestmount=/tmp/guestmount report \
-i perf.data

# Unmount sshfs to the guest once finished:
fusermount -u /tmp/guestmount