Best practices for accurate timekeeping for Red Hat Enterprise Linux running on Red Hat Virtualization

Solution Verified - Updated -

Environment

  • Red Hat Virtualization (RHV)
  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7

Issue

  • How do I prevent time drift on Red Hat Enterprise Linux (RHEL) virtual machines under Red Hat Virtualization (RHV)?
  • How to deal with NTP issue for RHEL guests on RHV
  • What adjustment is needed for synchronizing clock of RHEL system under RHV environment?
  • If ntpd updates the system clock on the KVM host, how long before the hw-clock is updated?

Resolution

The use of NTP is always recommended on both the RHV host as well as in the RHEL or Windows guest due to timing issues present with all virtual machines.

However, a Request for Feature Enhancement (RFE) has been filed to add "time sync" functionality in a future release of qemu-guest-agent available for RHV virtual machines. The RFE is being tracked in Red Hat Bug 1291205. For more information or to also request this feature, please open a case with Red Hat Support1.

Note: If the virtual machine is RHEL 5, review these recommendations. Support for the para-virtualized clock (kvm clock) began in Red Hat Enterprise Linux 5.4, and proved to be substantially more reliable than time keeping practices in earlier kernels. This ensures more appropriate clock accuracy and IRQ interrupts. However, ntp is still the most reliable source of time.

  1. How do I open and manage a support case on the Customer Portal? ↩︎

Root Cause

It is recommended to enable NTP on the RHEL guest in addition to the RHV/KVM host because it is not guaranteed that NTP will keep the hwclock and the system clock in sync on the host. See "Automatic Hardware Clock Synchronization By the Kernel" in hwclock(8).

If the RHV/KVM host's time is updated by NTP, how long before the system clock (kvm-clock) on the VM is updated?

Here is the code flow according to the pvclock_vcpu_time_info:

struct pvclock_vcpu_time_info {
        u32   version;
        u32   pad0;
        u64   tsc_timestamp;
        u64   system_time;
        u32   tsc_to_system_mul;
        s8    tsc_shift;
        u8    flags;
        u8    pad[2];
} __attribute__((__packed__));

This structure is a "best guess", an ideal approximation. In particular, the hwclock of the guest might not be updated at all when the host ntpd applies changes. And pvclock_vcpu_time_info gets updated on events, such as when the vcpu is migrated from one pcpu to another, and also at least at a rate of every 5 minutes, i.e. kvm-clock on all vcpus is updated every 5 minutes (unless the host is running the realtime kernel, in which case that update is disabled to avoid the periodic latencies it induces).

This is why Red Hat recommends running NTP in the virtual machine if accurate timekeeping is required.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments