About behavior of ntp
Host:XenServer6.2
Guest
OS:Red Hat Enterprise Linux Server release 6.2 (Santiago)
kernel:2.6.32-220.el6.i686
ntp:ntp-4.2.6p5-2.el6_6.i686
We operate a virtual machine on XenServer.
Guest is you are NTP synchronization, but 'offset' is largely shifted.
1 hour = + 10 seconds
Deviation of 'jitter' is also large.
Guest of the NTP that are running on the same host is operating normally.
Configuration file uses the file of essentially the same content.
Ran the command.
・echo "" > /var/lib/ntp/drift
・/etc/init.d/ntp stop
・ntpdate Host
・hwclock -w
・/etc/init.d/ntp start
・ntpq -p
But the problem is not resolved.
Possible causes What are?
Also it had been output in '/var/log/messages'
'freq_set kernel -3310.291'
What are cause this value is high?
Restart want to avoid.
Responses
Hello
You say the config files are the same as other guests which are working OK, but what about on the GRUB command line?
The Deployment Guide says "para-virtualized clocks should be provided by the virtualization application in use" . See Managing the Time on Virtual Machines
Now I realise that when you wrote "Although there was described to tell me your site" you were referring to KVM guest timing management
My colleague translated the Japanese into English as follows:
I believe that the deviation of the serve time is caused by
the server time going faster.
According to 'Managing the Time on Virtual Machines',
the configuration of /etc/sysconfig/cpuspeed should be changed (to solve this problem).
Can I correct the server time going faster by changing the configuration of /etc/sysconfig/cpuspeed?
According to the description of "/etc/sysconfig/cpuspeed",
this configuration should not be changed.
I have asked a colleague for his advice, but in the meantime, consider this:
Editing /etc/sysconfig/cpuspeed is mentioned in the Virtualization Host Configuration and Guest Installation Guide here Configuring hosts without a constant Time Stamp Counter where it also says "These instructions are for AMD revision F CPUs only. " and "Disable cpufreq (only necessary on hosts without the constant_tsc) ".
Do those conditions apply in your case? i.e. you have no TSC? To find out:
$ cat /proc/cpuinfo | grep constant_tsc
If not, i.e. you do have a TSC, then please check if your processor is listed in Required parameters for Red Hat Enterprise Linux guests and follow the steps.
HTH
My colleague translated the Japanese into English as follows:
$ cat /proc/cpuinfo | grep constant_tsc
As a result, I was able to confirm 'constant_tsc'.
Next, as for a guest parameter required for Red Hat Enterprise Linux,
my server processor is xeon64/intel64.
This is not included in 'Table 14.1. Kernel parameter requirements'.
Also, I am using 'Red Hat Enterprise Linux 6'.
It is explained (i don't know where) that a guest is not affected on RHEL6.
Is this manual not applicable to my server?
So I think you have a TSC on the host (I googled and I think that is true for Xeon 64 CPU)
and there is no specific advice in the table for this host's CPU.
Reading here:
http://support.ntp.org/bin/view/Support/KnownOsIssues
I see: Kernel 2.6 Mis-Detecting CPU TSC Frequency
TSC frequency depends on the core frequency, which can be changed dynamically according to the load. So power saving can actually influence the clock if the OS fails to closely monitor TSC speed changes; one symptom for this kind of problem is a system clock with a very high drift of a few seconds per hour.
You can use
cat /sys/devices/system/clocksource/clocksource0/available_clocksource
to find out what clock sources are available on your system; 'hpet' seems to be preferable over 'acpi_pm' which in turn seems to better than 'pit'. Try the available ones in order to get a stable clock.
You said you wanted to avoid a reboot if possible. You can read how to temporarily change clock source setting here: How to change the clock source in the system
Translation:
I am trying to change clocksource.
I understand that an output 'kernel: Switching to clocksource hpet'
in /var/log/messages implies that the change to clocksource has been reflected.
Am I right here?
Also, I have changed the clocksource from tsc to hpet.
Does this affect something other than timestamp?
On my server, it was as follows;
cat /sys/devices/system/clocksource/clocksource0/current_clocksource
tsc
Let's assume that the clock has lost or gained time for some reason when booting, and that the clock can be corrected by rebooting.
In this case, if I change the clocksource from tsc to hpet,
then change the clocksource back to tsc,
does this improve the situation?
The messages in /var/log/messages are an indication that something has happened. The output from cat /sys/devices/system/clocksource/clocksource0/current_clocksource is evidence of what the system is currently using. So it is better to trust that and not rely on the logs.
The purpose of changing the clock source is to try and find a more stable clock, so I cannot think why this would change anything other than the accuracy of the time stamps.
I do not think changing the clock source from tsc to hpet and back again is going to help. I think you should make one change and then monitor the accuracy of the time over, say, 100 seconds, before making another change and testing again over the same time period.
Translation:
By the way,
why did only one virtual server, among many with the same configuration,
had an incorrect time?
(The clock gains time even though it is synchronized via ntp.)
Do you know the cause of this problem?
Has there been a similar case before?
What can I do to find a possible cause?
About the jitter problem
The Deployment Guide says "jitter: difference of successive time values from server (high jitter could be due to an unstable clock or, more likely, poor network performance) "
then later it says to use the burst option against local time servers to improve the average quality of the time-offset calculations. Deployment Guide: Configuring the Burst Option
I presume you have the host server listed as an NTP server in your VM's config file. If so, can you add the burst option?
I have been told that in your case this will not help (but will not hurt). My colleague says:
"If the clock drifts so much that ntpd can't lock it, it will report a large offset and also large jitter, but the actual jitter may not really be large, it's just how ntpd works with offsets."
So finding a more stable clock is the priority then.
As usual, below is the translation.
The symptom that the clock on virtual server gains so much time that
even a perfectly working ntpd can not correct it was resolved by
rebooting the virtual machine, so it seems.
To summarize my observation so far,
1. Clock on virtual server gains so much time that ntp(slew) can not correct it.
2. Configuration is more or less same as other working servers.
3. Setting the clock with ntpdate or hwclock commands, and rebooting ntpd do not help.
4. ntp.conf has slew option only
5. Clock source has not been changed.
6. After rebooting the virtual server, the symptom to gain time seemed to have been resolved.
Has there been a similar case before?
What do you think is a cause of this problem?
Your help is much appreciated.
-
Clock on virtual server gains so much time that ntp(slew) can not correct it.
Ans: This suggests that the clock source is not working properly, can you ask the virtualization software provider for advice? -
Configuration is more or less same as other working servers.
- Setting the clock with ntpdate or hwclock commands, and rebooting ntpd do not help.
-
ntp.conf has slew option only
-
Clock source has not been changed.
Ans: It would be nice if you could experiment, on a test VM, with different clock sources to see how stable they are. If possible, test when host is under light load and normal working load, or create an artificial heavy load, to see if that makes a difference. -
After rebooting the virtual server, the symptom to gain time seemed to have been resolved.
Ans: Perhaps something was recalibrated. We were hoping you would have time to experiment with other clock sources. You could also ask the virtualization software provider about this.
Has there been a similar case before?
ANS: Not that I can find, please ask the virtualization software provider.
What do you think is a cause of this problem?
Ans: We do not know. It might be worthwhile to check if the virtualization software provider has any detailed information on their clock sources, how they work, known issues and so on.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
