About behavior of ntp

Latest response

Host:XenServer6.2

Guest
OS:Red Hat Enterprise Linux Server release 6.2 (Santiago)
kernel:2.6.32-220.el6.i686
ntp:ntp-4.2.6p5-2.el6_6.i686

We operate a virtual machine on XenServer.
Guest is you are NTP synchronization, but 'offset' is largely shifted.
1 hour = + 10 seconds
Deviation of 'jitter' is also large.

Guest of the NTP that are running on the same host is operating normally.
Configuration file uses the file of essentially the same content.

Ran the command.
・echo "" > /var/lib/ntp/drift
・/etc/init.d/ntp stop
・ntpdate Host
・hwclock -w
・/etc/init.d/ntp start
・ntpq -p

But the problem is not resolved.
Possible causes What are?

Also it had been output in '/var/log/messages'
'freq_set kernel -3310.291'
What are cause this value is high?

Restart want to avoid.

Responses

Hello

You say the config files are the same as other guests which are working OK, but what about on the GRUB command line?

The Deployment Guide says "para-virtualized clocks should be provided by the virtualization application in use" . See Managing the Time on Virtual Machines

Reply Thank you.

The cause of the deviation of the server's time,
ntp it is that would go faster time than of the destination.
Although there was described to tell me your site,
/ etc / sysconfig / cpuspeed By rewriting,
Will you become measures to that time of the server goes faster?

The description of the /etc/sysconfig/cpuspeed,
While this not change the setting was written with good.

I am sorry, I cannot understand, please post your comment in Mandarin.

Thank you

Now I realise that when you wrote "Although there was described to tell me your site" you were referring to KVM guest timing management

Sorry for the poor English
Is Japanese

サーバの時刻ずれの原因はサーバの時間の進みが速いことにあると思いますが、
教えていただいた'Managing the Time on Virtual Machines'には、
/etc/sysconfig/cpuspeedの設定を変更するということが書かれていたと思います。
/etc/sysconfig/cpuspeedの設定を変えることでサーバの時間の進みを修正できますか?
/etc/sysconfig/cpuspeecdの説明書きには、
この設定は変更しない方が良いと書かれていました。

My colleague translated the Japanese into English as follows:

I believe that the deviation of the serve time is caused by
the server time going faster.

According to 'Managing the Time on Virtual Machines',
the configuration of /etc/sysconfig/cpuspeed should be changed (to solve this problem).

Can I correct the server time going faster by changing the configuration of /etc/sysconfig/cpuspeed?

According to the description of "/etc/sysconfig/cpuspeed",
this configuration should not be changed.

Yes, you are right.

Thank you

I have asked a colleague for his advice, but in the meantime, consider this:

Editing /etc/sysconfig/cpuspeed is mentioned in the Virtualization Host Configuration and Guest Installation Guide here Configuring hosts without a constant Time Stamp Counter where it also says "These instructions are for AMD revision F CPUs only. " and "Disable cpufreq (only necessary on hosts without the constant_tsc) ".

Do those conditions apply in your case? i.e. you have no TSC? To find out:

$ cat /proc/cpuinfo | grep constant_tsc

If not, i.e. you do have a TSC, then please check if your processor is listed in Required parameters for Red Hat Enterprise Linux guests and follow the steps.

HTH

Thank you reply.

$ cat /proc/cpuinfo | grep constant_tsc

結果、'constant_tsc'を確認できました。
次にRed Hat Enterprise Linux に必要なゲストパラメータですが、
私のサーバのプロセッサは、xeon64/intel64です。
'Table 14.1. Kernel parameter requirements'には該当しません。
また、'Red Hat Enterprise Linux 6'を使用しています。
RHEL6ではゲストに影響は無いと書かれてあります。

私の使用しているサーバはこのマニュアルには該当しないのでしょうか?

My colleague translated the Japanese into English as follows:

$ cat /proc/cpuinfo | grep constant_tsc

As a result, I was able to confirm 'constant_tsc'.

Next, as for a guest parameter required for Red Hat Enterprise Linux,
my server processor is xeon64/intel64.
This is not included in 'Table 14.1. Kernel parameter requirements'.
Also, I am using 'Red Hat Enterprise Linux 6'.

It is explained (i don't know where) that a guest is not affected on RHEL6.

Is this manual not applicable to my server?

So I think you have a TSC on the host (I googled and I think that is true for Xeon 64 CPU)

and there is no specific advice in the table for this host's CPU.

Reading here:
http://support.ntp.org/bin/view/Support/KnownOsIssues

I see: Kernel 2.6 Mis-Detecting CPU TSC Frequency

TSC frequency depends on the core frequency, which can be changed dynamically according to the load. So power saving can actually influence the clock if the OS fails to closely monitor TSC speed changes; one symptom for this kind of problem is a system clock with a very high drift of a few seconds per hour.

You can use

cat /sys/devices/system/clocksource/clocksource0/available_clocksource

to find out what clock sources are available on your system; 'hpet' seems to be preferable over 'acpi_pm' which in turn seems to better than 'pit'. Try the available ones in order to get a stable clock.

You said you wanted to avoid a reboot if possible. You can read how to temporarily change clock source setting here: How to change the clock source in the system

Thank you reply, and, I'm sorry would reply is delayed.

clocksource の変更を試しています。
/var/log/messagesに'kernel: Switching to clocksource hpet'と出力されたら、
clocksource変更が反映されているという認識でよいでしょうか?
また、clocksourceをtscからhpetへ変更しましたが、
timestamp以外への影響はありますでしょうか?

私のサーバでは、

cat /sys/devices/system/clocksource/clocksource0/current_clocksource

tsc

でしたが、
たとえば起動時に何らかの不具合が起きて時刻が狂い、
rebootすれば症状が改善されると仮定します。
この場合にclocksourceをtscからhpetへ変更し、
その後にclocksourceをtscへ戻したら症状が改善される可能性はありますでしょうか?

Translation:

I am trying to change clocksource.
I understand that an output 'kernel: Switching to clocksource hpet'
in /var/log/messages implies that the change to clocksource has been reflected.
Am I right here?
Also, I have changed the clocksource from tsc to hpet.
Does this affect something other than timestamp?

On my server, it was as follows;
cat /sys/devices/system/clocksource/clocksource0/current_clocksource

tsc

Let's assume that the clock has lost or gained time for some reason when booting, and that the clock can be corrected by rebooting.
In this case, if I change the clocksource from tsc to hpet,
then change the clocksource back to tsc,
does this improve the situation?

The messages in /var/log/messages are an indication that something has happened. The output from cat /sys/devices/system/clocksource/clocksource0/current_clocksource is evidence of what the system is currently using. So it is better to trust that and not rely on the logs.

The purpose of changing the clock source is to try and find a more stable clock, so I cannot think why this would change anything other than the accuracy of the time stamps.

I do not think changing the clock source from tsc to hpet and back again is going to help. I think you should make one change and then monitor the accuracy of the time over, say, 100 seconds, before making another change and testing again over the same time period.

話は変わりますが、

そもそも何故複数台の同じ設定がされている仮想サーバのうち、
1台だけが時刻ずれを起こしたのでしょうか?
(ntpで同期していても時刻が早く進んでしまう事象)
この原因と、同様の事象が起きたことはないか教えてもらえますか?
また原因を調べる方法があれば具体的に教えて頂けると助かります。

Translation:
By the way,
why did only one virtual server, among many with the same configuration,
had an incorrect time?
(The clock gains time even though it is synchronized via ntp.)
Do you know the cause of this problem?
Has there been a similar case before?
What can I do to find a possible cause?

About the jitter problem

The Deployment Guide says "jitter: difference of successive time values from server (high jitter could be due to an unstable clock or, more likely, poor network performance) "

then later it says to use the burst option against local time servers to improve the average quality of the time-offset calculations. Deployment Guide: Configuring the Burst Option

I presume you have the host server listed as an NTP server in your VM's config file. If so, can you add the burst option?

I have been told that in your case this will not help (but will not hurt). My colleague says:
"If the clock drifts so much that ntpd can't lock it, it will report a large offset and also large jitter, but the actual jitter may not really be large, it's just how ntpd works with offsets."

So finding a more stable clock is the priority then.

Thank you reply, and, I'm sorry would reply is delayed.

いろいろとアドバイスをしていただき、
ありがとうございます。
結局は仮想サーバを再起動することにより、
"ntpdが正常に起動していても修正できないほど仮想サーバの時刻が速く進んでしまう"
という現象は、現在のところ解消されたように思われます。

ここまでの経過を整理すると、
1、仮想サーバの時刻がntp(slew)の修正以上に早く進んでしまう。
2、ほかの正常なサーバと設定はほぼ同じ。
3、ntpdateコマンドやhwclockコマンドで時計を合わせ、ntpdの再起動をしても意味がない。
4、ntp.confにはslew以外のオプションは設定していない。
5、クロックソースは変更していない。
6、仮想サーバの再起動後は時刻が早く進んでしまう現象は解消されたと思える。

この現象が過去に例がないか?
なにが原因と思われるか?
回答して頂けると助かります。

As usual, below is the translation.

The symptom that the clock on virtual server gains so much time that
even a perfectly working ntpd can not correct it was resolved by
rebooting the virtual machine, so it seems.

To summarize my observation so far,
1. Clock on virtual server gains so much time that ntp(slew) can not correct it.
2. Configuration is more or less same as other working servers.
3. Setting the clock with ntpdate or hwclock commands, and rebooting ntpd do not help.
4. ntp.conf has slew option only
5. Clock source has not been changed.
6. After rebooting the virtual server, the symptom to gain time seemed to have been resolved.

Has there been a similar case before?
What do you think is a cause of this problem?

Your help is much appreciated.

  1. Clock on virtual server gains so much time that ntp(slew) can not correct it.
    Ans: This suggests that the clock source is not working properly, can you ask the virtualization software provider for advice?

  2. Configuration is more or less same as other working servers.

  3. Setting the clock with ntpdate or hwclock commands, and rebooting ntpd do not help.
  4. ntp.conf has slew option only

  5. Clock source has not been changed.
    Ans: It would be nice if you could experiment, on a test VM, with different clock sources to see how stable they are. If possible, test when host is under light load and normal working load, or create an artificial heavy load, to see if that makes a difference.

  6. After rebooting the virtual server, the symptom to gain time seemed to have been resolved.
    Ans: Perhaps something was recalibrated. We were hoping you would have time to experiment with other clock sources. You could also ask the virtualization software provider about this.

Has there been a similar case before?
ANS: Not that I can find, please ask the virtualization software provider.

What do you think is a cause of this problem?
Ans: We do not know. It might be worthwhile to check if the virtualization software provider has any detailed information on their clock sources, how they work, known issues and so on.

Thank you answer the question.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.