Chapter 13. Configuring time synchronization
Accurate timekeeping in an IT environment is important. A consistent time across all network devices improves the traceability of log files and certain protocols rely on synchronized clocks. For example, Kerberos uses time stamps to prevent replay attacks.
13.1. Using the Chrony suite to configure NTP
Accurate timekeeping is important for several reasons in IT. In networking for example, accurate time stamps in packets and logs are required. In Linux systems, the
NTP protocol is implemented by a daemon running in user space.
The user space daemon updates the system clock running in the kernel. The system clock can keep time by using various clock sources. Usually, the Time Stamp Counter (TSC) is used. The TSC is a CPU register which counts the number of cycles since it was last reset. It is very fast, has a high resolution, and there are no interruptions.
Starting with Red Hat Enterprise Linux 8, the
NTP protocol is implemented by the
chronyd daemon, available from the repositories in the
The following sections describe how to use the chrony suite to configure NTP.
13.1.1. Introduction to chrony suite
chrony is an implementation of the
Network Time Protocol (NTP). You can use chrony:
To synchronize the system clock with
- To synchronize the system clock with a reference clock, for example a GPS receiver
- To synchronize the system clock with a manual time input
NTPv4(RFC 5905)server or peer to provide a time service to other computers in the network
chrony performs well in a wide range of conditions, including intermittent network connections, heavily congested networks, changing temperatures (ordinary computer clocks are sensitive to temperature), and systems that do not run continuously, or run on a virtual machine.
Typical accuracy between two machines synchronized over the Internet is within a few milliseconds, and for machines on a LAN within tens of microseconds. Hardware timestamping or a hardware reference clock may improve accuracy between two machines synchronized to a sub-microsecond level.
chrony consists of
chronyd, a daemon that runs in user space, and chronyc, a command line program which can be used to monitor the performance of
chronyd and to change various operating parameters when it is running.
The chrony daemon,
chronyd, can be monitored and controlled by the command line utility chronyc. This utility provides a command prompt which allows entering a number of commands to query the current state of
chronyd and make changes to its configuration. By default,
chronyd accepts only commands from a local instance of chronyc, but it can be configured to accept monitoring commands also from remote hosts. The remote access should be restricted.
13.1.2. Using chronyc to control chronyd
You can control
chronyd using the chronyc command line utility.
To make changes to the local instance of
chronydusing the command line utility chronyc in interactive mode, enter the following command as
chronyc must run as
rootif some of the restricted commands are to be used.
The chronyc command prompt will be displayed as follows:
To list all of the commands, type
Alternatively, the utility can also be invoked in non-interactive command mode if called together with a command as follows:
Changes made using chronyc are not permanent, they will be lost after a
chronyd restart. For permanent changes, modify
13.2. Using Chrony
The following sections describe how to install, start, and stop
chronyd, and how to check if
chrony is synchronized. Sections also describe how to manually adjust System Clock.
13.2.1. Managing chrony
The following procedure describes how to install, start, stop, and check the status of
The chrony suite is installed by default on Red Hat Enterprise Linux. To ensure that it is, run the following command as
# dnf install chrony
The default location for the chrony daemon is
/usr/sbin/chronyd. The command line utility will be installed to
To check the status of
chronyd, issue the following command:
systemctl status chronydchronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled) Active: active (running) since Wed 2013-06-12 22:23:16 CEST; 11h ago
chronyd, issue the following command as
# systemctl start chronyd
chronydstarts automatically at system start, issue the following command as
# systemctl enable chronyd
chronyd, issue the following command as
# systemctl stop chronyd
chronydfrom starting automatically at system start, issue the following command as
# systemctl disable chronyd
13.2.2. Checking if chrony is synchronized
The following procedure describes how to check if chrony is synchronized with the use of the
To check chrony tracking, issue the following command:
chronyc trackingReference ID : CB00710F (ntp-server.example.net) Stratum : 3 Ref time (UTC) : Fri Jan 27 09:49:17 2017 System time : 0.000006523 seconds slow of NTP time Last offset : -0.000006747 seconds RMS offset : 0.000035822 seconds Frequency : 3.225 ppm slow Residual freq : 0.000 ppm Skew : 0.129 ppm Root delay : 0.013639022 seconds Root dispersion : 0.001100737 seconds Update interval : 64.2 seconds Leap status : Normal
The sources command displays information about the current time sources that
chronydis accessing. To check chrony sources, issue the following command:
$ chronyc sources 210 Number of sources = 3 MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== #* GPS0 0 4 377 11 -479ns[ -621ns] /- 134ns ^? a.b.c 2 6 377 23 -923us[ -924us] +/- 43ms ^ d.e.f 1 6 377 21 -2629us[-2619us] +/- 86ms
You can specify the optional
-vargument to print more verbose information. In this case, extra caption lines are shown as a reminder of the meanings of the columns.
sourcestatscommand displays information about the drift rate and offset estimation process for each of the sources currently being examined by
chronyd. To check chrony source statistics, issue the following command:
chronyc sourcestats210 Number of sources = 1 Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev =============================================================================== abc.def.ghi 11 5 46m -0.001 0.045 1us 25us
The optional argument
-vcan be specified, meaning verbose. In this case, extra caption lines are shown as a reminder of the meanings of the columns.
13.2.3. Manually adjusting the System Clock
The following procedure describes how to manually adjust the System Clock.
To step the system clock immediately, bypassing any adjustments in progress by slewing, issue the following command as
# chronyc makestep
rtcfile directive is used, the real-time clock should not be manually adjusted. Random adjustments would interfere with chrony's need to measure the rate at which the real-time clock drifts.
13.2.4. Disabling a chrony dispatcher script
chrony dispatcher script manages the online and offline state of the NTP servers. As a system administrator, you can disable the dispatcher script to keep
chronyd polling the servers constantly.
If you enable NetworkManager on your system to manage networking configuration, the NetworkManager executes the
chrony dispatcher script during interface reconfiguration, stop or start operations. However, if you configure certain interfaces or routes outside of NetworkManager, you can encounter the following situation:
- The dispatcher script might run when no route to the NTP servers exists, causing the NTP servers to switch to the offline state.
- If you establish the route later, the script does not run again by default, and the NTP servers remain in the offline state.
To ensure that
chronyd can synchronize with your NTP servers, which have separately managed interfaces, disable the dispatcher script.
- You installed NetworkManager on your system and enabled it.
- Root access
To disable the
chronydispatcher script, create a symlink to
# ln -s /dev/null /etc/NetworkManager/dispatcher.d/20-chrony-onofflineNote
After this change, the NetworkManager cannot execute the dispatcher script, and the NTP servers remain in the online state at all times.
13.2.5. Setting up chrony for a system in an isolated network
For a network that is never connected to the Internet, one computer is selected to be the primary timeserver. The other computers are either direct clients of the server, or clients of clients. On the server, the drift file must be manually set with the average rate of drift of the system clock. If the server is rebooted, it will obtain the time from surrounding systems and calculate an average to set its system clock. Thereafter it resumes applying adjustments based on the drift file. The drift file will be updated automatically when the
settime command is used.
The following procedure describes how to set up chrony for a system in an isolated network.
On the system selected to be the server, using a text editor running as
driftfile /var/lib/chrony/drift commandkey 1 keyfile /etc/chrony.keys initstepslew 10 client1 client3 client6 local stratum 8 manual allow 192.0.2.0
192.0.2.0is the network or subnet address from which the clients are allowed to connect.
On the systems selected to be direct clients of the server, using a text editor running as
root, edit the
server ntp1.example.net driftfile /var/lib/chrony/drift logdir /var/log/chrony log measurements statistics tracking keyfile /etc/chrony.keys commandkey 24 local stratum 10 initstepslew 20 ntp1.example.net allow 192.0.2.123
192.0.2.123is the address of the server, and
ntp1.example.netis the host name of the server. Clients with this configuration will resynchronize with the server if it restarts.
On the client systems which are not to be direct clients of the server, the
/etc/chrony.conf file should be the same except that the
allow directives should be omitted.
In an isolated network, you can also use the
local directive that enables a local reference mode, which allows
chronyd operating as an
NTP server to appear synchronized to real time, even when it was never synchronized or the last update of the clock happened a long time ago.
To allow multiple servers in the network to use the same local configuration and to be synchronized to one another, without confusing clients that poll more than one server, use the
orphan option of the
local directive which enables the orphan mode. Each server needs to be configured to poll all other servers with
local. This ensures that only the server with the smallest reference ID has the local reference active and other servers are synchronized to it. When the server fails, another one will take over.
13.2.6. Configuring remote monitoring access
chronyc can access
chronyd in two ways:
- Internet Protocol, IPv4 or IPv6.
Unix domain socket, which is accessible locally by the
By default, chronyc connects to the Unix domain socket. The default path is
/var/run/chrony/chronyd.sock. If this connection fails, which can happen for example when chronyc is running under a non-root user, chronyc tries to connect to 127.0.0.1 and then ::1.
Only the following monitoring commands, which do not affect the behavior of
chronyd, are allowed from the network:
- manual list
The set of hosts from which
chronyd accepts these commands can be configured with the
cmdallow directive in the configuration file of
chronyd, or the
cmdallow command in chronyc. By default, the commands are accepted only from localhost (127.0.0.1 or ::1).
All other commands are allowed only through the Unix domain socket. When sent over the network,
chronyd responds with a
Not authorised error, even if it is from localhost.
The following procedure describes how to access chronyd remotely with chronyc.
Allow access from both IPv4 and IPv6 addresses by adding the following to the
Allow commands from the remote IP address, network, or subnet by using the
Add the following content to the
Open port 323 in the firewall to connect from a remote system:
# firewall-cmd --zone=public --add-port=323/udp
Optionally, you can open port 323 permanently using the
# firewall-cmd --permanent --zone=public --add-port=323/udp
If you opened port 323 permanently, reload the firewall configuration:
13.2.7. Managing time synchronization using RHEL System Roles
You can manage time synchronization on multiple target machines using the
timesync role. The
timesync role installs and configures an NTP or PTP implementation to operate as an NTP or PTP client to synchronize the system clock.
timesync role replaces the configuration of the given or detected provider service on the managed host. Previous settings are lost, even if they are not specified in the role variables. The only preserved setting is the choice of provider if the
timesync_ntp_provider variable is not defined.
The following example shows how to apply the
timesync role in a situation with just one pool of servers.
Example 13.1. An example playbook applying the timesync role for a single pool of servers
--- - hosts: timesync-test vars: timesync_ntp_servers: - hostname: 2.rhel.pool.ntp.org pool: yes iburst: yes roles: - rhel-system-roles.timesync
For a detailed reference on
timesync role variables, install the
rhel-system-roles package, and see the
README.html files in the
13.2.8. Additional resources
- Frequently Asked Questions
13.3. Chrony with HW timestamping
Hardware timestamping is a feature supported in some Network Interface Controller (NICs) which provides accurate timestamping of incoming and outgoing packets.
NTP timestamps are usually created by the kernel and chronyd with the use of the system clock. However, when HW timestamping is enabled, the NIC uses its own clock to generate the timestamps when packets are entering or leaving the link layer or the physical layer. When used with
NTP, hardware timestamping can significantly improve the accuracy of synchronization. For best accuracy, both
NTP servers and
NTP clients need to use hardware timestamping. Under ideal conditions, a sub-microsecond accuracy may be possible.
Another protocol for time synchronization that uses hardware timestamping is
PTP relies on assistance in network switches and routers. If you want to reach the best accuracy of synchronization, use
PTP on networks that have switches and routers with
PTP support, and prefer
NTP on networks that do not have such switches and routers.
The following sections describe how to:
- Verify support for hardware timestamping
- Enable hardware timestamping
- Configure client polling interval
- Enable interleaved mode
- Configure server for large number of clients
- Verify hardware timestamping
- Configure PTP-NTP bridge
13.3.1. Verifying support for hardware timestamping
To verify that hardware timestamping with
NTP is supported by an interface, use the
ethtool -T command. An interface can be used for hardware timestamping with
ethtool lists the
SOF_TIMESTAMPING_TX_SOFTWARE capabilities and also the
HWTSTAMP_FILTER_ALL filter mode.
Example 13.2. Verifying support for hardware timestamping on a specific interface
# ethtool -T eth0
Timestamping parameters for eth0: Capabilities: hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE) software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE) hardware-receive (SOF_TIMESTAMPING_RX_HARDWARE) software-receive (SOF_TIMESTAMPING_RX_SOFTWARE) software-system-clock (SOF_TIMESTAMPING_SOFTWARE) hardware-raw-clock (SOF_TIMESTAMPING_RAW_HARDWARE) PTP Hardware Clock: 0 Hardware Transmit Timestamp Modes: off (HWTSTAMP_TX_OFF) on (HWTSTAMP_TX_ON) Hardware Receive Filter Modes: none (HWTSTAMP_FILTER_NONE) all (HWTSTAMP_FILTER_ALL) ptpv1-l4-sync (HWTSTAMP_FILTER_PTP_V1_L4_SYNC) ptpv1-l4-delay-req (HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ) ptpv2-l4-sync (HWTSTAMP_FILTER_PTP_V2_L4_SYNC) ptpv2-l4-delay-req (HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ) ptpv2-l2-sync (HWTSTAMP_FILTER_PTP_V2_L2_SYNC) ptpv2-l2-delay-req (HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ) ptpv2-event (HWTSTAMP_FILTER_PTP_V2_EVENT) ptpv2-sync (HWTSTAMP_FILTER_PTP_V2_SYNC) ptpv2-delay-req (HWTSTAMP_FILTER_PTP_V2_DELAY_REQ)
13.3.2. Enabling hardware timestamping
To enable hardware timestamping, use the
hwtimestamp directive in the
/etc/chrony.conf file. The directive can either specify a single interface, or a wildcard character can be used to enable hardware timestamping on all interfaces that support it. Use the wildcard specification in case that no other application, like ptp4l from the
linuxptp package, is using hardware timestamping on an interface. Multiple
hwtimestamp directives are allowed in the chrony configuration file.
Example 13.3. Enabling hardware timestamping by using the hwtimestamp directive
hwtimestamp eth0 hwtimestamp eth1 hwtimestamp *
13.3.3. Configuring client polling interval
The default range of a polling interval (64-1024 seconds) is recommended for servers on the Internet. For local servers and hardware timestamping, a shorter polling interval needs to be configured in order to minimize offset of the system clock.
The following directive in
/etc/chrony.conf specifies a local
NTP server using one second polling interval:
server ntp.local minpoll 0 maxpoll 0
13.3.4. Enabling interleaved mode
NTP servers that are not hardware
NTP appliances, but rather general purpose computers running a software
NTP implementation, like chrony, will get a hardware transmit timestamp only after sending a packet. This behavior prevents the server from saving the timestamp in the packet to which it corresponds. In order to enable
NTP clients receiving transmit timestamps that were generated after the transmission, configure the clients to use the
NTP interleaved mode by adding the
xleave option to the server directive in
server ntp.local minpoll 0 maxpoll 0 xleave
13.3.5. Configuring server for large number of clients
The default server configuration allows a few thousands of clients at most to use the interleaved mode concurrently. To configure the server for a larger number of clients, increase the
clientloglimit directive in
/etc/chrony.conf. This directive specifies the maximum size of memory allocated for logging of clients' access on the server:
13.3.6. Verifying hardware timestamping
To verify that the interface has successfully enabled hardware timestamping, check the system log. The log should contain a message from
chronyd for each interface with successfully enabled hardware timestamping.
Example 13.4. Log messages for interfaces with enabled hardware timestamping
chronyd: Enabled HW timestamping on eth0 chronyd: Enabled HW timestamping on eth1
chronyd is configured as an
NTP client or peer, you can have the transmit and receive timestamping modes and the interleaved mode reported for each
NTP source by the
chronyc ntpdata command:
Example 13.5. Reporting the transmit, receive timestamping and interleaved mode for each NTP source
# chronyc ntpdata
Remote address : 203.0.113.15 (CB00710F) Remote port : 123 Local address : 203.0.113.74 (CB00714A) Leap status : Normal Version : 4 Mode : Server Stratum : 1 Poll interval : 0 (1 seconds) Precision : -24 (0.000000060 seconds) Root delay : 0.000015 seconds Root dispersion : 0.000015 seconds Reference ID : 47505300 (GPS) Reference time : Wed May 03 13:47:45 2017 Offset : -0.000000134 seconds Peer delay : 0.000005396 seconds Peer dispersion : 0.000002329 seconds Response time : 0.000152073 seconds Jitter asymmetry: +0.00 NTP tests : 111 111 1111 Interleaved : Yes Authenticated : No TX timestamping : Hardware RX timestamping : Hardware Total TX : 27 Total RX : 27 Total valid RX : 27
Example 13.6. Reporting the stability of NTP measurements
# chronyc sourcestats
With hardware timestamping enabled, stability of
NTP measurements should be in tens or hundreds of nanoseconds, under normal load. This stability is reported in the
Std Dev column of the output of the
chronyc sourcestats command:
210 Number of sources = 1 Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev ntp.local 12 7 11 +0.000 0.019 +0ns 49ns
13.3.7. Configuring PTP-NTP bridge
If a highly accurate Precision Time Protocol (
PTP) primary timeserver is available in a network that does not have switches or routers with
PTP support, a computer may be dedicated to operate as a
PTP client and a stratum-1
NTP server. Such a computer needs to have two or more network interfaces, and be close to the primary timeserver or have a direct connection to it. This will ensure highly accurate synchronization in the network.
Configure the ptp4l and phc2sys programs from the
linuxptp packages to use one interface to synchronize the system clock using
chronyd to provide the system time using the other interface:
Example 13.7. Configuring chronyd to provide the system time using the other interface
bindaddress 203.0.113.74 hwtimestamp eth1 local stratum 1
13.4. Overview of Network Time Security (NTS) in chrony
Network Time Security (NTS) is an authentication mechanism for Network Time Protocol (NTP), designed to scale substantial clients. It verifies that the packets received from the server machines are unaltered while moving to the client machine. Network Time Security (NTS) includes a Key Establishment (NTS-KE) protocol that automatically creates the encryption keys used between the server and its clients.
13.4.1. Enabling Network Time Security (NTS) in the client configuration file
By default, Network Time Security (NTS) is not enabled. You can enable NTS in the
/etc/chrony.conf. For that, perform the following steps:
- Server with the NTS support
In the client configuration file:
Specify the server with the
ntsoption in addition to the recommended
For example: server time.example.com iburst nts server nts.netnod.se iburst nts server ptbtime1.ptb.de iburst nts
To avoid repeating the Network Time Security-Key Establishment (NTS-KE) session during system boot, add the following line to
chrony.conf, if it is not present:
To disable synchronization with Network Time Protocol (NTP) servers provided by
DHCP, comment out or remove the following line in
chrony.conf, if it is present:
- Save your changes.
systemctl restart chronyd
Verify if the
NTSkeys were successfully established:
# chronyc -N authdata Name/IP address Mode KeyID Type KLen Last Atmp NAK Cook CLen ================================================================ time.example.com NTS 1 15 256 33m 0 0 8 100 nts.sth1.ntp.se NTS 1 15 256 33m 0 0 8 100 nts.sth2.ntp.se NTS 1 15 256 33m 0 0 8 100
KLenshould have non-zero values. If the value is zero, check the system log for error messages from
Verify the client is making NTP measurements:
# chronyc -N sources MS Name/IP address Stratum Poll Reach LastRx Last sample ========================================================= time.example.com 3 6 377 45 +355us[ +375us] +/- 11ms nts.sth1.ntp.se 1 6 377 44 +237us[ +237us] +/- 23ms nts.sth2.ntp.se 1 6 377 44 -170us[ -170us] +/- 22ms
Reachcolumn should have a non-zero value; ideally 377. If the value rarely gets 377 or never gets to 377, it indicates that NTP requests or responses are getting lost in the network.
13.4.2. Enabling Network Time Security (NTS) on the server
If you run your own Network Time Protocol (NTP) server, you can enable the server Network Time Security (NTS) support to facilitate its clients to synchronize securely.
If the NTP server is a client of other servers, that is, it is not a Stratum 1 server, it should use NTS or symmetric key for its synchronization.
Server private key in
Server certificate with required intermediate certificates in
Specify the private key and the certificate file in
For example: ntsserverkey /etc/pki/tls/private/<ntp-server.example.net>.key ntsservercert /etc/pki/tls/certs/<ntp-server.example.net>.crt
Ensure that both the key and certificate files are readable by the chrony system user, by setting the group ownership.
For example: chown :chrony /etc/pki/tls/*/<ntp-server.example.net>.*
ntsdumpdir /var/lib/chronydirective is present in the
systemctl restart chronydImportant
If the server has a firewall, it needs to allow both the
TCP 4460ports for NTP and Network Time Security-Key Establishment (NTS-KE).
Perform a quick test from a client machine with the following command:
$ chronyd -Q -t 3 'server ntp-server.example.net iburst nts maxsamples 1' 2021-09-15T13:45:26Z chronyd version 4.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 +DEBUG) 2021-09-15T13:45:26Z Disabled control of system clock 2021-09-15T13:45:28Z System clock wrong by 0.002205 seconds (ignored) 2021-09-15T13:45:28Z chronyd exiting
System clock wrongmessage indicates the NTP server is accepting NTS-KE connections and responding with NTS-protected NTP messages.
Verify the NTS-KE connections and authenticated NTP packets observed on the server:
# chronyc serverstats NTP packets received : 7 NTP packets dropped : 0 Command packets received : 22 Command packets dropped : 0 Client log records dropped : 0 NTS-KE connections accepted: 1 NTS-KE connections dropped : 0 Authenticated NTP packets: 7
If the value of the
NTS-KE connections acceptedand
Authenticated NTP packetsfield is a non-zero value, it means that at least one client was able to connect to the NTS-KE port and send an authenticated NTP request.