Contents

1. Introduction
2. Upcoming Leap Second Events
3. Environment
4. Understanding the Leap Second
5. Handling of the Leap Second
6. Known Issues
7. Realtime Kernels
8. Testing
9. Example Configurations
10. Additional Resources

Introduction

Leap seconds are a periodic one-second adjustment of Coordinated Universal Time(UTC) in order to keep a system's time of day close to the mean solar time. However, the Earth's rotation speed varies in response to climatic and geological events, and due to this, UTC leap seconds are irregularly spaced and unpredictable. This article seeks to provide information regarding the leap seconds themselves, along with addressing how Red Hat Enterprise Linux handles the insertion of one.

Upcoming Leap Second Events

At this time, we are not aware of upcoming leap second events.

Environment

• Red Hat Enterprise Linux versions 4, 5, 6, 7 and 8

Understanding the Leap Second

The basic time for mostly all of the world's local time zones is called Coordinated Universal Time, UTC, which is derived from a bunch of atomic clocks which are distributed in several countries all over the world. The rotation of the earth is not very constant and varies a bit over time, while decreasing the mean rotation speed slowly. This is the reason why so called leap seconds are inserted into the UTC time scale, they adjust the UTC time to the real earth rotation.

Why this extra second? It exists because the rotation of the Earth on its axis, which determines the passing of days and nights, slows down over a long period, mainly as a consequence of Moon-Sun attraction effects. In addition, the Earth is affected by its internal (core, mantle) and external (atmosphere, oceans) constituents. Nowadays, though, time is measured largely by 250 atomic clocks belonging to several countries, which function by measuring the transition of energy levels in an atom. Together these clocks are used to calculate UTC, and as this time measuring mechanism is independent of the Earth periodically the two must be brought into alignment with a leap second. In addition, we have to consider that the length of the day is nowadays 2 ms longer than in the year 1820. Not surprisingly then, the Earth's rotation slowly gets out of synchronization with UTC.

The International Earth Rotation Service, IERS, is measuring the true earth rotation and determines when a leap second has to be inserted. Insertion of a leap second is always scheduled for the end of the last day of a month, preferably at the end of June or December, at UTC midnight. In the past all leap seconds had been inserted at either one of those times(*). Announcements whether a leap second is scheduled or not are published by the IERS in their Bulletin C. The current Bulletin C is published about half a year before the next possible date for a leap second.

The IERS Bulletin C #30 from July 2005 announced a leap second to be inserted at the end of December 31, 2005, at UTC midnight. This was the first leap second that had been inserted since the end of 1998. This is why many applications which had been developed during the previous 7 years could not handle the leap second correctly.

Since, the leap second was inserted at the same moment all over the world, the local (civil) time of the insertion depends on the local time offset from UTC, e.g. if the time zone is UTC +3h then the leap second will be inserted when the wall clocks show 3 hours after midnight.

The standard way to count UTC time across a leap second is:

  2011-12-31 23.59.57
  2011-12-31 23.59.58
  2011-12-31 23.59.59
  2011-12-31 23.59.60 <-- leap second
  2012-01-01 00.00.00
  2012-01-01 00.00.01
  2012-01-01 00.00.02

If the system clock is kept in TAI and a right/* timezone is used then 23:59:60 may be listed; however, as 23:59:60 does not exist in Unix's implementation of UTC then the linux kernel inserts the leap second by stepping the system clock back by one second on the first clock update after 0:00 UTC. In local time counting depends on the time zone offset, e.g. for UTC+3h you would observe the following:

2012-01-01  02:58:00.000
2012-01-01  02:58:00.500
2012-01-01  02:59:00.000
2012-01-01  02:59:00.500
2012-01-01  03:00:00.0xx <-- leap second inserted
2012-01-01  02:59:00.0xx
2012-01-01  02:59:00.500
2012-01-01  03:00:00.000
2012-01-01  03:00:00.500

Leap seconds are a discontinuity of civil time. The time does not continue to increase monotonically but it is stepped by one second. Let's have a look at the time stamps of a leap second which is inserted, and the second after that leap second:

  2011-12-31 23.59.60 <-- leap second
  2012-01-01 00.00.00

We can normalize the time and date of the leap second:
60 seconds are 1 minute, which lets the minutes increment from 59 to 60
60 minutes are 1 hour, which lets the hours increment from 23 to 24
24 hours are 1 day, which lets the date increment, and so on.
Finally we can say that both lines represent exactly the same time, or 2 consequent seconds have the same time stamp.

Several time dissemination services also propagate the announcement of a leap second after this has been determined by the IERS. This includes for example the German long wave transmitter DCF77 and the satellite based navigation system GPS, so receivers which decode the signals from those systems can also decode the leap second announcement. Applications which read the time from those receivers can also determine the leap second announcement if this information is included in the applied protocol, e.g the time string transmitted by the receiver.

Please note that time code receivers can only pass the announcement of a leap second to the application, and count the time correctly over that period. It's the task of the application and/or operating system to handle that leap second correctly.

(*): An additional leap second notification was erroneously issued by some NTP (Network Time Protocol) servers within the North America region for midnight UTC on August 31, 2012. Two stratum 1 servers -- truechimer.cites.illinois.edu and time-b.nist.gov -- were identified as erroneously issuing the leap second notification. Any higher stratum (stratum 2, 3, 4....) NTP server with either of these two servers in their hierarchy should be considered as having issued the leap second incorrectly. NTP clients using these servers will set the kernel's leap indicator flag, which cannot be cleared until the system is rebooted.

Handling of the Leap Second

The leap second will be handled in two primary ways, depending on how the system clock is configured.

The system clock is keeping time with UTC and synchronized via NTP or PTP

As mentioned previously, the leap second must be skipped when the system is using UTC, as this second does not exist. Therefore, to prevent the system clock from being one second ahead of UTC, the clock must be corrected after the leap second occurs. This correction may occur in a variety of ways, outlined below:

Notify the kernel, allowing it to step back the clock

The default approach by most daemons is to notify the kernel, and allow it to insert the leap second. For instance, during the last day before a leap second correction NTP servers should notify their clients that a leap second will occur, and at 23:59:59 UTC, the Linux kernel should add or remove an extra second by making the 60th second occur twice or removing it entirely. Thus, Red Hat Enterprise Linux systems running an NTP client, with a default configuration, during the last leap second correction should have counted time as follows:

2008-12-31 23:59:59:052549000 UTC    <-- 1st occurrence of the 60th second
2008-12-31 23:59:59:259988000 UTC
2008-12-31 23:59:59:465214000 UTC
2008-12-31 23:59:59:669629000 UTC
2008-12-31 23:59:59:873936000 UTC
2008-12-31 23:59:59:079184000 UTC    <-- 2nd occurrence of the 60th second
2008-12-31 23:59:59:284011000 UTC
2008-12-31 23:59:59:488648000 UTC
2008-12-31 23:59:59:692691000 UTC
2008-12-31 23:59:59:896577000 UTC
2009-01-01 00:00:00:052378000 UTC

When the leap second occurs, the kernel prints a message to the system log.

Have the daemon step back the clock

Instead of notifying the kernel, and allowing it to step the time back, the daemon may step the system clock backwards. Time will be counted similarly to the kernel inserting the leap second; however, as the daemon performs the actual insertion any bugs specific to the kernel will not be encountered.

Slew the leap second

RHEL 6.8 and RHEL 7.2 introduced an option to slew the leap second for systems using Chrony, allowing this second to be spread out over a period of time instead of an immediate step back in time. This slew is handled as a one-second offset accumulated at 00:00:00 UTC, for leap second insertions, or 23:59:59 UTC, for leap second deletions, that is smoothed out over time. This method may be preferred over stepping when applications running on the system are sensitive to jumps in system time and it is acceptable that the clock will be off for a longer time.

Ignore the leap second

Instead of addressing the leap second daemons may be configured to ignore it entirely, allowing time to be slowly corrected as any other unexpected change in time. The clock will continue with the incorrect time until it is addressed through normal operation. If multiple systems are configured with this method it is possible that their times will drift apart from each other, as the correction process is random to a certain extent. This method is not recommended for systems that require in-sync communication.

Leap smear on NTP Server

RHEL 6.8 and RHEL 7.2 introduced an option to smear the leap second when Chrony is running on the NTP server(s). Instead of having the clients slew the leap second, it may instead be suppressed on the server and slowly correct the served time by slewing the leap second instead of stepping it. Clients do not need any special configuration, as they do not know there is a leap second, and will continue to follow the server time until they are in sync with UTC. Smearing the leap second on the server is much slower than doing so on individual clients; however, it allows all clients to connect.

When configuring a NTP server to smear the leap second it is necessary to ensure that all clients using it are only pointing to servers that are performing the smear in an identical fashion. If clients are configured to receive updates from multiple servers, and these servers handle the leap second in a different fashion, inconsistent results will occur.

Table of Options

The following table highlights the options available to each daemon:

A list of example configurations may be found at the bottom of this page, in Example Configurations.

The following table demonstrates the effects of each configuration method:

* Ignoring the discontinuity in time

The system clock is keeping time in TAI-10 seconds, is not synchronized, and is using a right/* timezone

Systems may also be configured to report time corrected for leap seconds by updating the tzdata package to the latest version available, copying the appropriate file from the /usr/share/zoneinfo/right directory hierarchy to /etc/localtime, and resetting the clock to the correct local time. The files in /usr/share/zoneinfo/right contain local time information corrected for all leap seconds that have occurred since the beginning of the Epoch on 1970-01-01 00:00:00 UTC. The other time zone files in /usr/share/zoneinfo do not have leap second corrections added. After the 2008 leap second, there were 24 leap seconds added since the Epoch. Any application that expects the time to be in UTC will have issues if a right/* timezone is used.

As an example, if a system is in the America/Los_Angeles (US Pacific) time zone, you can reconfigure the system to report leap-second-corrected time by running the following and resetting the clock to Pacific Time:

cp /usr/share/zoneinfo/right/America/Los_Angeles /etc/localtime

After /etc/localtime has been changed glibc will reload this file automatically; in addition, note that adjusting timezones does not change the system time, only the conversion from the system time to local time that happens in applications via glibc functions, such as localtime(), ctime(), etc. A system restart is not required after this update; however, if any applications are caching results from these functions then these applications may need to be restarted after tzdata has been updated.

Note: Red Hat does not recommend running without a source of time synchronization enabled as there is no guarantee that systems will remain in sync with each other. Time drift may occur on each system as determined by its clock source. The decision to run without NTP/PTP/Chrony will need to be made on a per-system or per-cluster basis with the understanding that time drift may occur.

By default, Linux systems not using NTP or PTP to synchronize their timekeeping will not correct for leap seconds, and the time reported by these systems will have a one-second difference relative to UTC after the leap second correction. You should reset the clock manually after leap seconds occur.

Known Issues

RHEL 4

1. There is a chance that the printing of this message can cause the kernel to crash in Red Hat Enterprise Linux 4; this issue is documented in System hangs on printing the leap second insertion message.
2. Slew mode does not prevent ntp from setting the kernel flag; this issue is documented in In Red Hat Enterprise Linux 4, slew mode does not prevent ntp from setting the kernel flag.
3. For systems not synchronized by ntpd or ptp an updated tzdata package that contains the December 31st leap second is required. The updated tzdata package was released as part of RHEA-2016-1982, and any systems using RHEL 4 that are not synchronized by ntpd or ptp should update to tzdata-2016g-2.el4, or a later version, to receive this fix.

RHEL 5

1. There is a chance that the printing of this message can cause the kernel to crash in Red Hat Enterprise Linux 5; this issue is documented in System hangs on printing the leap second insertion message.
2. When running ntp in slew mode time is not adjusted properly by the leap second; this is documented in Time is not adjusted by a leap second when ntpd runs with -x option.
3. For systems not synchronized by ntpd or ptp an updated tzdata package that contains the December 31st leap second is required. The updated tzdata package was released as part of RHEA-2016-1982, and any systems using RHEL 5 that are not synchronized by ntpd or ptp should update to tzdata-2016g-2.el5, or a later version, to receive this fix.

RHEL 6

1. There is a chance that a system can hang once it receives notification of the insertion of a leap second; this issue is documented in Systems hang due to leap-second livelock.
2. After the leap second has been inserted futex heavy applications began consuming a large amount of CPU; this issue is documented in Why is there high CPU usage after inserting the leap second?.
3. The TAI offset is not updated correctly during the leap second; this issue is documented in TAI offset is incorrect during the leap second.
4. Using -x with ntp still results in instantaneous clock changes when leap second occurs; this issue is documented in Does Red Hat plan to release xleap.patch with ntp?.
5. Absolute timers may fire early when the leap second is inserted; this issue is documented in Absolute Timers that Expire at Midnight UTC May Fire Early When the Leap Second is Inserted.
6. For systems not synchronized by ntpd or ptp an updated tzdata package that contains the December 31st leap second is required. The updated tzdata package was released as part of RHEA-2016-1982, and any systems using RHEL 6 that are not synchronized by ntpd or ptp should update to tzdata-2016g-2.el6, or a later version, to receive this fix.
7. There is a chance that chronyd may crash when smearing the leap second; this issue is documented in Chronyd crashes when performing server leap smear.
8. Some versions of NTP will continue to announce the leap second after it has been inserted; this issue is documented in The ntpd leap status is not reset after inserting a leap second.

RHEL 7

1. Using -x with ntp still results in instantaneous clock changes when leap second occurs; this issue is documented in Does Red Hat plan to release xleap.patch with ntp?.
2. Absolute timers may fire early when the leap second is inserted; this issue is documented in Absolute Timers that Expire at Midnight UTC May Fire Early When the Leap Second is Inserted.
3. For systems not synchronized by ntpd or ptp an updated tzdata package that contains the December 31st leap second is required. The updated tzdata package was released as part of RHEA-2016-1982, and any systems using RHEL 7 that are not synchronized by ntpd or ptp should update to tzdata-2016g-2.el7, or a later version, to receive this fix.
4. There is a chance that chronyd may crash when smearing the leap second; this issue is documented in Chronyd crashes when performing server leap smear.
5. There is a chance that hrtimers may fire early when the leap second is inserted; this issue is documented in Hrtimers may expire early when a leap second is inserted.
6. Some versions of NTP will continue to announce the leap second after it has been inserted; this issue is documented in The ntpd leap status is not reset after inserting a leap second.

In addition to the issues tracked above it is possible that application-specific issues will arise if the leap second was not considered during development. Issues of this nature are documented in Libraries and Applications do not account for the Leap Second.

Note: Red Hat recommends that customers using PPC and IA64 architectures use the method described under Systems not running NTP or PTP.

Realtime Kernels

A separate article has been created for tracking issues with realtime kernels. For more information refer to Resolve Leap Second Issues in Realtime (RT) kernels.

Testing

Red Hat continues to test for any issues, and as information is available regarding the completed tests this article will be updated.

A lab has been developed to perform this test, and is available at Leap Second Issue Detector.

In addition, Red Hat strongly recommends that customers test their builds and environments, and instructions on how to test, including a sample program to do so, are provided at Are we susceptible to a leap second event?.

Example Configurations

Stepping the leap second with Chrony

To step the leap second, add the following options to the /etc/chrony.conf:

leapsecmode step

These options will instruct chronyd to correct the system clock by stepping the leap second, bypassing any bugs with the kernel's insertion of the leap second.

Slewing the leap second with chrony

To slew the leap second, add the following options to the /etc/chrony.conf:

leapsecmode slew

These options will instruct chronyd to correct the system clock by slewing the leap second instead of stepping back immediately.

Smearing the leap second for clients on chrony NTP server

To smear the leap second, add the following options to the /etc/chrony.conf:

leapsecmode slew
maxslewrate 1000
smoothtime 400 0.001 leaponly

These options will instruct chronyd to correct the system clock by slewing the leap second and limit the slewing rate of the local clock to 1000 parts per million (ppm).

Ignore the leap second with NTPD

To ignore the leap second, allowing it to be corrected by normal slewing operations, ensure the following line is present in /etc/sysconfig/ntpd:

OPTIONS="-g -x"

With this flag added the kernel's leap second correction is disabled, and the leap second will be corrected over a period of time.

Additional Resources

Additional information on leap seconds and how they are handled in Linux and by NTP can be found at the following links:

• Leap Seconds
• The NTP Timescale and Leap Seconds
• NTP in real Life: What happens during a Leap Second?
• Leap second on December 31st 2016
• Five different ways to handle leap seconds with NTP