How to set timezone for pods in OpenShift Container Platform?

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 3
    • 4

Issue

  • Timezone issues with application logs in OpenShift.
  • Why is the application log timezone different then the timezone set in OpenShift?

Resolution

By default, the timezone configured for OpenShift clusters is UTC, while container's timezone is set based on the base container image. At the time of this writing, using the worker node as source is not done automatically by OpenShift. However, a Request for Feature Enhancement (RFE-1283) has been filed to enable this functionality.

As it is explained in is changing timezone of RHCOS and pods in OpenShift 4 supported, Red Hat recommends against configuring timezone for nodes and pods, as it may result in confusion in log aggregation scenarios where timestamps are recorded differently as strings requiring processing to align events, and also daylight saving time could cause confusion with logs and issues in some applications.

Important: changing the timezone of the OpenShift platform containers is not supported, and they are required to keep the UTC timezone.

Non-recommended workarounds

If still wanting to change the timezone in custom workloads, even if it is not recommended, there are some workarounds. Be aware some of them set on the deployment/deploymentconfig, therefore are non-applicable for most Operator based applications, which controls the deployment/deploymentconfig/statefulset.

IMPORTANT NOTE: there could be different issues when applying the non-recommended workarounds, like (but not limited to) the following ones:

Workaround 1

Mount /etc/localtime to use the same timezone in host machine using the "host path" volume mount method as described in the OpenShift Dev Guide on Volumes.

Workaround 2

Mount /etc/localtime to use the timezone stored in a configmap:

$ oc create configmap tz-london --from-file=localtime=/usr/share/zoneinfo/Europe/London
$ oc set volumes deployments/dc_name --add \
        --type=configmap --name=tz --configmap-name=tz-london \
        --mount-path=/etc/localtime --sub-path=localtime

Workaround 3

Set an environment variable. This sets TZ in any containers to the timezone specified:

$ oc get deployments  
OR
$ oc get deploymentconfig
...
$ oc set env deployments/dc_name TZ=Asia/Tokyo 
OR
$ oc set env deploymentconfig/dc_name TZ=Asia/Tokyo

Root Cause

By default, the timezone configured for OpenShift clusters is UTC, and container's timezone is set based on the base container image.

Diagnostic Steps

Pod logs are reported, when using --timestamps, in current local time - 1, which might be a summer/daylight savings time related issue:

$ oc logs pod_name --timestamps
$ oc rsh pod_name
$ env | grep TZ    --> to confirm the updated timezone

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments