How to set timezone for pods in OpenShift Container Platform?

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 3
    • 4

Issue

  • Timezone issues with application logs in OpenShift.
  • Why is the application log timezone different then the timezone set in OpenShift?

Resolution

By default, the timezone configured for OpenShift clusters is UTC, while container's timezone is set based on the base container image. At the time of this writing, using the worker node as source is not done automatically by OpenShift. However, a Request for Feature Enhancement (RFE-1283) has been filed to enable this functionality.

Red Hat recommends against configuring timezone for nodes and pods, as it may result in confusion in log aggregation scenarios where timestamps are recorded differently as strings requiring processing to align events, and also daylight saving time could cause confusion with logs and issues in some applications.

Important: Changing the timezone of the OpenShift platform containers is not supported, and they are required to keep the UTC timezone.

Workarounds

If still wanting to change the timezone in custom workloads, even if it is not recommended, there are some workarounds. Be aware some of them set on the deployment/deploymentconfig, therefore are non-applicable for most Operator based applications, which controls the deployment/deploymentconfig/statefulset.

Workaround 1

Mount /etc/localtime to use the same timezone in host machine using the "host path" volume mount method as described in the OpenShift Dev Guide on Volumes.

Workaround 2

Mount /etc/localtime to use the timezone stored in a configmap:

$ oc create configmap tz-london --from-file=localtime=/usr/share/zoneinfo/Europe/London
$ oc set volumes deployments/dc_name --add \
        --type=configmap --name=tz --configmap-name=tz-london \
        --mount-path=/etc/localtime --sub-path=localtime

Workaround 3

Set an environment variable. This sets TZ in any containers to the timezone specified:

$ oc get deployments  
OR
$ oc get deploymentconfig
...
$ oc set env deployments/dc_name TZ=Asia/Tokyo 
OR
$ oc set env deploymentconfig/dc_name TZ=Asia/Tokyo

Note: Deployments with parent ubi8-minimal image does not reflect correct TZ environment variable value. Kindly refer Why is /usr/share/zoneinfo empty in ubi8-minimal images despite the tzdata package being present? for more information.

If after changing the timezone in a pod, the Daylight Saving Time (DST) is not properly applied, please refer to Pods with TZ variable have wrong date after DST change in OpenShift 4.

Root Cause

By default, the timezone configured for OpenShift clusters is UTC, and container's timezone is set based on the base container image.

Diagnostic Steps

Pod logs are reported, when using --timestamps, in current local time - 1, which might be a summer/daylight savings time related issue:

$ oc logs pod_name --timestamps
$ oc rsh pod_name
$ env | grep TZ    --> to confirm the updated timezone

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments