Chapter 2. Red Hat OpenShift support for Windows Containers release notes

2.1. About Red Hat OpenShift support for Windows Containers

Red Hat OpenShift support for Windows Containers enables running Windows compute nodes in an OpenShift Container Platform cluster. Running Windows workloads is possible by using the Red Hat Windows Machine Config Operator (WMCO) to install and manage Windows nodes. With Windows nodes available, you can run Windows container workloads in OpenShift Container Platform.

These release notes track the development of the WMCO, which provides all Windows container workload capabilities in OpenShift Container Platform.

2.2. Getting support

Red Hat OpenShift support for Windows Containers is provided and available as an optional, installable component. Red Hat OpenShift support for Windows Containers is not part of the OpenShift Container Platform subscription. It requires an additional Red Hat subscription and is supported according to the Scope of coverage and Service level agreements.

You must have this separate subscription to receive support for Red Hat OpenShift support for Windows Containers. Without this additional Red Hat subscription, deploying Windows container workloads in production clusters is not supported. You can request support through the Red Hat Customer Portal.

For more information, see the Red Hat OpenShift Container Platform Life Cycle Policy document for Red Hat OpenShift support for Windows Containers.

If you do not have this additional Red Hat subscription, you can use the Community Windows Machine Config Operator, a distribution that lacks official support.

2.3. Release notes for Red Hat Windows Machine Config Operator 7.2.1

This release of the WMCO provides new features and bug fixes for running Windows compute nodes in an OpenShift Container Platform cluster. The components of the WMCO 7.2.1 were released in RHBA-2024:1476.

2.3.1. Bug fixes

  • Previously, the WMCO did not properly wait for Windows virtual machines (VMs) to finish rebooting. This led to occasional timing issues where the WMCO would attempt to interact with a node that was in the middle of a reboot, causing WMCO to log an error and restart node configuration. Now, the WMCO waits for the instance to completely reboot. (OCPBUGS-23036)
  • Previously, the WMCO configuration was missing the DeleteEmptyDirData: true field, which is required for draining nodes that have emptyDir volumes attached. As a consequence, customers that had nodes with emptyDir volumes would see the following error in the logs: cannot delete Pods with local storage. With this fix, the DeleteEmptyDirData: true field was added to the node drain helper struct in the WMCO. As a result, customers are able to drain nodes with emptyDir volumes attached. (OCPBUGS-23081)
  • Previously, because of bad logic in the networking configuration script, the WICD was incorrectly reading carriage returns in the CNI configuration file as changes, and identified the file as modified. This caused the CNI configuration to be unnecessarily reloaded, potentially resulting in container restarts and brief network outages. With this fix, the WICD now reloads the CNI configuration only when the CNI configuration is actually modified. (OCPBUGS-27771)
  • Previously, the WMCO incorrectly approved the node certificate signing requests (CSR) for all nodes trying to join a cluster, not just Windows node CSRs. With this fix, the WMCO approves CSRs for only Windows nodes as expected. (OCPBUGS-27139)
  • Previously, because of routing issues present in Windows Server 2019, under certain conditions and after more than one hour of running time, workloads on Windows Server 2019 could have experienced packet loss when communicating with other containers in the cluster. This fix enables Direct Server Return (DSR) routing within kube-proxy. As a result, DSR now causes request and response traffic to use a different network path, circumventing the bug within Windows Server 2019. (OCPBUGS-28254)
  • Previously, because the upgrade path from WMCO 6.x to WMCO 7.x included previously released versions, the WMCO would fail during the upgrade. With this fix, you can successfully upgrade from WMCO 6.x to WMCO 7.x. (OCPBUGS-27775)
  • Previously, because of a lack of synchronization between Windows compute machine set nodes and Bring-Your-Own-Host (BYOH) instances, during an update the compute machine set nodes and the BYOH instances could update simultaneously, which could have impacted running workloads. This fix introduces a locking mechanism so that compute machine set nodes and BYOH instances update individually. (OCPBUGS-23020)

2.4. Release notes for Red Hat Windows Machine Config Operator 7.1.0

This release of the WMCO provides new features and bug fixes for running Windows compute nodes in an OpenShift Container Platform cluster. The components of the WMCO 7.1.0 were released in RHSA-2023:4025.

Important

Due to a known issue, your OpenShift Container Platform cluster must be on version 4.12.3 or greater before updating the WMCO from version 7.0.1 to version 7.1.0. The update fails if the cluster is lower than version 4.12.3.

2.4.1. Bug fixes

  • Previously, the containerd container runtime reported an incorrect version on each Windows node because repository tags were not propagated to the build system. This configuration caused containerd to report its go build version as the version of each Windows node. With this update, the correct version is injected into the binary during build time, so that containerd reports the correct version for each Windows node. (OCPBUGS-7843)
  • Previously, the Windows Machine Config Operator (WMCO) could not drain daemon set workloads. This issue caused Windows daemon set pods to block Windows nodes that the WMCO attempted to remove or update. With this update, the WMCO includes additional role-based access control (RBAC) permissions, so that the WMCO can remove daemon set workloads. The WMCO can also delete any processes that were created with the containerd shim, so that daemon set containers do not exist on a Windows instance after a WMCO removes a node from a cluster. (OCPBUGS-8056)
  • Previously, on an Azure Windows Server 2019 platform that does not have Azure container services installed, WMCO would fail to deploy Windows instances and would display the Install-WindowsFeature : Win32 internal error "Access is denied" 0x5 occurred while reading the console output buffer error message. The failure occurred because the Microsoft Install-WindowsFeature cmdlet displays a progress bar, which cannot be sent over an SSH connection. This fix hides the progress bar. As a result, Windows instances can be deployed as nodes. (OCPBUGS-14445)

2.5. Release notes for Red Hat Windows Machine Config Operator 7.0.1

This release of the WMCO provides new features and bug fixes for running Windows compute nodes in an OpenShift Container Platform cluster. The components of the WMCO 7.0.1 were released in RHBA-2023:0748.

2.5.1. Bug fixes

  • Previously, WMCO 7.0.0 did not support running in a namespace other than openshift-windows-machine-operator. With this fix, you can run WMCO in a custom namespace and can upgrade clusters that have WMCO installed in a custom namespace. (OCPBUGS-5065)

2.6. Release notes for Red Hat Windows Machine Config Operator 7.0.0

This release of the WMCO provides new features and bug fixes for running Windows compute nodes in an OpenShift Container Platform cluster. The components of the WMCO 7.0.0 were released in RHSA-2022:9096.

2.6.1. New features and improvements

2.6.1.1. Windows Instance Config Daemon (WICD)

The Windows Instance Config Daemon (WICD) is now performing many of the tasks that were previously performed by the Windows Machine Config Bootstrapper (WMCB). The WICD is installed on your Windows nodes. Users do not need to interact with the WICD and should not experience any difference in WMCO operation.

2.6.1.2. Support for clusters running on Google Cloud Platform

You can now run Windows Server 2022 nodes on a cluster installed on Google Cloud Platform (GCP). You can create a Windows MachineSet object on GCP to host Windows Server 2022 compute nodes. For more information, see Creating a Windows MachineSet object on vSphere.

2.6.2. Bug fixes

  • Previously, restarting the WMCO in a cluster with running Windows Nodes caused the windows exporter endpoint to be removed. Because of this, each Windows node could not report any metrics data. After this fix, the endpoint is retained when the WMCO is restarted. As a result, metrics data is reported properly after restarting WMCO. (BZ#2107261)
  • Previously, the test to determine if the Windows Defender antivirus service is running was incorrectly checking for any process whose name started with Windows Defender, regardless of state. This resulted in an error when creating firewall exclusions for containerd on instances without Windows Defender installed. This fix now checks for the presence of the specific running process associated with the Windows Defender antivirus service. As a result, the WMCO can properly configure Windows instances as nodes regardless of whether Windows Defender is installed or not. (OCPBUGS-3573)

2.6.3. Known issues

The following known limitations have been announced after the previous WMCO release:

  • OpenShift Serverless, Horizontal Pod Autoscaling, and Vertical Pod Autoscaling are not supported on Windows nodes.
  • Red Hat OpenShift support for Windows Containers does not support adding Windows nodes to a cluster through a trunk port. The only supported networking configuration for adding Windows nodes is through an access port that carries traffic for the VLAN.
  • WMCO 7.0.0 does not support running in a namespace other than openshift-windows-machine-operator. If you are using a custom namespace, it is recommended that you not upgrade to WMCO 7.0.0. Instead, you should upgrade to WMCO 7.0.1 when it is released. If your WMCO is configured with the Automatic update approval strategy, you should change it to Manual for WMCO 7.0.0. See the installation instructions for information on changing the approval strategy.

Additional resources

See the full list of known limitations

2.7. Windows Machine Config Operator prerequisites

The following information details the supported platform versions, Windows Server versions, and networking configurations for the Windows Machine Config Operator. See the vSphere documentation for any information that is relevant to only that platform.

2.7.1. WMCO 7.2.x supported platforms and Windows Server versions

The following table lists the Windows Server versions that are supported by WMCO 7.2.0 and 7.2.1, based on the applicable platform. Unlisted Windows Server versions are not supported and attempting to use them will cause errors.

PlatformSupported Windows Server version

Amazon Web Services (AWS)

  • Windows Server 2022, OS Build 20348.681 or later
  • Windows Server 2019, version 1809

Microsoft Azure

  • Windows Server 2022, OS Build 20348.681 or later
  • Windows Server 2019, version 1809

VMware vSphere

  • Windows Server 2022, OS Build 20348.681 or later

Google Cloud Platform (GCP)

  • Windows Server 2022, OS Build 20348.681 or later

Bare metal or provider agnostic

  • Windows Server 2022, OS Build 20348.681 or later
  • Windows Server 2019, version 1809

2.7.2. WMCO 7.0 and 7.1 supported platforms and Windows Server versions

The following table lists the Windows Server versions that are supported by WMCO 7.0.0, 7.0.1, and 7.1.0, based on the applicable platform. Unlisted Windows Server versions are not supported and attempting to use them will cause errors.

PlatformSupported Windows Server version

Amazon Web Services (AWS)

  • Windows Server 2019, version 1809

Microsoft Azure

  • Windows Server 2022, OS Build 20348.681 or later
  • Windows Server 2019, version 1809

VMware vSphere

  • Windows Server 2022, OS Build 20348.681 or later

Google Cloud Platform (GCP)

  • Windows Server 2022, OS Build 20348.681 or later

Bare metal or provider agnostic

  • Windows Server 2022, OS Build 20348.681 or later
  • Windows Server 2019, version 1809

2.7.3. Supported networking

Hybrid networking with OVN-Kubernetes is the only supported networking configuration. See the additional resources below for more information on this functionality. The following tables outline the type of networking configuration and Windows Server versions to use based on your platform. You must specify the network configuration when you install the cluster.

Note

The WMCO does not support OVN-Kubernetes without hybrid networking or OpenShift SDN.

Table 2.1. Platform networking support

PlatformSupported networking

Amazon Web Services (AWS)

Hybrid networking with OVN-Kubernetes

Microsoft Azure

Hybrid networking with OVN-Kubernetes

VMware vSphere

Hybrid networking with OVN-Kubernetes with a custom VXLAN port

Google Cloud Platform (GCP)

Hybrid networking with OVN-Kubernetes

Bare metal or provider agnostic

Hybrid networking with OVN-Kubernetes

Table 2.2. Hybrid OVN-Kubernetes Windows Server support

Hybrid networking with OVN-KubernetesSupported Windows Server version

Default VXLAN port

  • Windows Server 2022, OS Build 20348.681 or later
  • Windows Server 2019, version 1809

Custom VXLAN port

Windows Server 2022, OS Build 20348.681 or later

2.8. Known limitations

Note the following limitations when working with Windows nodes managed by the WMCO (Windows nodes):

  • The following OpenShift Container Platform features are not supported on Windows nodes:

    • Image builds
    • OpenShift Pipelines
    • OpenShift Service Mesh
    • OpenShift monitoring of user-defined projects
    • OpenShift Serverless
    • Horizontal Pod Autoscaling
    • Vertical Pod Autoscaling
  • The following Red Hat features are not supported on Windows nodes:

  • Windows nodes do not support pulling container images from private registries. You can use images from public registries or pre-pull the images.
  • Windows nodes do not support workloads created by using deployment configs. You can use a deployment or other method to deploy workloads.
  • Windows nodes are not supported in clusters that use a cluster-wide proxy. This is because the WMCO is not able to route traffic through the proxy connection for the workloads.
  • Windows nodes are not supported in clusters that are in a disconnected environment.
  • Red Hat OpenShift support for Windows Containers does not support adding Windows nodes to a cluster through a trunk port. The only supported networking configuration for adding Windows nodes is through an access port that carries traffic for the VLAN.
  • Red Hat OpenShift support for Windows Containers supports only in-tree storage drivers for all cloud providers.
  • Kubernetes has identified the following node feature limitations :

    • Huge pages are not supported for Windows containers.
    • Privileged containers are not supported for Windows containers.
  • Kubernetes has identified several API compatibility issues.