Adding numeric value tags to machinesets in OpenShift 4 on AWS

Solution Verified - Updated -

Issue

When creating a machineset definition yaml file it is possible to add a tags section containing a list of tags to apply to the host instances created by the cloud provider (AWS):

apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
...
spec:
      providerSpec:
        value:
          ...
          tags:
          - name: kubernetes.io/cluster/koshka-j9f8v
            value: owned
          - name: Environment
            value: Test
          - name: POBox
            value: 09080

If any of the tags has a numeric value, as shown in the example above, when the machineset is created the corresponding machines never get created:

$ oc create -f machineset1.yaml
machineset.machine.openshift.io/koshka-82qmp-worker-eu-west-3a2 created

$ oc get machinesets -n openshift-machine-api
NAME                              DESIRED   CURRENT   READY   AVAILABLE   AGE
...
koshka-82qmp-worker-eu-west-3a2   1         1                              50m

$ oc get machines
NAME                                   PHASE         TYPE       REGION      ZONE         AGE
putri-qnc9k-worker-eu-west-3a-mzt72                                                      65m

The logs from the machine controller pod contains error messages like the following:

$ oc logs machine-api-controllers-7f744c5666-k4k9k -c machine-controller -n openshift-machine-api
...
E0806 12:24:32.307766       1 actuator.go:512] koshka-82qmp-worker-eu-west-3a2-8j6j5: Error decoding MachineProviderConfig: decoding failure: v1beta1.AWSMachineProviderConfig.Tags: []v1beta1.TagSpecification: v1beta1.TagSpecification.Value: ReadString: expects " or n, but found 4, error found in #10 byte of ...|,"value":41805}],"us|..., bigger context ...|Planet","value":"Earth"},{"name":"POBox","value":41805}],"userDataSecret":{"name":"worker-user-data"|...
E0806 12:24:32.307778       1 actuator.go:493] koshka-82qmp-worker-eu-west-3a2-8j6j5: Error getting existing instances: decoding failure: v1beta1.AWSMachineProviderConfig.Tags: []v1beta1.TagSpecification: v1beta1.TagSpecification.Value: ReadString: expects " or n, but found 4, error found in #10 byte of ...|,"value":41805}],"us|..., bigger context ...|Planet","value":"Earth"},{"name":"POBox","value":41805}],"userDataSecret":{"name":"worker-user-data"|...
E0806 12:24:32.307791       1 controller.go:266] koshka-82qmp-worker-eu-west-3a2-8j6j5: failed to check if machine exists: decoding failure: v1beta1.AWSMachineProviderConfig.Tags: []v1beta1.TagSpecification: v1beta1.TagSpecification.Value: ReadString: expects " or n, but found 4, error found in #10 byte of ...|,"value":41805}],"us|..., bigger context ...|Planet","value":"Earth"},{"name":"POBox","value":41805}],"userDataSecret":{"name":"worker-user-data"|...

If the stuck machineset is deleted, the corresponding machine stays in Deleting state for ever and the following event appears in the machine-api namespace:

$ oc delete machineset koshka-82qmp-worker-eu-west-3a2 -n openshift-machine-api
machineset.machine.openshift.io "koshka-82qmp-worker-eu-west-3a2" deleted

$ oc get machines -n openshift-machine-api
NAME                                    PHASE      TYPE        REGION      ZONE         AGE
...
koshka-82qmp-worker-eu-west-3a2-8j6j5   Deleting                                        12m

$ oc get events -n openshift-machine-api
100s        Warning   FailedDelete        machine/koshka-82qmp-worker-eu-west-3a2-8j6j5      error decoding MachineProviderConfig: decoding failure: v1beta1.AWSMachineProviderConfig.Tags: []v1beta1.TagSpecification: v1beta1.TagSpecification.Value: ReadString: expects " or n, but found 4, error found in #10 byte of ...|,"value":41805}],"us|..., bigger context ...|Planet","value":"Earth"},{"name":"POBox","value":41805}],"userDataSecret":{"name":"worker-user-data"|...

Deleting the machine with the following command will cause the terminal to block until canceled with CTRL-C, and the machine is not deleted:

$ oc delete  machine koshka-82qmp-worker-eu-west-3a2-8j6j5 -n openshift-machine-api
machine.machine.openshift.io "koshka-82qmp-worker-eu-west-3a2-6t2vx" deleted
^C
$ $ oc get machines -n openshift-machine-api
NAME                                    PHASE      TYPE        REGION      ZONE         AGE
...
koshka-82qmp-worker-eu-west-3a2-8j6j5   Deleting                                        87m

To delete the machine, edit the yaml definition and remove the finalizers section

$ oc edit machine koshka-82qmp-worker-eu-west-3a2-8j6j5 -n openshift-machine-api

apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
  creationTimestamp: "2020-08-06T12:52:11Z"
  deletionGracePeriodSeconds: 0
  deletionTimestamp: "2020-08-06T14:18:06Z"
  finalizers:                     <-------------- Delete this line
  - machine.machine.openshift.io  <--------------------- Delete this line

This issue is not present when adding user defined tags with numeric values to the install-config.yaml file used by the Openshift installer, as can be seen in this example in the Openshift documentation. In this case the list of user defined tags is properly added to the machinesets for the worker nodes.

Environment

  • Red Hat OpenShift Container Platform
    • 4.2
    • 4.3
    • 4.4

The issue is fixed in OCP 4.5

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content