Failed to upgrade kube-scheduler when upgrading the cluster to 4.0.0-0.11
Environment
- Openshift Container Platform
- 4.x
Issue
- Failed to upgrade
kube-scheduler
when upgrading the cluster to 4.0.0-0.11 - The errors are:
Warning FailedCreatePodSandBox 105s (x732 over 6h58m) kubelet, ip-192.168.0.1.ec2.internal (combined from similar events): Failed create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_revision-pruner-8-ip-192.168.0.1.ec2.internal_openshift-kube-apiserver_854dfb00-59d3-11e9-a0c2-0ea8a7cdd834_0(be2160403b10ac987e500a234ecc2c0a02fba0b6447c034365121c59b443dbfe): Multus: Err adding pod to network "openshift-sdn": Multus: error in invoke Delegate add - "openshift-sdn": CNI request failed with status 400: 'Unauthorized
- The journalctl logs shows:
Apr 08 15:10:18 ip-192.168.0.1 crio[943]: time="2019-04-08 15:10:18.388209692Z" level=error msg="Error while adding to cni network: Multus: Err adding pod to network "openshift-sdn": Multus: error in invoke Delegate add - "openshift-sdn": CNI request failed with status 400: 'Unauthorized
Apr 08 15:10:18 ip-192.168.0.1 crio[943]: '"
Apr 08 15:10:18 ip-192.168.0.1 crio[943]: 2019-04-08T15:10:18Z [error] Multus: Err adding pod to network "openshift-sdn": Multus: error in invoke Delegate add - "openshift-sdn": CNI request failed with status 400: 'Unauthorized
Apr 08 15:10:18 ip-192.168.0.1 crio[943]: '
Apr 08 15:10:18 ip-192.168.0.1 crio[943]: time="2019-04-08 15:10:18.404559460Z" level=error msg="Error adding network: Multus: Err adding pod to network "openshift-sdn": Multus: error in invoke Delegate add - "openshift-sdn": CNI request failed with status 400: 'Unauthorized
Apr 08 15:10:18 ip-192.168.0.1 crio[943]: '"
Apr 08 15:10:18 ip-192.168.0.1 crio[943]: time="2019-04-08 15:10:18.404601392Z" level=error msg="Error while adding to cni network: Multus: Err adding pod to network "openshift-sdn": Multus: error in invoke Delegate add - "openshift-sdn": CNI request failed with status 400: 'Unauthorized
Apr 08 15:10:18 ip-192.168.0.1 crio[943]: '"
Apr 08 15:10:18 ip-192.168.0.1 systemd[1]: crio-65981de1f1aa0fc7c65e02c7bfad64ce9994e6a30f5cfebd519ecbe1331d4f54.scope: Consumed 37ms CPU time
-- Subject: Resources consumed by unit runtime
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
Resolution
- This is being tracked in the Bugzilla 1697520
Diagnostic Steps
- Check the
journalctl logs
from the node wherekube-scheduler
pod is running.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments