Why draining a node is stuck with error 'failed calling webhook "migration-create-validator.kubevirt.io"'?
Issue
- The OpenShift node was having many virtual machines running (50+ VMs) and I am unable to drain the node. The creation of VirtualMachineInstanceMigration(VMIM) is failing with following error:
8s (x4 over 39m) Warning FailedCreate VirtualMachineInstance/test-vm-29 Error creating a Migration: Internal error occurred: failed calling webhook "migration-create-validator.kubevirt.io": failed to call webhook: Post "https://virt-api.openshift-cnv.svc:443/migration-validate-create?timeout=10s": context deadline exceeded
- The virt-api pod has the following logs, with the "wait time" continuously increasing:
I0121 20:17:22.042821 1 request.go:697] Waited for 3m33.071923903s due to client-side throttling, not priority and fairness, request: GET:https://172.31.0.1:443/api/v1/namespaces/default/pods/virt-launcher-testvm-146-jbw27
Environment
- OpenShift Virtualization 4.16, 4.17.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.