openshift-master fails to start with ssl errors
Issue
- Came in today to find my cluster was down, on attempting to restart the master I got the following error
> /usr/bin/openshift start master --config=${CONFIG_FILE} $OPTIONS
W0820 08:59:09.203981 77744 start_master.go:251] serviceAccountConfig.masterCA: invalid value '': master CA information will not be automatically injected into pods, which will prevent verification of the API server from inside a pod
I0820 08:59:09.204162 77744 start_master.go:308] Starting OpenShift master on 0.0.0.0:443 (v3.0.1.0-525-geddc479)
I0820 08:59:09.204178 77744 start_master.go:309] Public master address is https://master.example.com:443
F0820 09:00:35.685653 77744 start_master.go:100] could not reach etcd: 501: All the given peers are not reachable (failed to propose on members [https://host1.example.com:2379 https://host2.example.com:2379 https://host3.example.com:2379] twice [last error: Get https://host1.example.com:2379/v2/keys/?quorum=false&recursive=false&sorted=false: x509: certificate has expired or is not yet valid]) [0]
- I resolved this by adding
serviceAccountConfig:
masterCA: ca.crt
- On restarting the master now I get this error
> /usr/bin/openshift start master --config=${CONFIG_FILE} $OPTIONS
I0820 11:03:57.267976 78469 start_master.go:308] Starting OpenShift master on 0.0.0.0:443 (v3.0.1.0-525-geddc479)
I0820 11:03:57.268084 78469 start_master.go:309] Public master address is https://host1.example.com:443
F0820 11:05:23.804764 78469 start_master.go:100] could not reach etcd: 501: All the given peers are not reachable (failed to propose on members [https://host1.example.com:2379 https://host2.example.com:2379 https://host3.example.com:2379] twice [last error: Get https://host2.example.com:2379/v2/keys/?quorum=false&recursive=false&sorted=false: x509: certificate has expired or is not yet valid]) [0]
- I note the etcd master cert is expired
> for i in *.crt;do echo $i; openssl x509 -in $i -noout -enddate; done
admin.crt
notAfter=Jul 18 16:01:46 2016 GMT
ca.crt
notAfter=Jul 18 16:01:40 2016 GMT
etcd.server.crt
notAfter=Jul 18 16:01:49 2016 GMT
master.etcd-ca.crt
notAfter=Aug 18 16:18:12 2015 GMT
master.etcd-client.crt
notAfter=Jul 18 16:21:33 2016 GMT
master.kubelet-client.crt
notAfter=Jul 18 16:01:41 2016 GMT
master.server.crt
notAfter=Jul 18 16:01:42 2016 GMT
openshift-master.crt
notAfter=Jul 18 16:01:42 2016 GMT
openshift-registry.crt
notAfter=Jul 18 16:01:51 2016 GMT
openshift-router.crt
notAfter=Jul 18 16:01:49 2016 GMT
Environment
- Openshift Enterprise 3.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.