Grafana deployment failed CrashLoopBackOff

Latest response

While Grafana deployment both pods are getting CrashLoopBackOff

Below is the description and log output.

oc describe pod grafana-deployment-85b95cc867-7fmqj -n grafana

Name: grafana-deployment-85b95cc867-7fmqj
Namespace: grafana
Priority: 0
Node: worker4.asmprd01.ncrtc.co.in/10.48.132.11
Start Time: Sun, 12 Mar 2023 14:24:10 +0530
Labels: app=grafana
pod-template-hash=85b95cc867
Annotations: k8s.v1.cni.cncf.io/network-status:
[{
"name": "openshift-sdn",
"interface": "eth0",
"ips": [
"10.129.2.80"
],
"default": true,
"dns": {}
}]
k8s.v1.cni.cncf.io/networks-status:
[{
"name": "openshift-sdn",
"interface": "eth0",
"ips": [
"10.129.2.80"
],
"default": true,
"dns": {}
}]
openshift.io/scc: restricted
prometheus.io/port: 3000
prometheus.io/scrape: true
Status: Running
IP: 10.129.2.80
IPs:
IP: 10.129.2.80
Controlled By: ReplicaSet/grafana-deployment-85b95cc867
Init Containers:
grafana-plugins-init:
Container ID: cri-o://0f6d916dc61ef98925d7744e9bab4430a375f4f5c94a7b9152bddd58aaa12525
Image: quay.io/grafana-operator/grafana_plugins_init:0.1.0
Image ID: quay.io/grafana-operator/grafana_plugins_init@sha256:d01fd0b563187b1962b0bde24f6d52d001021d995d7a01dcf2bf7efabe2eab70
Port:
Host Port:
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 12 Mar 2023 14:24:13 +0530
Finished: Sun, 12 Mar 2023 14:24:13 +0530
Ready: True
Restart Count: 0
Limits:
cpu: 1
memory: 512Mi
Requests:
cpu: 250m
memory: 128Mi
Environment:
GRAFANA_PLUGINS:
Mounts:
/opt/plugins from grafana-plugins (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rsrbg (ro)
Containers:
grafana:
Container ID: cri-o://5f37247772a531a36bc9e9c9ef7aec613aea8a0e5a60bb70cf2210746aade920
Image: docker.io/grafana/grafana:7.5.17
Image ID: docker.io/grafana/grafana@sha256:15abb652aa82eeb9f45589278b34ae6ef0e96f74c389cadde31831eb0b1ce228
Port: 3000/TCP
Host Port: 0/TCP
Args:
-config=/etc/grafana/grafana.ini
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Sun, 12 Mar 2023 16:58:55 +0530
Finished: Sun, 12 Mar 2023 16:58:55 +0530
Ready: False
Restart Count: 35
Limits:
cpu: 500m
memory: 1Gi
Requests:
cpu: 100m
memory: 256Mi
Liveness: http-get http://:3000/api/health delay=60s timeout=30s period=10s #success=1 #failure=10
Readiness: http-get http://:3000/api/health delay=5s timeout=3s period=10s #success=1 #failure=1
Environment:
LAST_CONFIG:
LAST_DATASOURCES:
LAST_CREDENTIALS: bf8d1955d3c229eabd562a4d323a9e776c8b72bbec33ae296a2b859f4e4f5804
GF_SECURITY_ADMIN_USER: Optional: false
GF_SECURITY_ADMIN_PASSWORD: Optional: false
Mounts:
/etc/grafana/ from grafana-config (rw)
/etc/grafana/provisioning/dashboards from grafana-provision-dashboards (rw)
/etc/grafana/provisioning/datasources from grafana-datasources (rw)
/etc/grafana/provisioning/notifiers from grafana-provision-notifiers (rw)
/etc/grafana/provisioning/plugins from grafana-provision-plugins (rw)
/var/lib/grafana from grafana-data (rw)
/var/lib/grafana/plugins from grafana-plugins (rw)
/var/log/grafana from grafana-logs (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rsrbg (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
grafana-provision-plugins:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
grafana-provision-dashboards:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
grafana-provision-notifiers:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
grafana-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: grafana-config
Optional: false
grafana-logs:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
grafana-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: grafana-pvc
ReadOnly: false
grafana-plugins:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
grafana-datasources:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: grafana-datasources
Optional: false
kube-api-access-rsrbg:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional:
DownwardAPI: true
ConfigMapName: openshift-service-ca.crt
ConfigMapOptional:
QoS Class: Burstable
Node-Selectors:
Tolerations: node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 155m default-scheduler Successfully assigned grafana/grafana-deployment-85b95cc867-7fmqj to worker4.asmprd01.ncrtc.co.in
Normal AddedInterface 155m multus Add eth0 [10.129.2.80/23] from openshift-sdn
Normal Pulled 155m kubelet Container image "quay.io/grafana-operator/grafana_plugins_init:0.1.0" already present on machine
Normal Created 155m kubelet Created container grafana-plugins-init
Normal Started 155m kubelet Started container grafana-plugins-init
Normal Pulled 154m (x4 over 155m) kubelet Container image "docker.io/grafana/grafana:7.5.17" already present on machine
Normal Created 154m (x4 over 155m) kubelet Created container grafana
Normal Started 154m (x4 over 155m) kubelet Started container grafana
Warning BackOff 27s (x771 over 155m) kubelet Back-off restarting failed container

oc logs grafana-deployment-85b95cc867-7fmqj -n grafana

service init failed: failed to create admin user: no such column: help_flags1

Responses