JON fails to show metrics for monitored resources due storage node AuthenticationException
Issue
- Stopped showing metrics for all A-MQ queues
- Stopped receiving any alerts due to metrics not collected
- Metrics are not being stored by the storage node
- Storage client subsystem wasn't initialized
-
Server log contains the following warning:
WARN [org.rhq.enterprise.server.storage.StorageClientManagerBean] (pool-7-thread-1) Storage client subsystem wasn't initialized. The RHQ server will be set to MAINTENANCE mode. Please verify that the storage cluster is operational.: java.lang.RuntimeException: Authentication error on host jboss-on-storage-01.example.com/192.168.5.50: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level ONE at org.rhq.enterprise.server.storage.StorageClientManagerBean.checkSchemaCompability(StorageClientManagerBean.java:257) [rhq-server.jar:4.9.0.JON320GA] at org.rhq.enterprise.server.storage.StorageClientManagerBean.createSession(StorageClientManagerBean.java:341) [rhq-server.jar:4.9.0.JON320GA] at org.rhq.enterprise.server.storage.StorageClientManagerBean.init(StorageClientManagerBean.java:154) [rhq-server.jar:4.9.0.JON320GA] ... at org.rhq.enterprise.server.storage.StorageClientManagerBean$$$view133.init(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] at org.rhq.enterprise.server.core.StartupBean.initStorageClient(StartupBean.java:485) [rhq-server.jar:4.9.0.JON320GA] at org.rhq.enterprise.server.core.StartupBean.init(StartupBean.java:200) [rhq-server.jar:4.9.0.JON320GA] ... Caused by: com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host jboss-on-storage-01.example.com/192.168.5.50: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level ONE at com.datastax.driver.core.Connection.initializeTransport(Connection.java:168) [cassandra-driver-core-1.0.2.jar:] at com.datastax.driver.core.Connection.<init>(Connection.java:129) [cassandra-driver-core-1.0.2.jar:] at com.datastax.driver.core.Connection.<init>(Connection.java:56) [cassandra-driver-core-1.0.2.jar:] at com.datastax.driver.core.Connection$Factory.open(Connection.java:412) [cassandra-driver-core-1.0.2.jar:] at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:205) [cassandra-driver-core-1.0.2.jar:] ... at org.rhq.cassandra.util.ClusterBuilder.build(ClusterBuilder.java:130) [rhq-cassandra-util-4.9.0.JON320GA.jar:4.9.0.JON320GA] at org.rhq.cassandra.schema.SessionManager.initSession(SessionManager.java:33) [rhq-cassandra-schema-4.9.0.JON320GA.jar:4.9.0.JON320GA] at org.rhq.cassandra.schema.AbstractManager.initClusterSession(AbstractManager.java:100) [rhq-cassandra-schema-4.9.0.JON320GA.jar:4.9.0.JON320GA] at org.rhq.cassandra.schema.AbstractManager.initClusterSession(AbstractManager.java:90) [rhq-cassandra-schema-4.9.0.JON320GA.jar:4.9.0.JON320GA] at org.rhq.cassandra.schema.VersionManager.checkCompatibility(VersionManager.java:257) [rhq-cassandra-schema-4.9.0.JON320GA.jar:4.9.0.JON320GA] at org.rhq.cassandra.schema.SchemaManager.checkCompatibility(SchemaManager.java:112) [rhq-cassandra-schema-4.9.0.JON320GA.jar:4.9.0.JON320GA] at org.rhq.enterprise.server.storage.StorageClientManagerBean.checkSchemaCompability(StorageClientManagerBean.java:253) [rhq-server.jar:4.9.0.JON320GA] ... 149 more
Environment
- Red Hat JBoss Operations Network (ON) 3.2
- At least two JBoss ON storage nodes installed in a single storage cluster
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
