After upgrade from JBoss ON 3.2.3 to 3.3.0, some rhq column families are unavailable and compaction operations fail
Issue
- After upgrade from JBoss ON 3.2.3 to JBoss ON 3.3.0, some column families are reported as unavailable: one_hour_metrics, six_hour_metrics, twenty_four_hour_metrics and metrics_index.
- On disk the corresponding directories are not accessed any more and previously scheduled manual compaction task fails.
- Manual compaction operation fails after JBoss ON upgrade from 3.2.3 to 3.3.0
- Manual compaction operation fails with the exception:
org.mc4j.ems.connection.EmsInvocationException: Exception on invocation of [forceTableCompaction]javax.management.RuntimeMBeanException: java.lang.IllegalArgumentException: Unknown table/cf pair (rhq.metrics_index)
at org.mc4j.ems.impl.jmx.connection.bean.operation.DOperation.invoke(DOperation.java:133)
at org.rhq.plugins.cassandra.util.KeyspaceService.compact(KeyspaceService.java:101)
at org.rhq.plugins.cassandra.KeyspaceComponent.compactKeyspace(KeyspaceComponent.java:168)
at org.rhq.plugins.cassandra.ColumnFamilyComponent.invokeOperation(ColumnFamilyComponent.java:70)
...
Caused by: javax.management.RuntimeMBeanException: java.lang.IllegalArgumentException: Unknown table/cf pair (rhq.metrics_index)
...
Caused by: java.lang.IllegalArgumentException: Unknown table/cf pair (rhq.metrics_index)
at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:168)
at org.apache.cassandra.service.StorageService.getValidColumnFamilies(StorageService.java:2407)
at org.apache.cassandra.service.StorageService.forceTableCompaction(StorageService.java:2246)
...
Environment
- Red Hat JBoss Operations Network (JON) - upgrade done from 3.2.3 to 3.3.0 version;
- In 3.2.3 version - scheduled
Force Major Column Family Compactionoperation for folders:metrics_index,one_hour_metrics,raw_metrics,schema_version,six_hour_metricsandtwenty_four_hour_metrics. - Above operation is scheduled to be executed
nowandrepeated indefinitely.
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.