JBoss ON storage node log indicates GCInspector.java reports Heap is full and you may need to reduce memtable and/or cache sizes

Solution Unverified - Updated -

Issue

  • rhq-storage.log includes repeated WARN messages:

    GCInspector.java (line 142) Heap is 0.7637708699279387 full.  You may need to reduce memtable and/or cache sizes.  Cassandra will now flush up to the two largest memtables to free up memory.  Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
    StorageService.java (line 3616) Flushing CFS(Keyspace='rhq', ColumnFamily='raw_metrics') to relieve memory pressure
    GCInspector.java (line 142) Heap is 0.7517777149403199 full.  You may need to reduce memtable and/or cache sizes.  Cassandra will now flush up to the two largest memtables to free up memory.  Adjust flush_largest_memtables_at threshold in cassandra.yaml if you don't want Cassandra to do this automatically
    StorageService.java (line 3729) Flushing CFS(Keyspace='rhq', ColumnFamily='aggregate_metrics') to relieve memory pressure
    
  • OutOfMemoryError: Java heap space in rhq-storage.log:

    ERROR [FlushWriter:1] ... CassandraDaemon.java (line 193) Exception in thread Thread[FlushWriter:1,5,main] 
    java.lang.OutOfMemoryError: Java heap space
    

Environment

  • Red Hat JBoss Operations Network (ON) 3.2, 3.3
  • JBoss ON Storage Node

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content