JON Agent takes too much time for metric collection

Solution Verified - Updated -

Issue

  • JON agent using high CPU

  • With agent debug logging enabled, the following messages appear:

    DEBUG [MeasurementManager.collector-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection is falling behind... Missed requested time by [32145ms]

  • Agent uses up to 95% CPU

  • Datasources and other resources are not showing metrics
  • Not getting metrics for Resources under each JVM
  • Not getting metrics for Datasource (current active connections, etc..)
  • Not getting metrics for AJP connectors (active thread count, etc.,)
  • RHQ agent takes too much time for metric collection
  • Agent performance issues may be causing false DOWN alerts
  • Agent performance issues may be causing missed metric collections
  • Metric collection is taking more then 25 seconds each time regardless of the number of metrics being collected: 
    • INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [1186] metrics took 30497ms - sending report to Server...
INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [9] metrics took 22947ms - sending report to Server...
INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [756] metrics took 27429ms - sending report to Server...
INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [8] metrics took 22377ms - sending report to Server...
INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [480] metrics took 29827ms - sending report to Server...
INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [14] metrics took 24391ms - sending report to Server...
  • Total Time Collecting Measurements per Minute (Agent Measurement Subsystem) shows more then 52 seconds
  • Agent's metric collection is falling behind metric collection intervals
  • Metric collection is falling behind on an agent resulting in missed metric collections
  • Agent logs contain messages similar to: 
    • INFO  [MeasurementManager.collector-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- [PERF] Collection of measurements for [org.rhq.plugins.jbossas5.WebApplicationContextComponent@b51d49a] took [4254ms]

Environment

  • JBoss Operations Network (ON) 2.4, 2.4.1, 2.4.2, 3.0, 3.0.1, 3.1, 3.1.1, 3.1.2, 3.2, 3.3
  • ON agent monitoring at least one server resource

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.