- Openshift Online
- Openshift Enterprise 2.2
- Metrics cartridge for Openshift Online (without dedicated node)
- Metrics collection for Openshift Enterprise or Openshift Online (with dedicated node)
- Monitoring cartridge in Openshift
- We need to provide a way to monitor applications and gears on our OSE platform. Our operations teams use tools like Patrol, Nagios or Zabbix to monitor traditionnal applications (deployed on virtual machine) but we need to adapt this to OpenShift V2 architecture and gears.
Metrics cartridge for Openshift Online (without dedicated node)
There are options to use downloadable/custom cartridges as below with your application :
Note: These cartridges can be used with Openshift application/gears but these cartridges are not supported by us.
Metrics collection for Openshift Enterprise or Openshift Online (with dedicated node)
As of OSE 2.1, metrics are basically just implemented as log messages that begin with “type=metric”. These can be generated in a number of ways:
The application itself can actively generate log messages at any time; if your application framework provides a scheduler, just have it periodically output to stdout beginning with “type=metric” and logshifter will bundle these messages into the rest of your gear logs for analysis.
Edit 2014-06-25: Note that these have a different format and tag than the watchman-generated metrics described next, which appear under the “openshift-platform” tag and aren’t processed by the mmopenshift rsyslog plugin. So you may need to do some work to have your log analyzer consider these metrics.
Metrics can be passively generated in a run by the openshift-watchman service in a periodic node-wide run. This can generate metrics in several ways:
By default it generates standard metrics out of cgroups for every gear. These include RAM, CPU, and storage.
Each cartridge can indicate in its manifest that it supports metrics , in which case the bin/metrics script is executed and its output is logged as metrics. No standard cartridges shipped with OSE support metrics at this time, but custom cartridges could.
Each application can create a metrics action hook script in its git repo, which is executed with each watchman run and its output logged as metrics. This enables the application owner to add custom metrics per app.
It should be noted that the cartridge and action hook metrics scripts have a limited time to run, so that they can’t indefinitely block the metrics run for the rest of the gears on the node. All of this is configurable with watchman settings in node.conf. Also it should be noted that watchman-generated logs are tagged with “openshift-platform” e.g.:
Jun 10 16:25:39 vm openshift-platform: type=metric appName=php6 gear=53961099e659c55b08000102 app=53961099e659c55b08000102 ns=demo quota.blocks.used=988 quota.blocks.limit=1048576 quota.files.used=229 quota.files.limit=80000
The example rsyslog7 and openshift-watchman configuration will route watchman-generated entries differently from application-server entries since the app UUID parameter is specified differently (“app=” vs “appUuid=”). This is all very configurable.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.