The polling frequency of each of the provider-specific event catchers is defined in the :event_catcher section of the Configuration→Advanced settings. The default settings for CloudForms Management Engine 5.8 are as follows:

    :event_catcher:
        :poll: 1.seconds
      :event_catcher_ansible_tower:
        :poll: 20.seconds
      :event_catcher_embedded_ansible:
        :poll: 20.seconds
      :event_catcher_redhat:
        :poll: 15.seconds
      :event_catcher_openstack:
        :poll: 15.seconds
      :event_catcher_openstack_infra:
        :poll: 15.seconds
      :event_catcher_openstack_network:
        :poll: 15.seconds
      :event_catcher_hawkular:
        :poll: 10.seconds
      :event_catcher_hawkular_datawarehouse:
        :poll: 1.minute
      :event_catcher_google:
        :poll: 15.seconds
      :event_catcher_kubernetes:
        :poll: 1.seconds
      :event_catcher_lenovo:
        :poll: 4.minutes
      :event_catcher_openshift:
        :poll: 1.seconds
      :event_catcher_cinder:
        :poll: 10.seconds
      :event_catcher_swift:
        :poll: 10.seconds
      :event_catcher_amazon:
        :poll: 15.seconds
      :event_catcher_azure:
        :poll: 15.seconds
      :event_catcher_vmware:
        :poll: 1.seconds
      :event_catcher_vmware_cloud:
        :poll: 15.seconds

VMware vCenter management systems use an event type called EventEx as a catch-all event. Several VMware components issue EventEx events with a subtype to record state changes, problems, and recovery from problems. They appear as [EventEx]-[subtype], for example: 

Until the cause of the event storm is identified and corrected, the quickest way to restore any operation for the CloudForms environment is to to prevent the continued growth of the miq_queue table. The simplest techniques are to blacklist the event(s) causing the storm (see Section 9.4.1, “Blacklisting Events”), or to disable the event monitor role on all CFME appliance in the provider’s zone. 

In critical situations with many hundreds of thousands to millions of queued messages, it may be necessary to selectively delete message instances from the miq_queue table. Since the overwhelming number of messages expected to be in this table will be of type 'event', the following SQL statement can be used to remove all such instances from the miq_queue table:

delete from miq_queue where role = 'event' and class_name = 'EmsEvent';

Before running this query the following points should be noted:

Some provider events occur relatively frequently, but are either uninteresting to CloudForms, or processing them would consume excessive resources (such as those typically associated with event storms). Events such as these can be skipped or blacklisted. The event catchers write a list of blacklisted events to evm.log when they start, for example:

... MIQ(ManageIQ::Providers::Redhat::InfraManager::EventCatcher:: ⏎
Runner#after_initialize) EMS [rhevm.bit63.net] as [cfme@internal] ⏎
Event Catcher skipping the following events:
... INFO -- :   - UNASSIGNED
... INFO -- :   - USER_REMOVE_VG
... INFO -- :   - USER_REMOVE_VG_FAILED
... INFO -- :   - USER_VDC_LOGIN
... INFO -- :   - USER_VDC_LOGIN_FAILED
... INFO -- :   - USER_VDC_LOGOUT

These events are defined in the blacklisted_events table in the VMDB. The default rows in the table are as follows:

vmdb_production=# select event_name,provider_model ⏎
from blacklisted_events;
               event_name               |    provider_model
----------------------------------------+------------------------------
 storageAccounts_listKeys_BeginRequest  | ...Azure::CloudManager
 storageAccounts_listKeys_EndRequest    | ...Azure::CloudManager
 identity.authenticate                  | ...Openstack::CloudManager
 scheduler.run_instance.start           | ...Openstack::CloudManager
 scheduler.run_instance.scheduled       | ...Openstack::CloudManager
 scheduler.run_instance.end             | ...Openstack::CloudManager
 ConfigurationSnapshotDeliveryCompleted | ...Amazon::CloudManager
 ConfigurationSnapshotDeliveryStarted   | ...Amazon::CloudManager
 ConfigurationSnapshotDeliveryFailed    | ...Amazon::CloudManager
 UNASSIGNED                             | ...Redhat::InfraManager
 USER_REMOVE_VG                         | ...Redhat::InfraManager
 USER_REMOVE_VG_FAILED                  | ...Redhat::InfraManager
 USER_VDC_LOGIN                         | ...Redhat::InfraManager
 USER_VDC_LOGOUT                        | ...Redhat::InfraManager
 USER_VDC_LOGIN_FAILED                  | ...Redhat::InfraManager
 AlarmActionTriggeredEvent              | ...Vmware::InfraManager
 AlarmCreatedEvent                      | ...Vmware::InfraManager
 AlarmEmailCompletedEvent               | ...Vmware::InfraManager
 AlarmEmailFailedEvent                  | ...Vmware::InfraManager
 AlarmReconfiguredEvent                 | ...Vmware::InfraManager
 AlarmRemovedEvent                      | ...Vmware::InfraManager
 AlarmScriptCompleteEvent               | ...Vmware::InfraManager
 AlarmScriptFailedEvent                 | ...Vmware::InfraManager
 AlarmSnmpCompletedEvent                | ...Vmware::InfraManager
 AlarmSnmpFailedEvent                   | ...Vmware::InfraManager
 AlarmStatusChangedEvent                | ...Vmware::InfraManager
 AlreadyAuthenticatedSessionEvent       | ...Vmware::InfraManager
 EventEx                                | ...Vmware::InfraManager
 UserLoginSessionEvent                  | ...Vmware::InfraManager
 UserLogoutSessionEvent                 | ...Vmware::InfraManager
 identity.authenticate                  | ...Openstack::InfraManager
 scheduler.run_instance.start           | ...Openstack::NetworkManager
 scheduler.run_instance.scheduled       | ...Openstack::NetworkManager
 scheduler.run_instance.end             | ...Openstack::NetworkManager
 ConfigurationSnapshotDeliveryCompleted | ...Amazon::NetworkManager
 ConfigurationSnapshotDeliveryStarted   | ...Amazon::NetworkManager
 ConfigurationSnapshotDeliveryFailed    | ...Amazon::NetworkManager
(37 rows)

If processing of any of the events in the blacklisted_events table is required, the enabled field can be set to false and the provider-specific event catcher restarted.

An EMS can also report some minor object property changes as events, even though these are not modelled in the CloudForms VMDB. For VMware providers such event types can be added to the "Vim Broker Exclude List" so that they can be discarded without processing. The exclude list is found under :broker_notify_properties in the Configuration → Advanced settings, as follows:

:broker_notify_properties:
  :exclude:
    :HostSystem:
    - config.consoleReservation
    - config.dateTimeInfo
    - config.network
    - config.service
    - summary
    - summary.overallStatus
    - summary.runtime.bootTime
    - summary.runtime.healthSystemRuntime.systemHealthInfo. ⏎
         numericSensorInfo
    :VirtualMachine:
    - config.locationId
    - config.memoryAllocation.overheadLimit
    - config.npivWorldWideNameType
    - guest.disk
    - guest.guestFamily
    - guest.guestFullName
    - guest.guestId
    - guest.ipStack
    - guest.net
    - guest.screen
    - guest.screen.height
    - guest.screen.width
    - guest.toolsRunningStatus
    - guest.toolsStatus
    - resourceConfig
    - summary
    - summary.guest.guestFullName
    - summary.guest.guestId
    - summary.guest.toolsRunningStatus
    - summary.overallStatus
    - summary.runtime.bootTime
    - summary.runtime.memoryOverhead
    - summary.runtime.numMksConnections
    - summary.storage
    - summary.storage.committed
    - summary.storage.unshared

CloudForms recently introduced the concept of flood monitoring for the provider-specific event catchers. This stops provider events from being queued when too many duplicates are received in a short time. By default an event is considered as flooding if it is received 30 times in one minute.

Flood monitoring is a generic concept for event processing, but requires the appropriate supporting methods to be added to each provider. As of CloudForms Management Engine 5.8 only the VMware provider supports this functionality.

The :event_catcher section is one of the largest of the Configuration → Advanced settings, and it defines the configuration of each type of event catcher. For example the following extract shows the settings for the ManageIQ::Providers::Openstack::InfraManager::EventCatcher worker:

    :event_catcher:
...
      :event_catcher_openstack:
        :poll: 15.seconds
        :topics:
          :nova: notifications.*
          :cinder: notifications.*
          :glance: notifications.*
          :heat: notifications.*
        :duration: 10.seconds
        :capacity: 50
        :amqp_port: 5672
        :amqp_heartbeat: 30
        :amqp_recovery_attempts: 4
        :ceilometer:
          :event_types_regex: "\\A(?!firewall|floatingip|gateway| ⏎
          net|port|router|subnet|security_group|vpn)"
...

The configuration settings rarely need to be changed from their defaults.