3.3. Activity Events Collection

Before going in detail about how the Runtime Governance collects and processes the events, let us first understand what these activity events are. The Activity Model defines a set of events that are reported to Red Hat JBoss Fuse, in order to identify what is happening during the execution of a business transaction.

3.3.1. Activity Model

The Activity Model comprises the following components:
Activity Unit
The Activity Unit is the top-level and most important component of the Activity Model. The Activity Unit provides grouping capability. It aggregates a set of activities that relate to a particular transaction. The Activity Unit includes:
  • Id: This uniquely identifies the activity unit for historical retrieval purposes.
  • Origin: This information identifies the environment in which the activities were recorded.
  • A set of contexts: This provides contextual information to help relate the activities with other activities recorded in other units.
  • A group of activity types: This is the list of the actual activities or situations that occurred.
Origin
As mentioned above, Origin is a component of the Activity Unit. It represents information about the source of the activities associated with the Activity Unit. Origin provides the following information about an activity:
  • Principal: This represents the user (if any) associated with the activities.
  • Thread: This represents the execution path and is useful in diagnostic situations when used in conjunction with the host information.
  • Host: This is the host name.
  • Node: This is the name of the node, in the case when the server is part of a cluster.
Context
The Context items represent information used for correlating the activities within the unit against other Activity Units. It also provides identify information that may be useful when attempting to retrieve the unit. The Context contains the following information:
  • type: This is the context type.
  • value: This is the value of the context information.
  • timeframe: This is an optional value used with a Link context type, to identify the time period in which the context is valid.
Here is the list of various context types you can define:

Table 3.1. Context Types

Type Description
Context.Type.Conversation The conversation id, which can be used to correlate activities across service boundaries. It is unique to a particular business transaction instance.
Context.Type.Endpoint The endpoint id, which can be used to correlate activities within a service boundary such as a BPM process instance id. It is unique to a particular business transaction instance.
Context.Type.Message The unique id for a message being sent or received. The message id may only be valid within the scope of an endpoint, as it's value may not be carried with the message contents to the recipient. A common usage is to correlate a response against the originating request within the same endpoint.
Context.Type.Link This type represents a correlation between two activity events based on information that is unique and valid only for a limited time period.
Activity Type
All activity events are derived from an Activity Type superclass. This class has the following information:
  • Activity unit id
  • Activity unit index
  • Timestamp
  • Principal
  • A set of contexts
  • A set of properties
Out of these, the only piece of information that the reporting component needs provide is the Timestamp, and optionally some activity type specific contexts. The other information are initialized by the infrastructure before persisting the Activity Unit, in order to make the specific Activity Type instance locatable.

3.3.2. BPM and SOA Specific Activity Events

The BPM (Business Process Management) specific activity events are used to record the lifecycle and state transitions that occur when a business process (associated with a description language, such as BPMN2 or WS-BPEL) is executed within a runtime engine, in support of a business transaction. These business processes are long running, and they handle multiple requests and responses over a period of time, all being correlated to the same process instance. This means that the activities generated as a result of this execution must also correlate to:
  • The specific XA transaction in which they are performed,
  • The process instance that holds their state information in the BPM engine, and
  • The conversation associated with the particular business transaction.
However, this does not mean that all Activity Units that contain activity information from the BPM engine need to have these three types of correlation information. For example, the initial Activity Unit for a business process instance may identify the first two and establish a unique process instance id. A subsequent Activity Unit may then define the same process id for the second one, as well as the third one, that can then be used to tie any Activity Unit related with the process instance id to that conversation. Therefore, all Activity Units with the same process instance id become directly or indirectly correlated to the conversation id that may only be declared in some of the Activity Units.
Here is a list of BPM and SOA specific activity types:

Table 3.2. BPM Activity Types

Activity Type Description
ProcessStarted This activity type is recorded when a process instance is initially started. Attributes include: process type, instance id and version.
ProcessCompleted This activity type is recorded when a process instance completes. Attributes include: process type, instance id and status (either success or fail)
ProcessVariableSet This activity type is recorded when a process variable value is set or modified. Attributes include: process type, instance id and variable name/type/value.

Table 3.3. SOA Activity Types

Activity Type Description
RequestReceived and RequestSent This activity type is recorded when a service invocation (request) is received or sent. Attributes include: message type, content and message id.
ResponseReceived and ResponseSent This activity type is recorded when a service invocation returns. Attributes include: message type, content, message id and replyTo id (used to correlate the response to the original request)

3.3.3. Collecting Activity Events

The Runtime Governance Server collects activity information from events in the following two ways:
  • Integrating an Activity Collector into the execution environment.
  • Manually report the activity information to the Runtime Governance Server through a publicly available API such as REST service.

3.3.4. Integrating an Activity Collector

In this approach, the Runtime Governance Server integrates an Activity Collector into the execution environment. The Activity Collector intercepts activities and automatically reports them to the Runtime Governance Server. The Activity Collector makes use of Information Processors to enable information associated with the activity event to be pre-processed. The Activity Collector makes use of Activity Validators to install event processing capabilities. This section provides details on how to define and register Information Processors and Activity Validators.

3.3.4.1. Defining Information Processors

For integrating an Activity Collector inside the execution environment, you can use Information processors to extract relevant context and property values from activity events. Here is an example that illustrates the configuration of a single Information Processor: 
[{
        "name":"OrderManagementIP",
        "version":"1",
        "typeProcessors":{
                "{urn:switchyard-quickstart-demo:orders:1.0}submitOrder":{
                        "contexts":[{
                                "type":"Conversation",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"order/orderId"
                                }
                        }],
                        "properties":[{
                                "name":"customer",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"order/customer"
                                }
                        },{
                                "name":"item",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"order/itemId"
                                }
                        }]
                },
                "{urn:switchyard-quickstart-demo:orders:1.0}submitOrderResponse":{
                        "contexts":[{
                                "type":"Conversation",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"orderAck/orderId"
                                }
                        }],
                        "properties":[{
                                "name":"customer",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"orderAck/customer"
                                }
                        },{
                                "name":"total",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"orderAck/total"
                                }
                        }]
                },
                "org.switchyard.quickstarts.demos.orders.Order":{
                        "contexts":[{
                                "type":"Conversation",
                                "evaluator":{
                                        "type":"mvel",
                                        "expression":"orderId"
                                }
                        }],
                        "properties":[{
                                "name":"customer",
                                "evaluator":{
                                        "type":"mvel",
                                        "expression":"customer"
                                }
                        },{
                                "name":"itemId",
                                "evaluator":{
                                        "type":"mvel",
                                        "expression":"itemId"
                                }
                        }]
                },
                "org.switchyard.quickstarts.demos.orders.OrderAck":{
                        "contexts":[{
                                "type":"Conversation",
                                "evaluator":{
                                        "type":"mvel",
                                        "expression":"orderId"
                                }
                        }],
                        "properties":[{
                                "name":"customer",
                                "evaluator":{
                                        "type":"mvel",
                                        "expression":"customer"
                                }
                        },{
                                "name":"total",
                                "evaluator":{
                                        "type":"mvel",
                                        "expression":"total"
                                }
                        }]
                },
                "{urn:switchyard-quickstart-demo:orders:1.0}makePayment":{
                        "properties":[{
                                "name":"customer",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"payment/customer"
                                }
                        },{
                                "name":"amount",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"payment/amount"
                                }
                        }]
                },
                "{urn:switchyard-quickstart-demo:orders:1.0}makePaymentResponse":{
                        "properties":[{
                                "name":"customer",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"receipt/customer"
                                }
                        },{
                                "name":"amount",
                                "evaluator":{
                                        "type":"xpath",
                                        "expression":"receipt/amount"
                                }
                        }]
                },
                "org.switchyard.quickstarts.demos.orders.Receipt":{
                        "properties":[{
                                "name":"customer",
                                "evaluator":{
                                        "type":"mvel",
                                        "expression":"customer"
                                }
                        },{
                                "name":"amount",
                                "evaluator":{
                                        "type":"mvel",
                                        "expression":"amount"
                                }
                        }]
                },
                "org.switchyard.quickstarts.demos.orders.ItemNotFoundException":{
                        "script":{
                                "type":"mvel",
                                "expression":"activity.fault = \"ItemNotFound\""
                        }
                }
        }
}]

Table 3.4. Information Processor Example

Field Description
name Name of the Information Processor.
version Version of the Information Processor. If multiple versions Information Processor are installed with the same name, only the newest version is used.
typeProcesors The map of type processors . There is one map per type, with the type name being the map key.
The type processor element in the example is associated with a particular information type. The fields associated with this component are:

Table 3.5. Type Processor

Field Description
contexts List of context evaluators.
properties List of property evaluators.
script An optional script evaluator that is used to do any other processing that may be required, such as setting additional properties in the activity event that are not necessarily derived from message content information.
transformer An optional transformer that determines how this information type is represented in an activity event.

Table 3.6. Context Evaluator

Field Description
type The context type, such as Conversation, Endpoint, Message, or Link.
timeframe The number of milliseconds associated with a Link context type. If not specified, then the context is assumed to represent the destination of the link, so the source of the link must define the timeframe.
header The optional header name. If not defined, then the expression is applied to the information content to obtain the context value.
evaluator The expression evaluator used to derive the context value.
The context types represent different ways in which the activity events can be related to each other or to a logical grouping (for example, a business transaction). Not all activity events need to be associated directly with a global business transaction id. They can be indirectly associated based on transitive correlation. For example, if activity 1 is associated with the global business transaction id, activity 2 is associated with activity 1 by a message context type, and activity 3 is associated with activity 2 based on an endpoint correlation id, then all the three activity events will be collectively correlated to the business transaction id.

Table 3.7. Context Type

Context Type Description
Conversation A conversation identifier can be used to correlate activity events to a business transaction associated with a globally unique identifer such as an order id.
Endpoint A globally unique identifier associated with one endpoint in a business transaction. For example, a process instance id associated with the business process executing within a service playing a particular role in the business transaction.
Message A globally unique identifier of a message sent from one part to another.
Link A temporal link between a source and a destination activity. The temporal nature of the association is intended to enable non-globally unique details to be used to correlate activities, where the id is considered unique within the defined timeframe.

Table 3.8. Property Evaluator

Field Description
name The property name being initialized.
header The optional header name. If not defined, then the expression is applied to the information content to obtain the property value.
evaluator The expression evaluator used to derive the property value.
In the context and property evaluator components, you can reference an expression evaluator to derive their value. The expression evaluator has the following fields:

Table 3.9. Expression Evaluator

Field Description
type The type of expression evaluator to use. Only mvel or xpath are supported.
expression The expression to evaluate.
optional It indicates whether the value being extracted by the expression is optional. The default is false. If a value is not optional, but the expression fails to locate a value, then an error is reported.
These expressions operate on the information being processed, to return a string value to be applied to the appropriate context or property.
The Type processor also has a Script field, which has the following fields:

Table 3.10. Script

Field Description
type The type of expression evaluator to use. Currently only mvel is supported.
expression The expression to evaluate.
You can use the following variables with the MVEL script evaluator:
  • information: The information being processed.
  • activity: - The activity event.
    The transformer field of the Type Processor has the following fields:

Table 3.11. Transformer

Field Descrition
type The type of transformer to use. Currently only serialize and mvel are supported. .
The serialize transformer does not take any other properties. It attempts to convert the representation of the information into a textual form for inclusion in the activity event. So this transformer type can be used where the complete information content is required.
You can use the following variables with the MVEL transformer script:

Table 3.12. MVEL Transformer

Field Descrition
expression The mvel expression to transform the supplied information.

3.3.4.2. Registering the Information Processors

The Information Processors are deployed within the JEE container as a WAR file with the following structure: 
warfile
|
|-META-INF
|    |- beans.xml
|
|-WEB-INF
|    |-classes
|    |    |-ip.json
|    |    |-<custom classes/resources>
|    |
|    |-lib
|       |-ip-loader-jee.jar
|       |-<additional libraries>
The ip.json file contains the JSON representation of the Information Processor configuration. The ip-loader-jee.jar acts as a bootstrapper to load and register the Information Processors. If custom classes are defined, then the associated classes and resources can be defined in the WEB-INF/classes folder or within additional libraries located in the WEB-INF/lib folder. A maven pom.xml that creates this structure is shown below: 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <groupId>....</groupId>
        <artifactId>....</artifactId>
        <version>....</version>
        <packaging>war</packaging>
        <name>....</name>

        <properties>
                <rtgov.version>....</rtgov.version>
        </properties>

        <dependencies>
                <dependency>
                        <groupId>org.overlord.rtgov.activity-management</groupId>
                        <artifactId>activity</artifactId>
                        <version>${rtgov.version}</version>
                        <scope>provided</scope>
                </dependency>
                <dependency>
                        <groupId>org.overlord.rtgov.activity-management</groupId>
                        <artifactId>ip-loader-jee</artifactId>
                        <version>${rtgov.version}</version>
                </dependency>
                ....
        </dependencies>

</project>
If you are deploying in JBoss Application Server, then you also need to include the following fragment to define the dependency on the core Runtime Governance modules: 
.....
        <build>
                <finalName>....</finalName>
                <plugins>
                        <plugin>
                                <artifactId>maven-war-plugin</artifactId>
                                <configuration>
                                        <failOnMissingWebXml>false</failOnMissingWebXml>
                                        <archive>
                                                <manifestEntries>
                                                        <Dependencies>deployment.overlord-rtgov.war</Dependencies>
                                                </manifestEntries>
                                        </archive>
                                </configuration>
                        </plugin>
                </plugins>
        </build>
        .....

3.3.4.3. Defining the Activity Validators

The Activity Validator mechanism provides the means to install event validation capabilities within the activity collection environment. In some execution environments these validators can be implicitly called as part of collecting the activity events. However, in some environments (such as, SwitchYard) these validators need to be explicitly invoked, as they may need to impact the execution behavior (that is, block the transaction). The Activity Validator then invokes the validation capability and reacts to any issues it detects.
You can define an Activity Validator as an object model or specified as a JSON representation for packaging in a suitable form, and subsequently de-serialized when deployed to the governed execution environment. The following is an example of the JSON representation of a list of Activity Validators: 
[{
  "name" : "RestrictUsage",
  "version" : "1",
  "predicate" : {
    "@class" : "org.overlord.rtgov.ep.mvel.MVELPredicate",
    "expression" : "event instanceof org.overlord.rtgov.activity.model.soa.RequestReceived && event.serviceType == \"{urn:switchyard-quickstart-demo:orders:0.1.0}OrderService\""
  },
  "eventProcessor" : {
    "@class" : "org.overlord.rtgov.ep.mvel.MVELEventProcessor",
    "script" : "VerifyLastUsage.mvel",
    "services" : {
      "CacheManager" : {
        "@class" : "org.overlord.rtgov.common.infinispan.service.InfinispanCacheManager"
      }
    }
  }
}]
This example illustrates the configuration of a single Activity Validator with the following top level elements:

Table 3.13. Activity Validator Example

Field Description
name The name of the Activity Validator.
version The version of the Activity Validator. If multiple versions of the same named Activity Validator are installed, only the newest version is used. Versions can be expressed using Numeric, Dot Format, or any alpha numeric and symbols scheme.
predicate The optional implementation of the org.overlord.rtgov.ep.Predicate interface, used to determine if the activity event is relevant. If relevant, the event can be supplied to the event processor.
eventProcessor Defines the details for the event processor implementation being used. At a minimum, the value for this field should define a @class property to specify the Java class name for the event process implementation to use. Another general field that can be configured is the map of services that can be used by the event processor. Depending upon which implementation is selected, the other fields within the value apply to the event processor implementation.
When comparing versions, for example, when determining whether a newly deployed Activity Validator has a higher version than an existing one with the same name, then initially the versions are compared as numeric values. If either are not numeric, then they are compared using dot format, with each field being compared first as numeric values, and if not based on lexical comparison. If both fields do not have a dot, then they are compared lexically.

3.3.4.4. Registering the Activity Validators

The Activity Validators are deployed within the JEE container as a WAR file with the following structure: 
warfile
|
|-META-INF
|    |- beans.xml
|
|-WEB-INF
|    |-classes
|    |    |-av.json
|    |    |-<custom classes/resources>
|    |
|    |-lib
|       |-av-loader-jee.jar
|       |-<additional libraries>
The av.json file contains the JSON representation of the Activity Validator configuration. The av-loader-jee.jar acts as a bootstrapper to load and register the Activity Validators. If custom classes are defined, then the associated classes and resources can be defined in the WEB-INF/classes folder or within additional libraries located in the WEB-INF/lib folder. A maven pom.xml that creates this structure is as shown below: 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <groupId>....</groupId>
        <artifactId>....</artifactId>
        <version>....</version>
        <packaging>war</packaging>
        <name>....</name>

        <properties>
                <rtgov.version>....</rtgov.version>
        </properties>

        <dependencies>
                <dependency>
                        <groupId>org.overlord.rtgov.activity-management</groupId>
                        <artifactId>activity</artifactId>
                        <version>${rtgov.version}</version>
                        <scope>provided</scope>
                </dependency>
                <dependency>
                        <groupId>org.overlord.rtgov.activity-management</groupId>
                        <artifactId>av-loader-jee</artifactId>
                        <version>${rtgov.version}</version>
                </dependency>
                ....
        </dependencies>

</project>
If deploying in JBoss Application Server, then you also need to include the following fragment to define the dependency on the core Runtime Governance modules: 
.....
        <build>
                <finalName>....</finalName>
                <plugins>
                        <plugin>
                                <artifactId>maven-war-plugin</artifactId>
                                <configuration>
                                        <failOnMissingWebXml>false</failOnMissingWebXml>
                                        <archive>
                                                <manifestEntries>
                                                        <Dependencies>deployment.overlord-rtgov.war</Dependencies>
                                                </manifestEntries>
                                        </archive>
                                </configuration>
                        </plugin>
                </plugins>
        </build>
        .....

3.3.5. Reporting and Querying Activity Events via Rest

In this approach, the Runtime Governance Server manually reports the activity information from the activity events to the Runtime Governance Server through a publicly available API such as REST service. This section provides details on how the Runtime Governance Server reports the activity event information, queries the events for relevant information, retrieves an Activity Unit, and retrieves events with a context value.

3.3.5.1. Reporting Activity Information

You can report Activity Information by using the POST request to the following URL: 
<host>/overlord-rtgov/activity/store
The request contains the list of ActivityUnit objects encoded in JSON. Here is an example: 
[{
    "id":"TestId1",
    "activityTypes":[{
        "type":"RequestSent",
        "context":[{
            "value":"12345"
        },{
            "value":"abc123",
            "type":"Endpoint"
        },{
            "value":"ABC123",
            "type":"Message"
        }],
        "content":"....",
        "serviceType":"{http://service}OrderService",
        "operation":"buy",
        "fault":"MyFault",
        "messageType":"{http://message}OrderRequest",
        "timestamp":1347028592880
    },{
        "type":"ResponseReceived",
        "context":[{
            "value":"12345"
        },{
            "value":"ABC124",
            "type":"Message"
        }],
        "content":"....",
        "serviceType":"{http://service}OrderService",
        "operation":"buy",
        "fault":"OutOfStock",
        "messageType":"{http://message}OutOfStock",
        "replyToId":"ABC123",
        "timestamp":1347028593010
    }],
    "origin":{
        "host":"Saturn",
        "principal":"Fred",
        "node":"Saturn1",
        "thread":"Thread-1"
    }
},{
    .....
}]

3.3.5.2. Querying Activity Events Using an Expression

You can query activity events using an expression from a POST request to the following URL: 
<host>/overlord-rtgov/activity/query
The request contains the JSON encoding of the Query Specification, which has the following properties:

Table 3.14. JSON Encoding Properties

Property Description
fromTimestamp Specifies (optionally) the start date/time for the activity units required. If not specified, then the query applies to the first recorded activity unit.
toTimestamp Specifies (optionally) the end date/time for the activity units required. If not specified, then the query relates to the most recently recorded activity units.
expression An expression (optional) that can be used to specify the activity events of interest.
format Specifies (optionally) the format of the expression. The configured activity store must support this value.

3.3.5.3. Retrieving an Activity Unit

You can retrieve an Activity Unit from a GET request to the following URL: 
<host>/overlord-rtgov/activity/unit?id=<unitId>
Here, the unitId represents the identifier associated with the retrieved ActivityUnit.

3.3.5.4. Retrieve Activity Events Associated with a Context Value

You can retrieve activity events associated with a context value from a GET request to the following URL: 
<host>/overlord-rtgov/activity/events?type=<contextType>&value=<identifier>
The service uses basic authentication by default, using a valid JBoss EAP Application Realm user (You can use the Governance user you configured when you installed Fuse 6).
Here,
  • The <contextType> represents the context type, such as Conversation, Endpoint, Message or Link. For more information, see the API documentation for org.overlord.rtgov.activity.model.Context.Type.
  • The <identifier> represents the correlation value associated with the ActivityType(s) that are being retrieved.
You can provide two additional optional query parameters, start for the start timestamp, and end for the end timestamp. You can use these parameters to scope the time period of the query.
The response is a list of ActivityType objects encoded in JSON. For more information, see the API documentation for org.overlord.rtgov.activity.model.ActivityType.