Oftentimes, when working with records of significant change of state in the application domain at a given point in time, known as events, it becomes necessary to identify and analyze patterns in such a way as to deduce higher-level impacts, time-based correlations, and relationships that can factor into or directly affect business processes, application code, decision-making, and the like. As a quick example, consider the gaming industry, where the actions performed by a player or set of players must be processed quickly and synchronously in order to best process their intent. The financial sector also lends itself to a simplified use case, wherein credit card transactions over a given period of time can be analyzed in order to assist in fraud detection and prevention. This process of detecting and selecting interesting events from within an event cloud, finding their relationships, and inferring new data is known as Complex Event Processing, or CEP.

When dealing with large amounts of events in a synchronous way, or even when considering time correlations amongst them, it’s important to recognize high-availability as a requirement for an enterprise application related to CEP. Without consistency and redundancy, significant events can be overlooked or missed entirely, thus creating the possibility of exclusions or faulty assumptions concluded from processing an event cloud, or group of events.

Likewise, when considering the sheer volume of events possible within various application spaces, which still must be consumed in a consistent and reliable fashion, scalability becomes a factor. While early on, it’s possible to simply increase hardware capacity for short-term gains, eventually, this model becomes unfeasible and thus, partitioning of capabilities and processing becomes a necessity.

This reference architecture reviews the HACEP, or Highly-Available and Scalable Complex Event Processing, framework and walks through the deployment and implementation of a few simplistic examples showcasing the various features that the framework offers. The goal of this reference architecture is to provide a thorough description of the steps required for establishing a highly-available, scalable, and CEP-capable architecture stack, while citing the rationale for inclusion of various toolings or patterns when applicable and the challenges overcome by utilizing the provided HACEP framework.