Chapter 2. Software Stack

2.1. Framework

Numerous frameworks are available for building microservices, and each provides various advantage and disadvantages. This reference architecture focuses on a microservice architecture built on top of the Spring Boot framework. The Spring Boot framework can use various versions of Tomcat, Jetty and Undertow as its embedded servlet containers. This paper focuses on the use of Spring Boot with an embedded Tomcat server, running on an OpenShift base image from Red Hat®, with a supported JVM and environment.

2.2. Client Library

2.2.1. Overview

While invoking a microservice is typically a simple matter of sending a JSON or XML payload over HTTP, various considerations have led to the prevalence of specialized client libraries, particularly in a Spring Boot environment. These libraries provide integration with not only Spring Boot, but also many other tools and libraries often required in a microservice architecture.

2.2.2. Ribbon

Ribbon is an Inter-Process Communication (remote procedure calls) library with built-in client-side load balancers. The primary usage model involves REST calls with various serialization scheme support.

This reference architecture uses Ribbon, without relying on it for much intelligence. The main reason for including and using Ribbon is its prevalence in Spring Boot microservice applications, and relatedly, its support for and integration with various tools and libraries commonly used in such applications.

2.2.3. gRPC

The more modern gRPC is a replacement for Ribbon that’s been developed by Google and adopted by a large number of projects.

While Ribbon uses simple text-based JSON or XML payloads over HTTP, gRPC relies on Protocol Buffers for faster and more compact serialization. The payload is sent over HTTP/2 in binary form. The result is better performance and security, at the expense of compatibility and tooling support in the existing market.

2.3. Service Registry

2.3.1. Overview

Microservice architecture often implies dynamic scaling of individual services, in a private, hybrid or public cloud where the number and address of hosts cannot always be predicted or statically configured in advance. The solution is the use of a service registry as a starting point for discovering the deployed instances of each service. This will often be paired by a client library or load balancer layer that seamlessly fails over upon discovering that an instance no longer exists, and caches service registry lookups. Taking things one step further, integration between a client library and the service registry can make this lookup and invoke process into a single step, and transparent to developers.

In modern cloud environments, such capability is often provided by the platform, and service replication and scaling is a core feature. This reference architecture is built on top of OpenShift, therefore benefiting from the Kubernetes Service abstraction.

2.3.2. Eureka

Eureka is a REST (REpresentational State Transfer) based service that is primarily used in the AWS cloud for locating services for the purpose of load balancing and failover of middle-tier servers.

Tight integration between Ribbon and Eureka allows declarative use of Eureka when the caller is using the Ribbon library.

2.3.3. Consul

Consul is a tool for discovering and configuring services in your infrastructure. It is provided both as part of the HashiCorp enterprise suite of software, as well as an open source component that is used in the Spring Cloud.

Integration with Ribbon within a Spring Cloud environment allows transparent and declarative lookups of services registered with Consul.

2.3.4. ZooKeeper

Apache ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

Once again, the support of ZooKeeper within Spring Cloud environments and integration with Ribbon allows declarative lookups of service instances before invocation.

2.3.5. OpenShift

In OpenShift, a Kubernetes service serves as an internal load balancer. It identifies a set of replicated pods in order to proxy the connections it receives to them. Additional backing pods can be added to, or removed from a service, while the service itself remains consistently available, enabling anything that depends on the service to refer to it through a consistent address.

Contrary to a third-party service registry, the platform in charge of service replication can provide a current and accurate report of service replicas at any moment. The service abstraction is also a critical platform component that is as reliable as the underlying platform itself. This means that the client does not need to keep a cache and account for the failure of the service registry itself. Ribbon can be declaratively configured to use OpenShift instead of a service registry, without any code changes.

2.4. Load Balancer

2.4.1. Overview

For client calls to stateless services, high availability (HA) translates to a need to look up the service from a service registry, and load balance among available instances. The client libraries previously mentioned include the ability to combine these two steps, but OpenShift makes both actions redundant by including load balancing capability in the service abstraction. OpenShift provides a single address where calls will be load balanced and redirected to an appropriate instance.

2.4.2. Ribbon

Ribbon allows load balancing among a static list of instances that are declared, or however many instances of the service that are discovered from a registry lookup.

2.4.3. gRPC

gRPC also provides load balancing capability within the same library layer.

2.4.4. OpenShift Service

OpenShift provides load balancing through its concept of service abstraction. The cluster IP address exposed by a service is an internal load balancer between any running replica pods that provide the service. Within the OpenShift cluster, the service name resolves to this cluster IP address and can be used to reach the load balancer. For calls from outside and when going through the router is not desirable, an external IP address can be configured for the service.

2.5. Circuit Breaker

2.5.1. Overview

The highly distributed nature of microservices implies a higher risk of failure of a remote call, as the number of such remote calls increases. The circuit breaker pattern can help avoid a cascade of such failures by isolating problematic services and avoiding damaging timeouts.

2.5.2. Hystrix

Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.

Hystrix implements both the circuit breaker and bulkhead patterns.

2.6. Externalized Configuration

2.6.1. Overview

Externalized configuration management solutions can provide an elegant alternative to the typical combination of configuration files, command line arguments, and environment variables that are used to make applications more portable and less rigid in response to outside changes. This capability is largely dependent on the underlying platform and is provided by ConfigMaps in OpenShift.

2.6.2. Spring Cloud Config

Spring Cloud Config provides server and client-side support for externalized configuration in a distributed system. With the Config Server you have a central place to manage external properties for applications across all environments.

2.6.3. OpenShift ConfigMaps

ConfigMaps can be used to store fine-grained information like individual properties, or coarse-grained information like entire configuration files or JSON blobs. They provide mechanisms to inject containers with configuration data while keeping containers agnostic of OpenShift Container Platform.

2.7. Distributed Tracing

2.7.1. Overview

For all its advantages, a microservice architecture is very difficult to analyze and troubleshoot. Each business request spawns multiple calls to, and between, individual services at various layers. Distributed tracing ties all individual service calls together, and associates them with a business request through a unique generated ID.

2.7.2. Sleuth/Zipkin

Spring Cloud Sleuth generates trace IDs for every call and span IDs at the requested points in an application. This information can be integrated with a logging framework to help troubleshoot the application by following the log files, or broadcast to a Zipkin server and stored for analytics and reports.

2.7.3. Jaeger

Jaeger, inspired by Dapper and OpenZipkin, is an open source distributed tracing system that fully conforms to the Cloud Native Computing Foundation (CNCF) OpenTracing standard. It can be used for monitoring microservice-based architectures and provides distributed context propagation and transaction monitoring, as well as service dependency analysis and performance / latency optimization.

2.8. Proxy/Routing

2.8.1. Overview

Adding a proxy in front of every service call enables the application of various filters before and after calls, as well as a number of common patterns in a microservice architecture, such as A/B testing. Static and dynamic routing rules can help select the desired version of a service.

2.8.2. Zuul

Zuul is an edge service that provides dynamic routing, monitoring, resiliency, security, and more. Zuul supports multiple routing models, ranging from declarative URL patterns mapped to a destination, to groovy scripts that can reside outside the application archive and dynamically determine the route.

2.8.3. Istio

Istio is an open platform-independent service mesh that provides traffic management, policy enforcement, and telemetry collection. Istio is designed to manage communications between microservices and applications. Istio is still in pre-release stages.

Red Hat is a participant in the Istio project.