Chapter 8. Integrate Spring Boot with Kubernetes

8.1. Introduction to Spring Boot with Kubernetes Integration

8.1.1. What are we Integrating?

The Spring Cloud Kubernetes plug-in currently enables you to integrate the following features of Spring Boot and Kubernetes:

8.1.2. Spring Boot Externalized Configuration

In Spring Boot, externalized configuration is the mechanism that enables you to inject configuration values from external sources into Java code. In your Java code, injection is typically enabled by annotating with the @Value annotation (to inject into a single field) or the @ConfigurationProperties annotation (to inject into multiple properties on a Java bean class).

The configuration data can come from a wide variety of different sources (or property sources). In particular, configuration properties are often set in a project’s application.properties file (or application.yaml file, if you prefer).

8.1.3. Kubernetes ConfigMap

A Kubernetes ConfigMap is a mechanism that can provide configuration data to a deployed application. A ConfigMap object is typically defined in a YAML file, which is then uploaded to the Kubernetes cluster, making the configuration data available to deployed applications.

8.1.4. Kubernetes Secrets

A Kubernetes Secrets is a mechanism for providing sensitive data (such as passwords, certificates, and so on) to deployed applications.

8.1.5. Spring Cloud Kubernetes Plug-In

The Spring Cloud Kubernetes plug-in implements the integration between Kubernetes and Spring Boot. In principle, you could access the configuration data from a ConfigMap using the Kubernetes API. It is much more convenient, however, to integrate Kubernetes ConfigMap directly with the Spring Boot externalized configuration mechanism, so that Kubernetes ConfigMaps behave as an alternative property source for Spring Boot configuration. This is essentially what the Spring Cloud Kubernetes plug-in provides.

8.1.6. How to Enable Spring Boot with Kubernetes Integration

In a typical Spring Boot Maven project, you can enable the integration by adding the following Maven dependency to your project’s POM file:

<project ...>
  ...
  <dependencies>
    ...
    <dependency>
      <groupId>io.fabric8</groupId>
      <artifactId>spring-cloud-kubernetes-core</artifactId>
    </dependency>
    ...
  </dependencies>
  ...
</project>

To complete the integration, you need to add some annotations to your Java source code, create a Kubernetes ConfigMap object, and modify the OpenShift service account permissions to allow your application to read the ConfigMap object. These steps are described in detail in Section 8.2, “Tutorial for ConfigMap Property Source”.

8.2. Tutorial for ConfigMap Property Source

The following tutorial is based on the spring-boot-camel-config-archetype Maven archetype, which enables you to experiment with setting Kubernetes Secrets and ConfigMaps. The Spring Cloud Kubernetes plug-in is also enabled, making it possible to integrate Kubernetes configuration objects with Spring Boot Externalized Configuration.

8.2.1. Build and run the spring-boot-camel-config quickstart

Perform the following steps to create a simple Camel Spring Boot project:

  1. Open a new shell prompt and enter the following Maven command:

    mvn org.apache.maven.plugins:maven-archetype-plugin:2.4:generate \
      -DarchetypeCatalog=https://maven.repository.redhat.com/ga/io/fabric8/archetypes/archetypes-catalog/2.2.0.fuse-720018-redhat-00001/archetypes-catalog-2.2.0.fuse-720018-redhat-00001-archetype-catalog.xml \
      -DarchetypeGroupId=org.jboss.fuse.fis.archetypes \
      -DarchetypeArtifactId=spring-boot-camel-config-archetype \
      -DarchetypeVersion=2.2.0.fuse-720018-redhat-00001

    The archetype plug-in switches to interactive mode to prompt you for the remaining fields:

    Define value for property 'groupId': : org.example.fis
    Define value for property 'artifactId': : fuse72-configmap
    Define value for property 'version':  1.0-SNAPSHOT: :
    Define value for property 'package':  org.example.fis: :
    [INFO] Using property: spring-boot-version = 1.5.16.RELEASE
    Confirm properties configuration:
    groupId: org.example.fis
    artifactId: fuse72-configmap
    version: 1.0-SNAPSHOT
    package: org.example.fis
    spring-boot-version: 1.5.16.RELEASE
     Y: :

    When prompted, enter org.example.fis for the groupId value and fuse72-configmap for the artifactId value. Accept the defaults for the remaining fields.

  2. Log in to OpenShift and switch to the OpenShift project where you will deploy your application. For example, to log in as the developer user and deploy to the test project, enter the following commands:

    oc login -u developer -p developer
    oc project test
  3. At the command line, change to the directory of the new fuse72-configmap project and create the Secret object for this application:

    oc create -f sample-secret.yml
    Note

    It is necessary to create the Secret object before you deploy the application, otherwise the deployed container enters a wait state until the Secret becomes available. If you subsequently create the Secret, the container will come out of the wait state.

  4. Build and deploy the quickstart application. From the top level of the fuse72-configmap project, enter:

    mvn fabric8:deploy -Popenshift
  5. View the application log as follows. Open the OpenShift console in your browser and select the relevant project namespace (for example, test). Click in the center of the circular pod icon for the fuse72-configmap service and then — in the Pods view — click on the pod Name to view the details of the running pod (alternatively, you will get straight through to the details page, if there is only one pod running). Now click on the Logs tag to view the application log and scroll down to find the log messages generated by the Camel application.
  6. The default recipient list, which is configured in src/main/resources/application.properties, sends the generated messages to two dummy endpoints: direct:async-queue and direct:file. This causes messages like the following to be written to the application log:

    5:44:57.376 [Camel (camel) thread #0 - timer://order] INFO  generate-order-route - Generating message message-44, sending to the recipient list
    15:44:57.378 [Camel (camel) thread #0 - timer://order] INFO  target-route-queue - ----> message-44 pushed to an async queue (simulation)
    15:44:57.379 [Camel (camel) thread #0 - timer://order] INFO  target-route-queue - ----> Using username 'myuser' for the async queue
    15:44:57.380 [Camel (camel) thread #0 - timer://order] INFO  target-route--file - ----> message-44 written to a file
  7. Before you can update the configuration of the fuse72-configmap application using a ConfigMap object, you must give the fuse72-configmap application permission to view data from the OpenShift ApiServer. Enter the following command to give the view permission to the fuse72-configmap application’s service account:

    oc policy add-role-to-user view system:serviceaccount:test:qs-camel-config
    Note

    A service account is specified using the syntax system:serviceaccount:PROJECT_NAME:SERVICE_ACCOUNT_NAME. The fis-config deployment descriptor defines the SERVICE_ACCOUNT_NAME to be qs-camel-config.

  8. To see the live reload feature in action, create a ConfigMap object as follows:

    oc create -f sample-configmap.yml

    The new ConfigMap overrides the recipient list of the Camel route in the running application, configuring it to send the generated messages to three dummy endpoints: direct:async-queue, direct:file, and direct:mail. This causes messages like the following to be written to the application log:

    16:25:24.121 [Camel (camel) thread #0 - timer://order] INFO  generate-order-route - Generating message message-9, sending to the recipient list
    16:25:24.124 [Camel (camel) thread #0 - timer://order] INFO  target-route-queue - ----> message-9 pushed to an async queue (simulation)
    16:25:24.125 [Camel (camel) thread #0 - timer://order] INFO  target-route-queue - ----> Using username 'myuser' for the async queue
    16:25:24.125 [Camel (camel) thread #0 - timer://order] INFO  target-route--file - ----> message-9 written to a file (simulation)
    16:25:24.126 [Camel (camel) thread #0 - timer://order] INFO  target-route--mail - ----> message-9 sent via mail

8.2.2. Configuration Properties bean

A configuration properties bean is a regular Java bean that can receive configuration settings by injection. It provides the basic interface between your Java code and the external configuration mechanisms.

8.2.2.1. Overview

Externalized Configuration and Bean Registry shows how Spring Boot Externalized Configuration works in the spring-boot-camel-config quickstart.

Externalized Configuration and Bean Registry

kube spring boot 01

The configuration mechanism has the following main parts:

Property Sources
Provides property settings for injection into configuration. The default property source is the application’s application.properties file, and this can optionally be overridden by a ConfigMap object or a Secret object.
Configuration Properties bean
Receives configuraton updates from the property sources. A configuration properties bean is a Java bean decorated by the @Configuration and @ConfigurationProperties annotations.
Spring bean registry
With the requisite annotations, a configuration properties bean is registered in the Spring bean registry.
Integration with Camel bean registry
The Camel bean registry is automatically integrated with the Spring bean registry, so that registered Spring beans can be referenced in your Camel routes.

8.2.2.2. QuickstartConfiguration class

The configuration properties bean for the fuse72-configmap project is defined as the QuickstartConfiguration Java class (under the src/main/java/org/example/fis/ directory), as follows:

package org.example.fis;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

@Configuration  1
@ConfigurationProperties(prefix = "quickstart")  2
public class QuickstartConfiguration {

    /**
     * A comma-separated list of routes to use as recipients for messages.
     */
    private String recipients;  3

    /**
     * The username to use when connecting to the async queue (simulation)
     */
    private String queueUsername;  4

    /**
     * The password to use when connecting to the async queue (simulation)
     */
    private String queuePassword;  5

    // Setters and Getters for Bean properties
    // NOT SHOWN
    ...
}
1
The @Configuration annotation causes the QuickstartConfiguration class to be instantiated and registered in Spring as the bean with ID, quickstartConfiguration. This automatically makes the bean accessible from Camel. For example, the target-route-queue route is able to access the queueUserName property using the Camel syntax ${bean:quickstartConfiguration?method=getQueueUsername}.
2
The @ConfigurationProperties annotation defines a prefix, quickstart, that must be used when defining property values in a property source. For example, a properties file would reference the recipients property as quickstart.recipients.
3
The recipient property is injectable from property sources.
4
The queueUsername property is injectable from property sources.
5
The queuePassword property is injectable from property sources.

8.2.3. How to set up the Secret

The Kubernetes Secret in this quickstart is set up in the standard way, apart from one additional required step: the Spring Cloud Kubernetes plug-in must be configured with the mount paths of the Secrets, so that it can read the Secrets at run time.

For more details, see the chapter on Secrets in the Kubernetes reference documentation.

8.2.3.1. Sample Secret object

The quickstart project provides a sample Secret, sample-secret.yml, as follows:

apiVersion: v1
kind: Secret
metadata:
  name: camel-config
type: Opaque
data:
  # The username is 'myuser'
  quickstart.queue-username: bXl1c2VyCg==
  quickstart.queue-password: MWYyZDFlMmU2N2Rm

Note the following settings:

metadata.name
Identifies the Secret. Other parts of the OpenShift system use this identifier to reference the Secret.
quickstart.queue-username
Is meant to be injected into the queueUsername property of the quickstartConfiguration bean. The value must be base64 encoded.
quickstart.queue-password
Is meant to be injected into the queuePassword property of the quickstartConfiguration bean. The value must be base64 encoded.

Property values in Secret objects are always base64 encoded (use the base64 command-line utility). When the Secret is mounted in a pod’s filesystem, the values are automatically decoded back into plain text.

Note

Kubernetes does not allow you to define property names in CamelCase (it requires property names to be all lowercase). To work around this limitation, use the hyphenated form queue-username, which Spring Boot matches with queueUsername. This takes advantage of Spring Boot’s relaxed binding rules for externalized configuration.

8.2.3.2. Configure volume mount for the Secret

The application must be configured to load the Secret at run time, by configuring the Secret as a volume mount. After the application starts, the Secret properties then become available at the specified location in the filesystem.

The Example 8.1, “deployment.yml file” listing shows the application’s deployment.yml file (located under src/main/fabric8/), which defines the volume mount for the Secret.

Example 8.1. deployment.yml file

spec:
  template:
    spec:
      serviceAccountName: "qs-camel-config"
      volumes: 1
        - name: "camel-config"
          secret:
            # The secret must be created before deploying this application
            secretName: "camel-config"
      containers:
        -
          volumeMounts: 2
            - name: "camel-config"
              readOnly: true
              # Mount the secret where spring-cloud-kubernetes is configured to read it
              # see src/main/resources/bootstrap.yml
              mountPath: "/etc/secrets/camel-config"
          resources:
#            requests:
#              cpu: "0.2"
#              memory: 256Mi
#            limits:
#              cpu: "1.0"
#              memory: 256Mi
             env:
              - name: SPRING_APPLICATION_JSON
               value: '{"server":{"undertow":{"io-threads":1, "worker-threads":2 }}}'
1
In the volumes section, the deployment declares a new volume named camel-config, which references the Secret named camel-config.
2
In the volumeMounts section, the deployment declares a new volume mount, which references the camel-config volume and specifies that the Secret volume should be mounted to the path /etc/secrets/camel-config in the pod’s filesystem.

8.2.3.3. Configure spring-cloud-kubernetes to read Secret properties

To integrate secrets with Spring Boot externalized configuration, the Spring Cloud Kubernetes plug-in must be configured with the secret’s mount path. Spring Cloud Kubernetes reads the secrets from the specified location and makes them available to Spring Boot as property sources.

The Spring Cloud Kubernetes plug-in is configured by settings in the bootstrap.yml file, located under src/main/resources in the quickstart project, as shown in the Example 8.2, “bootstrap.yml file” listing.

Example 8.2. bootstrap.yml file

# Startup configuration of Spring-cloud-kubernetes
spring:
  application:
    name: camel-config
  cloud:
    kubernetes:
      reload:
        # Enable live reload on ConfigMap change (disabled for Secrets by default)
        enabled: true
      secrets:
        paths: /etc/secrets/camel-config

The spring.cloud.kubernetes.secrets.paths property specifies the list of paths of secrets volume mounts in the pod.

Note

A bootstrap.properties file (or bootstrap.yml file) behaves similarly to an application.properties file, but it is loaded at an earlier phase of application start-up. It is more reliable to set the properties relating to the Spring Cloud Kubernetes plug-in in the bootstrap.properties file.

8.2.4. How to set up the ConfigMap

In addition to creating a ConfigMap object and setting the view permission appropriately, the integration with Spring Cloud Kubernetes requires you to match the ConfigMap’s metadata.name with the value of the spring.application.name property configured in the project’s bootstrap.yml file.

8.2.4.1. Sample ConfigMap object

The quickstart project provides a sample ConfigMap, sample-configmap.yml, as follows:

kind: ConfigMap
apiVersion: v1
metadata:
  # Must match the 'spring.application.name' property of the application
  name: camel-config
data:
  application.properties: |
    # Override the configuration properties here
    quickstart.recipients=direct:async-queue,direct:file,direct:mail

Note the following settings:

metadata.name
Identifies the ConfigMap. Other parts of the OpenShift system use this identifier to reference the ConfigMap.
data.application.properties
This section lists property settings that can override settings from the original application.properties file that was deployed with the application.
quickstart.recipients
Is meant to be injected into the recipients property of the quickstartConfiguration bean.

For more details about the format of this file, see Section 8.3, “ConfigMap PropertySource”.

8.2.4.2. Setting the view permission

As shown in the Example 8.1, “deployment.yml file” listing, the serviceAccountName is set to qs-camel-config in the project’s deployment.yml file. Hence, you need to enter the following command to enable the view permission on the quickstart application (assuming that it deploys into the test project namespace):

oc policy add-role-to-user view system:serviceaccount:test:qs-camel-config

8.2.4.3. Configuring the Spring Cloud Kubernetes plug-in

The Spring Cloud Kubernetes plug-in is configured by the following settings in the bootstrap.yml file, as shown in the Example 8.2, “bootstrap.yml file” listing:

spring.application.name
This value must match the metadata.name of the ConfigMap object (for example, as defined in sample-configmap.yml in the quickstart project). It defaults to application.
spring.cloud.kubernetes.reload.enabled
Setting this to true enables dynamic reloading of ConfigMap objects.

For more details about the supported properties, see Section 8.5, “PropertySource Reload”.

8.3. ConfigMap PropertySource

Kubernetes has the notion of ConfigMap for passing configuration to the application. The Spring cloud Kubernetes plug-in provides integration with ConfigMap to make config maps accessible by Spring Boot.

The ConfigMap PropertySource when enabled will look up Kubernetes for a ConfigMap named after the application (see spring.application.name). If the map is found it will read its data and do the following:

8.3.1. Apply Individual Properties

Let’s assume that we have a Spring Boot application named demo that uses properties to read its thread pool configuration.

  • pool.size.core
  • pool.size.max

This can be externalized to config map in YAML format:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  pool.size.core: 1
  pool.size.max: 16

8.3.2. Apply Property Named application.yaml

Individual properties work fine for most cases but sometimes we find YAML is more convenient. In this case we use a single property named application.yaml and embed our YAML inside it:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  application.yaml: |-
    pool:
      size:
        core: 1
        max:16

8.3.3. Apply Property Named application.properties

You can also define the ConfigMap properties in the style of a Spring Boot application.properties file. In this case we use a single property named application.properties and list the property settings inside it:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  application.properties: |-
    pool.size.core: 1
    pool.size.max: 16

8.3.4. Deploying a ConfigMap

To deploy a ConfigMap and make it accessible to a Spring Boot application, perform the following steps:

  1. In your Spring Boot application, use the externalized configuration mechanism to access the ConfigMap property source. For example, by annotating a Java bean with the @Configuration annotation, it becomes possible for the bean’s property values to be injected by a ConfigMap.
  2. In your project’s bootstrap.properties file (or bootstrap.yaml file), set the spring.application.name property to match the name of the ConfigMap.
  3. Enable the view permission on the service account that is associated with your application (by default, this would be the service account called default). For example, to add the view permission to the default service account:

    oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)

8.4. Secrets PropertySource

Kubernetes has the notion of Secrets for storing sensitive data such as password, OAuth tokens, etc. The Spring cloud Kubernetes plug-in provides integration with Secrets to make secrets accessible by Spring Boot.

The Secrets property source when enabled will look up Kubernetes for Secrets from the following sources:

  1. Reading recursively from secrets mounts
  2. Named after the application (see spring.application.name)
  3. Matching some labels

Please note that, by default, consuming Secrets via API (points 2 and 3 above) is not enabled.

If the secrets are found, their data is made available to the application.

8.4.1. Example of Setting Secrets

Let’s assume that we have a Spring Boot application named demo that uses properties to read its ActiveMQ and PostreSQL configuration.

amq.username
amq.password
pg.username
pg.password

These secrets can be externalized to Secrets in YAML format:

ActiveMQ Secrets
apiVersion: v1
kind: Secret
metadata:
  name: activemq-secrets
  labels:
    broker: activemq
type: Opaque
data:
  amq.username: bXl1c2VyCg==
  amq.password: MWYyZDFlMmU2N2Rm
PostreSQL Secrets
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secrets
  labels:
    db: postgres
type: Opaque
data:
  pg.username: dXNlcgo=
  pg.password: cGdhZG1pbgo=

8.4.2. Consuming the Secrets

You can select the Secrets to consume in a number of ways:

  1. By listing the directories where the secrets are mapped:

    -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets/activemq,etc/secrets/postgres

    If you have all the secrets mapped to a common root, you can set them like this:

    -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets
  2. By setting a named secret:

    -Dspring.cloud.kubernetes.secrets.name=postgres-secrets
  3. By defining a list of labels:

    -Dspring.cloud.kubernetes.secrets.labels.broker=activemq
    -Dspring.cloud.kubernetes.secrets.labels.db=postgres

8.4.3. Secrets Configuration Properties

You can use the following properties to configure the Secrets property source:

spring.cloud.kubernetes.secrets.enabled
Enable the Secrets property source. Type is Boolean and default is true.
spring.cloud.kubernetes.secrets.name
Sets the name of the secret to look up. Type is String and default is ${spring.application.name}.
spring.cloud.kubernetes.secrets.labels
Sets the labels used to lookup secrets. This property behaves as defined by Map-based binding. Type is java.util.Map and default is null.
spring.cloud.kubernetes.secrets.paths
Sets the paths where secrets are mounted. This property behaves as defined by Collection-based binding. Type is java.util.List and default is null.
spring.cloud.kubernetes.secrets.enableApi
Enable/disable consuming secrets via APIs. Type is Boolean and default is false.
Note

Access to secrets via API may be restricted for security reasons — the preferred way is to mount a secret to the POD.

8.5. PropertySource Reload

Some applications may need to detect changes on external property sources and update their internal status to reflect the new configuration. The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related ConfigMap or Secret change.

This feature is disabled by default and can be enabled using the configuration property spring.cloud.kubernetes.reload.enabled=true (for example, in the bootstrap.properties file).

The following levels of reload are supported (property spring.cloud.kubernetes.reload.strategy):

refresh

(default) only configuration beans annotated with @ConfigurationProperties or @RefreshScope are reloaded. This reload level leverages the refresh feature of Spring Cloud Context.

Note

The PropertySource reload feature can only be used for simple properties (that is, not collections) when the reload strategy is set to refresh. Properties backed by collections must not be changed at runtime.

restart_context
the whole Spring ApplicationContext is gracefully restarted. Beans are recreated with the new configuration.
shutdown
the Spring ApplicationContext is shut down to activate a restart of the container. When using this level, make sure that the lifecycle of all non-daemon threads is bound to the ApplicationContext and that a replication controller or replica set is configured to restart the pod.

8.5.1. Example

Assuming that the reload feature is enabled with default settings (refresh mode), the following bean will be refreshed when the config map changes:

@Configuration
@ConfigurationProperties(prefix = "bean")
public class MyConfig {

    private String message = "a message that can be changed live";

    // getter and setters

}

A way to see that changes effectively happen is creating another bean that prints the message periodically.

@Component
public class MyBean {

    @Autowired
    private MyConfig config;

    @Scheduled(fixedDelay = 5000)
    public void hello() {
        System.out.println("The message is: " + config.getMessage());
    }
}

The message printed by the application can be changed using a ConfigMap like the following one:

apiVersion: v1
kind: ConfigMap
metadata:
  name: reload-example
data:
  application.properties: |-
    bean.message=Hello World!

Any change to the property named bean.message in the Config Map associated with the pod will be reflected in the output of the program.

The full example is available in [spring-cloud-kubernetes-reload-example](spring-cloud-kubernetes-examples/spring-cloud-kubernetes-reload-example).

The reload feature supports two operating modes:

event
(default) watches for changes in ConfigMaps or secrets using the Kubernetes API (web socket). Any event will produce a re-check on the configuration and a reload in case of changes. The view role on the service account is required in order to listen for config map changes. A higher level role (eg. edit) is required for secrets (secrets are not monitored by default).
polling
re-creates the configuration periodically from config maps and secrets to see if it has changed. The polling period can be configured using the property spring.cloud.kubernetes.reload.period and defaults to 15 seconds. It requires the same role as the monitored property source. This means, for example, that using polling on file mounted secret sources does not require particular privileges.

The following properties can be used to configure the reloading feature:

spring.cloud.kubernetes.reload.enabled
Enables monitoring of property sources and configuration reload. Type is Boolean and default is false.
spring.cloud.kubernetes.reload.monitoring-config-maps
Allow monitoring changes in config maps. Type is Boolean and default is true.
spring.cloud.kubernetes.reload.monitoring-secrets
Allow monitoring changes in secrets. Type is Boolean and default is false.
spring.cloud.kubernetes.reload.strategy
The strategy to use when firing a reload (refresh, restart_context, shutdown). Type is Enum and default is refresh.
spring.cloud.kubernetes.reload.mode
Specifies how to listen for changes in property sources (event, polling). Type is Enum and default is event.
spring.cloud.kubernetes.reload.period
The period in milliseconds for verifying changes when using the polling strategy. Type is Long and default is 15000.

Note the following points:

  • The spring.cloud.kubernetes.reload.* properties should not be used in ConfigMaps or Secrets. Changing such properties at run time may lead to unexpected results;
  • Deleting a property or the whole config map does not restore the original state of the beans when using the refresh level.