Chapter 10. Server Cache Configuration

Red Hat Single Sign-On has two types of caches. One type of cache sits in front of the database to decrease load on the DB and to decrease overall response times by keeping data in memory. Realm, client, role, and user metadata is kept in this type of cache. This cache is a local cache. Local caches do not use replication even if you are in the cluster with more Red Hat Single Sign-On servers. Instead, they only keep copies locally and if the entry is updated an invalidation message is sent to the rest of the cluster and the entry is evicted. There is separate replicated cache work, which task is to send the invalidation messages to the whole cluster about what entries should be evicted from local caches. This greatly reduces network traffic, makes things efficient, and avoids transmitting sensitive metadata over the wire.

The second type of cache handles managing user sessions, offline tokens, and keeping track of login failures so that the server can detect password phishing and other attacks. The data held in these caches is temporary, in memory only, but is possibly replicated across the cluster.

This chapter discusses some configuration options for these caches for both clustered and non-clustered deployments.

Note

More advanced configuration of these caches can be found in the Infinispan section of the JBoss EAP Configuration Guide.

10.1. Eviction and Expiration

There are multiple different caches configured for Red Hat Single Sign-On. There is a realm cache that holds information about secured applications, general security data, and configuration options. There is also a user cache that contains user metadata. Both caches default to a maximum of 10000 entries and use a least recently used eviction strategy. Each of them is also tied to an object revisions cache that controls eviction in a clustered setup. This cache is created implicitly and has twice the configured size. The same applies for the authorization cache, which holds the authorization data. The keys cache holds data about external keys and does not need to have dedicated revisions cache. Rather it has expiration explicitly declared on it, so the keys are periodically expired and forced to be periodically downloaded from external clients or identity providers.

The eviction policy and max entries for these caches can be configured in the standalone.xml, standalone-ha.xml, or domain.xml depending on your operating mode. In the configuration file, there is the part with infinispan subsystem, which looks similar to this:

<subsystem xmlns="urn:jboss:domain:infinispan:9.0">
    <cache-container name="keycloak">
        <local-cache name="realms">
            <object-memory size="10000"/>
        </local-cache>
        <local-cache name="users">
            <object-memory size="10000"/>
        </local-cache>
        ...
        <local-cache name="keys">
            <object-memory size="1000"/>
            <expiration max-idle="3600000"/>
        </local-cache>
        ...
    </cache-container>

To limit or expand the number of allowed entries simply add or edit the object element or the expiration element of particular cache configuration.

In addition, there are also separate caches sessions, clientSessions, offlineSessions, offlineClientSessions, loginFailures and actionTokens. These caches are distributed in cluster environment and they are unbounded in size by default. If they are bounded, it would then be possible that some sessions will be lost. Expired sessions are cleared internally by Red Hat Single Sign-On itself to avoid growing the size of these caches without limit. If you see memory issues due to a large number of sessions, you can try to:

  • Increase the size of cluster (more nodes in cluster means that sessions are spread more equally among nodes)
  • Increase the memory for Red Hat Single Sign-On server process
  • Decrease the number of owners to ensure that caches are saved in one single place. See Section 10.2, “Replication and Failover” for more details
  • Disable l1-lifespan for distributed caches. See Infinispan documentation for more details
  • Decrease session timeouts, which could be done individually for each realm in Red Hat Single Sign-On admin console. But this could affect usability for end users. See Timeouts for more details.

There is an additional replicated cache, work, which is mostly used to send messages among cluster nodes; it is also unbounded by default. However, this cache should not cause any memory issues as entries in this cache are very short-lived.

10.2. Replication and Failover

There are caches like sessions, authenticationSessions, offlineSessions, loginFailures and a few others (See Section 10.1, “Eviction and Expiration” for more details), which are configured as distributed caches when using a clustered setup. Entries are not replicated to every single node, but instead one or more nodes is chosen as an owner of that data. If a node is not the owner of a specific cache entry it queries the cluster to obtain it. What this means for failover is that if all the nodes that own a piece of data go down, that data is lost forever. By default, Red Hat Single Sign-On only specifies one owner for data. So if that one node goes down that data is lost. This usually means that users will be logged out and will have to login again.

You can change the number of nodes that replicate a piece of data by change the owners attribute in the distributed-cache declaration.

owners

<subsystem xmlns="urn:jboss:domain:infinispan:9.0">
   <cache-container name="keycloak">
       <distributed-cache name="sessions" owners="2"/>
...

Here we’ve changed it so at least two nodes will replicate one specific user login session.

Tip

The number of owners recommended is really dependent on your deployment. If you do not care if users are logged out when a node goes down, then one owner is good enough and you will avoid replication.

Tip

It is generally wise to configure your environment to use loadbalancer with sticky sessions. It is beneficial for performance as Red Hat Single Sign-On server, where the particular request is served, will be usually the owner of the data from the distributed cache and will therefore be able to look up the data locally. See Section 9.4, “Sticky sessions” for more details.

10.3. Disabling Caching

To disable the realm or user cache, you must edit the standalone.xml, standalone-ha.xml, or domain.xml file in your distribution. The location of this file depends on your operating mode. Here’s what the config looks like initially.

    <spi name="userCache">
        <provider name="default" enabled="true"/>
    </spi>

    <spi name="realmCache">
        <provider name="default" enabled="true"/>
    </spi>

To disable the cache set the enabled attribute to false for the cache you want to disable. You must reboot your server for this change to take effect.

10.4. Clearing Caches at Runtime

To clear the realm or user cache, go to the Red Hat Single Sign-On admin console Realm Settings→Cache Config page. On this page you can clear the realm cache, the user cache or cache of external public keys.

Note

The cache will be cleared for all realms!