10.10. Rollout Plan

10.10.1. Rollout Plans

Operations targeted at domain or host level resources can potentially impact multiple servers. Such operations can include a roll out plan detailing the sequence in which the operation would be applied to the servers, as well as the policies for detailing whether the operation could be reverted if it fails to execute successfully on some servers.

Example 10.17. CLI format of a rollout plan

rollout (id=plan_id | server_group_list) [rollback-across-groups]

server_group_list := server_group [ (sequence_separator | concurrent_separator) server_group ]
sequence_separator := ','
concurrent_separator := '^'
server_group := server_group_name [group_policy_list]
group_policy_list := '(' policy_property_name=policy_property_value (, policy_property_name=policy_property_value)* ')'
policy_property_name := 'rolling-to-servers' | 'max-failed-servers' | 'max-failure-percentage'
The value of policy_property_value depends on the property. It can be a boolean, an integer, etc.
Rollout plans can potentially be long and complex. There is a possibility, though, to store them as a part of the domain management model and then later be referenced from commands and operations using their name (or ID in the definition above). Stored rollout plans are managed using the rollout-plan command.

Example 10.18. Rollout plan managed with the rollout-plan command

rollout-plan add --name=my-plan --content={rollout main-server-group^other-server-group}
:write-attribute(name=my-attr,value=my-value){rollout id=my-plan}

Example 10.19. Using a stored rollout plan

rollout-plan add --name=my-plan --content={rollout main-server-group^other-server-group}
:write-attribute(name=my-attr,value=my-value){rollout id=my-plan}

10.10.2. Operations with a Rollout Plan

The structure is of rollout-plan within an operation is as follows:
{
"operation" => "write-core-threads",
    "address" => [
        ("profile" => "production"),
        ("subsystem" => "threads"),
        ("bounded-queue-thread-pool" => "pool1")
    ],
    "count" => 0,
    "per-cpu" => 20,
    "operation-headers" => {
        "rollout-plan" => {
            "in-series" => [
                {
                    "concurrent-groups" => {
                        "groupA" => {
                            "rolling-to-servers" => true,
                            "max-failure-percentage" => 20
                        },
                        "groupB" => undefined
                    }
                },
                {
                   "server-group" => {
                        "groupC" => {
                            "rolling-to-servers" => false,
                            "max-failed-servers" => 1
                        }
                    }
                },
                {
                    "concurrent-groups" => {
                        "groupD" => {
                            "rolling-to-servers" => true,
                            "max-failure-percentage" => 20
                        },
                        "groupE" => undefined
                    }
                }
            ],
            "rollback-across-groups" => true
        }
    }
}
The rollout-plan is nested within the operation-headers structure. The root node of the structure allows two children:
  • in-series - A list of steps that are to be performed in series, with each step reaching completion before the next step is executed. Each step involves the application of the operation to the servers in one or more server groups. See below for details on each element in the list.
  • rollback-across-groups - A boolean that indicates whether the need to rollback the operation on all the servers in one server group triggers a rollback across all the server groups. This is an optional setting, and defaults to false.
Each element in the list under the in-series node must have one or the other of the following structures:
  • concurrent-groups - A map of server group names to policies controlling how the operation should be applied to that server group. For each server group in the map, the operation may be applied concurrently. See below for details on the per-server-group policy configuration.
  • server-group - A single key/value mapping of a server group name to a policy controlling how the operation should be applied to that server group. See below for details on the policy configuration. (Note: there is no difference in plan execution between this and a "concurrent-groups" map with a single entry.)
The policy controlling how the operation is applied to the servers within a server group has the following elements, each of which is optional:
  • rolling-to-servers - A boolean which if set to true, the operation will be applied to each server in the group in series. If false or not specified, the operation will be applied to the servers in the group concurrently.
  • max-failed-servers - An integer which takes the maximum number of servers in the group that can fail to apply the operation before it should be reverted on all servers in the group. The default value if not specified is zero; i.e. failure on any server triggers rollback across the group.
  • max-failure-percentage - An integer between 0 and 100 which takes the maximum percentage of the total number of servers in the group that can fail to apply the operation before it should be reverted on all servers in the group. The default value if not specified is zero; i.e. failure on any server triggers rollback across the group.
If both max-failed-servers and max-failure-percentage are set to non-zero values, max-failure-percentage takes precedence.
Looking at the (contrived) example above, application of the operation to the servers in the domain would be done in 3 phases. If the policy for any server group triggers a rollback of the operation across the server group, all other server groups will be rolled back as well. The 3 phases are:
  1. Server groups groupA and groupB will have the operation applied concurrently. The operation will be applied to the servers in groupA in series, while all servers in groupB will handle the operation concurrently. If more than 20% of the servers in groupA fail to apply the operation, it will be rolled back across that group. If any servers in groupB fail to apply the operation it will be rolled back across that group.
  2. Once all servers in groupA and groupB are complete, the operation will be applied to the servers in groupC. Those servers will handle the operation concurrently. If more than one server in groupC fails to apply the operation it will be rolled back across that group.
  3. Once all servers in groupC are complete, server groups groupD and groupE will have the operation applied concurrently. The operation will be applied to the servers in groupD in series, while all servers in groupE will handle the operation concurrently. If more than 20% of the servers in groupD fail to apply the operation, it will be rolled back across that group. If any servers in groupE fail to apply the operation it will be rolled back across that group.
Default Rollout Plan

All operations that impact multiple servers will be executed with a rollout plan. However, actually specifying the rollout plan in the operation request is not required. If no rollout-plan is specified, a default plan will be generated. The plan will have the following characteristics:

  • There will only be a single high level phase. All server groups affected by the operation will have the operation applied concurrently.
  • Within each server group, the operation will be applied to all servers concurrently.
  • Failure on any server in a server group will cause rollback across the group.
  • Failure of any server group will result in rollback of all other server groups.

10.10.3. Creating a Rollout Deployment Plan

How to create a roll out deployment plan to deploy applications in a clustered domain in JBoss EAP 6
  1. Create a rollout deployment plan using CLI with rolling-to-servers=true. The package will be deployed to each server in the server group in a serial manner.
    An example CLI deployment plan for serial deployment is provided below:
    deploy ClusterWebApp.war --name=ClusterWebApp.war --runtime-name=ClusterWebApp.war --server-groups=ha-server-group --headers={rollout ha-server-group(rolling-to-servers=true)}
    
  2. To apply a rollout plan to all the server-groups, you need to mention the names of each server-group in master host:
    deploy /NotBackedUp/PREVIOUS/ALLWAR/ClusterWebApp.war --name=ClusterWebApp.war --runtime-name=ClusterWebApp.war --server-groups=main-server-group,other-server-group --headers={rollout main-server-group(rolling-to-servers=true),other-server-group(rolling-to-servers=true)}