Red Hat Training

A Red Hat training course is available for RHEL 8

Chapter 22. Creating and restoring container checkpoints

Checkpoint/Restore In Userspace (CRIU) is a software that enables you to set a checkpoint on a running container or an individual application and store its state to disk. You can use data saved to restore the container after a reboot at the same point in time it was checkpointed.

Warning

The kernel does not support pre-copy checkpointing on AArch64.

22.1. Creating and restoring a container checkpoint locally

This example is based on a Python based web server which returns a single integer which is incremented after each request.

Prerequisites

  • The container-tools module is installed.

Procedure

  1. Create a Python based server: 

    # cat counter.py
#!/usr/bin/python3

import http.server

counter = 0

class handler(http.server.BaseHTTPRequestHandler):
    def do_GET(s):
        global counter
        s.send_response(200)
        s.send_header('Content-type', 'text/html')
        s.end_headers()
        s.wfile.write(b'%d\n' % counter)
        counter += 1


server = http.server.HTTPServer(('', 8088), handler)
server.serve_forever()

  2. Create a container with the following definition: 

    # cat Containerfile
FROM registry.access.redhat.com/ubi8/ubi

COPY counter.py /home/counter.py

RUN useradd -ms /bin/bash counter

RUN yum -y install python3 && chmod 755 /home/counter.py

USER counter
ENTRYPOINT /home/counter.py

    The container is based on the Universal Base Image (UBI 8) and uses a Python based server.

  3. Build the container: 

    # podman build . --tag counter

    Files counter.py and Containerfile are the input for the container build process (podman build). The built image is stored locally and tagged with the tag counter.

  4. Start the container as root: 

    # podman run --name criu-test --detach counter

  5. To list all running containers, enter: 

    # podman ps
CONTAINER ID  IMAGE  COMMAND  CREATED   STATUS  PORTS NAMES
e4f82fd84d48  localhost/counter:latest  5 seconds ago  Up 4 seconds ago  criu-test

  6. Display IP address of the container: 

    # podman inspect criu-test --format "{{.NetworkSettings.IPAddress}}"
10.88.0.247

  7. Send requests to the container: 

    # curl 10.88.0.247:8088
0
# curl 10.88.0.247:8088
1

  8. Create a checkpoint for the container: 

    # podman container checkpoint criu-test
  9. Reboot the system.

  10. Restore the container: 

    # podman container restore --keep criu-test

  11. Send requests to the container: 

    # curl 10.88.0.247:8080
2
# curl 10.88.0.247:8080
3
# curl 10.88.0.247:8080
4

    The result now does not start at 0 again, but continues at the previous value.

This way you can easily save the complete container state through a reboot.

Additional resources

22.2. Reducing startup time using container restore

You can use container migration to reduce startup time of containers which require a certain time to initialize. Using a checkpoint, you can restore the container multiple times on the same host or on different hosts. This example is based on the container from the Creating and restoring a container checkpoint locally.

Prerequisites

  • The container-tools module is installed.

Procedure

  1. Create a checkpoint of the container, and export the checkpoint image to a tar.gz file: 

    # podman container checkpoint criu-test --export /tmp/chkpt.tar.gz

  2. Restore the container from the tar.gz file: 

    # podman container restore --import /tmp/chkpt.tar.gz --name counter1
# podman container restore --import /tmp/chkpt.tar.gz --name counter2
# podman container restore --import /tmp/chkpt.tar.gz --name counter3

    The --name (-n) option specifies a new name for containers restored from the exported checkpoint.

  3. Display ID and name of each container: 

    # podman ps -a --format "{{.ID}} {{.Names}}"
a8b2e50d463c counter3
faabc5c27362 counter2
2ce648af11e5 counter1

  4. Display IP address of each container: 

    #️ podman inspect counter1 --format "{{.NetworkSettings.IPAddress}}"
10.88.0.248

#️ podman inspect counter2 --format "{{.NetworkSettings.IPAddress}}"
10.88.0.249

#️ podman inspect counter3 --format "{{.NetworkSettings.IPAddress}}"
10.88.0.250

  5. Send requests to each container: 

    #️ curl 10.88.0.248:8080
4
#️ curl 10.88.0.249:8080
4
#️ curl 10.88.0.250:8080
4

    Note, that the result is 4 in all cases, because you are working with different containers restored from the same checkpoint.

Using this approach, you can quickly start up stateful replicas of the initially checkpointed container.

Additional resources

22.3. Migrating containers among systems

You can migrate the running containers from one system to another, without losing the state of the applications running in the container. This example is based on the container from the Creating and restoring a container checkpoint locally section tagged with counter.

Important

Migrating containers among systems with the podman container checkpoint and podman container restore commands is supported only when the configurations of the systems match completely, as shown below:

  • Podman version
  • OCI runtime (runc/crun)
  • Network stack (CNI/Netavark)
  • Cgroups version
  • kernel version
  • CPU features

You can migrate to a CPU with more features, but not to a CPU which does not have a certain feature that you are using. The low-level tool doing the checkpointing (CRIU) has the possibility to check for CPU feature compatibility: https://criu.org/Cpuinfo.

Prerequisites

  • The container-tools module is installed.

  • The following steps are not necessary if the container is pushed to a registry as Podman will automatically download the container from a registry if it is not available locally. This example does not use a registry, you have to export previously built and tagged container (see Creating and restoring a container checkpoint locally).

    • Export previously built container: 

      # podman save --output counter.tar counter

    • Copy exported container image to the destination system (other_host): 

      # scp counter.tar other_host:

    • Import exported container on the destination system: 

      # ssh other_host podman load --input counter.tar

      Now the destination system of this container migration has the same container image stored in its local container storage.

Procedure

  1. Start the container as root: 

    # podman run --name criu-test --detach counter

  2. Display IP address of the container: 

    # podman inspect criu-test --format "{{.NetworkSettings.IPAddress}}"
10.88.0.247

  3. Send requests to the container: 

    # curl 10.88.0.247:8080
0
# curl 10.88.0.247:8080
1

  4. Create a checkpoint of the container, and export the checkpoint image to a tar.gz file: 

    # podman container checkpoint criu-test --export /tmp/chkpt.tar.gz

  5. Copy the checkpoint archive to the destination host: 

    # scp /tmp/chkpt.tar.gz other_host:/tmp/

  6. Restore the checkpoint on the destination host (other_host): 

    # podman container restore --import /tmp/chkpt.tar.gz

  7. Send a request to the container on the destination host (other_host): 

    # *curl 10.88.0.247:8080*
2

As a result, the stateful container has been migrated from one system to another without losing its state.

Additional resources