Values set by "podman update" are not persistent across host/container reboots

Solution In Progress - Updated -

Environment

  • Red Hat Enterprise Linux 8
  • Red Hat Enterprise Linux 9
  • podman

Issue

  • When an already running podman container is made some changes(like memory for example), the changes are reverted back once the container is restarted

Resolution

  • A JIRA has been created to track this so that the changes are persistent even when the container is restarted

Root Cause

  • This is deliberately done as a result of database limitations in Podman 4.x and below. Podman was built around a model of immutable containers which means no changes are to be done once the container is created
  • When podman inspect is run after performing podman update, it would still show the old values since the database hasn't been modified, any container restart will wipe the changes as well (this includes natural stopping and starting - it won't be limited to a full restart of the host)
  • This can not be changed for old boltDdb database due to various known limitations
  • With the new SQLite database driver that will probably be added in the future, it will be possible to fix this properly

Diagnostic Steps

  • Check for the stats of a running container
# podman run -itd --name memory-test ubi8
8650d01e83094622169599f4bc639f231c01c582d4228cf14dfa73bdedc02954

# podman stats memory-test
ID            NAME         CPU %       MEM USAGE / LIMIT  MEM %       NET IO          BLOCK IO    PIDS        CPU TIME    AVG CPU %
8650d01e8309  memory-test  1.12%       589.8kB / 3.915GB  0.02%       1.126kB / 542B  0B / 0B     1           94.229ms    1.12%
  • Modify the memory(or any desired parameter) and check the updated stats
# podman update --memory 1G memory-test 
8650d01e83094622169599f4bc639f231c01c582d4228cf14dfa73bdedc02954

# podman stats memory-test
ID            NAME         CPU %       MEM USAGE / LIMIT  MEM %       NET IO          BLOCK IO    PIDS        CPU TIME    AVG CPU %
8650d01e8309  memory-test  0.06%       589.8kB / 1.074GB  0.05%       1.406kB / 822B  0B / 0B     1           94.229ms    0.06%
  • Restart the container
# podman restart memory-test
  • Now, check the stats for the container
# podman stats memory-test
ID            NAME         CPU %       MEM USAGE / LIMIT  MEM %       NET IO          BLOCK IO      PIDS        CPU TIME    AVG CPU %
8650d01e8309  memory-test  5.75%       659.5kB / 3.915GB  0.02%       1.412kB / 472B  31.51MB / 0B  1           105.968ms   5.75%
  • We can see that the values got changed once the container is restarted

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments