octavia-undercloud fails with 503 Service Unavailable: Insufficient permissions on image storage media: Permission to write image storage media denied in Red Hat OpenStack Platform 13

Issue

During openstack overcloud update converge octavia-undercloud fails with 503 Service Unavailable: Insufficient permissions on image storage media: Permission to write image storage media denied.

/var/log/mistral/executor.log:

(...)
openstack image list --property owner=b8467eb710b2478bb764d4e78c3aaf8a --private --name octavia-amphora-13.0-20190827.1.x86_64 -c ID -f value
(...)
TASK [octavia-undercloud : upload image to glance] *****************************\ntask path: /usr/share/openstack-tripleo-common/playbooks/roles/octavia-undercloud/tasks/image_mgmt.yml:108\nMonday 26 August 2019  22:52:18 +0200 (0:00:00.059)       0:00:17.274 ********* \nUsing module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py\n<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mistral\n<localhost> EXEC /bin/sh -c \'OS_PROJECT_NAME=service COMPUTE_API_VERSION=1.1 OS_USERNAME=octavia OS_IDENTITY_API_VERSION=3 OS_USER_DOMAIN_NAME=Default OS_CLOUDNAME=overcloud OS_AUTH_URL=https://example.net:13000/v3 NOVA_VERSION=1.1 OS_IMAGE_API_VERSION=2 OS_PASSWORD=eEGGJqMdMQEwRq8eWCypVbWXz OS_PROJECT_DOMAIN_NAME=Default OS_AUTH_TYPE=password /usr/bin/python && sleep 0\'\nfatal: [localhost]: FAILED! => {\n    "changed": false, \n    "cmd": "openstack image create --disk-format qcow2 --container-format bare --tag amphora-image --file /usr/share/rhosp-director-images/octavia-amphora.qcow2 --property hw_architecture=x86_64 --private octavia-amphora-13.0-20190806.1.x86_64", \n    "delta": "0:00:06.666144", \n    "end": "2019-08-26 22:52:24.917753", \n    "invocation": {\n        "module_args": {\n            "_raw_params": "openstack image create --disk-format qcow2 --container-format bare --tag amphora-image --file /usr/share/rhosp-director-images/octavia-amphora.qcow2 --property hw_architecture=x86_64 --private octavia-amphora-13.0-20190806.1.x86_64", \n            "_uses_shell": true, \n            "argv": null, \n            "chdir": null, \n            "creates": null, \n            "executable": null, \n            "removes": null, \n            "stdin": null, \n            "warn": true\n        }\n    }, \n    "msg": "non-zero return code", \n    "rc": 1, \n    "start": "2019-08-26 22:52:18.251609", \n    "stderr": "503 Service Unavailable: Insufficient permissions on image storage media: Permission to write image storage media denied. (HTTP 503)", \n    "stderr_lines": [\n        "503 Service Unavailable: Insufficient permissions on image storage media: Permission to write image storage media denied. (HTTP 503)"\n    ], \n    "stdout": "", \n    "stdout_lines": []\n}\n\nPLAY RECAP *********************************************************************\nlocalhost                  : ok=11   changed=3    unreachable=0    failed=1   \n\nMonday 26 August 2019  22:52:24 +0200 (0:00:06.829)       0:00:24.104 ********* \n=============================================================================== \n'
2019-08-26 22:52:24.992 3230 ERROR mistral.executors.default_executor Stderr: u' [WARNING]: raw module does not support the environment keyword\n[DEPRECATION WARNING]: get_md5 has been deprecated along with the md5 return \nvalue, use get_checksum=True and checksum_algorithm=md5 instead. This feature \nwill be removed in version 2.9. Deprecation warnings can be disabled by setting\n deprecation_warnings=False in ansible.cfg.\n'

In pretty print:

TASK [octavia-undercloud : upload image to glance] *****************************
task path: /usr/share/openstack-tripleo-common/playbooks/roles/octavia-undercloud/tasks/image_mgmt.yml:108
Monday 26 August 2019  22:52:18 +0200 (0:00:00.059)       0:00:17.274 ********* 
Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mistral
<localhost> EXEC /bin/sh -c \'OS_PROJECT_NAME=service COMPUTE_API_VERSION=1.1 OS_USERNAME=octavia OS_IDENTITY_API_VERSION=3 OS_USER_DOMAIN_NAME=Default OS_CLOUDNAME=overcloud OS_AUTH_URL=https://example.net:13000/v3 NOVA_VERSION=1.1 OS_IMAGE_API_VERSION=2 OS_PASSWORD=eEGGJqMdMQEwRq8eWCypVbWXz OS_PROJECT_DOMAIN_NAME=Default OS_AUTH_TYPE=password /usr/bin/python && sleep 0\'
fatal: [localhost]: FAILED! => {
    "changed": false, 
    "cmd": "openstack image create --disk-format qcow2 --container-format bare --tag amphora-image --file /usr/share/rhosp-director-images/octavia-amphora.qcow2 --property hw_architecture=x86_64 --private octavia-amphora-13.0-20190806.1.x86_64", 
    "delta": "0:00:06.666144", 
    "end": "2019-08-26 22:52:24.917753", 
    "invocation": {
        "module_args": {
            "_raw_params": "openstack image create --disk-format qcow2 --container-format bare --tag amphora-image --file /usr/share/rhosp-director-images/octavia-amphora.qcow2 --property hw_architecture=x86_64 --private octavia-amphora-13.0-20190806.1.x86_64", 
            "_uses_shell": true, 
            "argv": null, 
            "chdir": null, 
            "creates": null, 
            "executable": null, 
            "removes": null, 
            "stdin": null, 
            "warn": true
        }
    }, 
    "msg": "non-zero return code", 
    "rc": 1, 
    "start": "2019-08-26 22:52:18.251609", 
    "stderr": "503 Service Unavailable: Insufficient permissions on image storage media: Permission to write image storage media denied. (HTTP 503)", 
    "stderr_lines": [
        "503 Service Unavailable: Insufficient permissions on image storage media: Permission to write image storage media denied. (HTTP 503)"
    ], 
    "stdout": "", 
    "stdout_lines": []
}

PLAY RECAP *********************************************************************
localhost                  : ok=11   changed=3    unreachable=0    failed=1   

Monday 26 August 2019  22:52:24 +0200 (0:00:06.829)       0:00:24.104 ********* 
=============================================================================== 
'
2019-08-26 22:52:24.992 3230 ERROR mistral.executors.default_executor Stderr: u' [WARNING]: raw module does not support the environment keyword
[DEPRECATION WARNING]: get_md5 has been deprecated along with the md5 return 
value, use get_checksum=True and checksum_algorithm=md5 instead. This feature 
will be removed in version 2.9. Deprecation warnings can be disabled by setting
 deprecation_warnings=False in ansible.cfg.

One also cannot upload the image manually:

source overcloudrc
openstack image create --disk-format qcow2 --container-format bare --tag amphora-image --file /usr/share/rhosp-director-images/octavia-amphora.qcow2 --property hw_architecture=x86_64 --property owner=b8467eb710b2478bb764d4e78c3aaf8a --private octavia-amphora-13.0-20190806.1.x86_64
503 Service Unavailable: Insufficient permissions on image storage media: Permission to write image storage media denied. (HTTP 503)