Retype (migration) of a volume with NULL volume type to a non-null volume type fails

Solution In Progress - Updated -

Issue

When trying to retype a volume with the cinder retype command (of os-volume-api-version 2):

cinder --os-volume-api-version 2 retype --migration-policy on-demand <volume-uuid> <destination-volume-type-name>

The retype fails with the following error message:
Returning exception Instance <VolumeTypes at 0xaaaaaaa> has a NULL identity key.
in method:
IMPL.finish_volume_migration(context, src_vol_id, dest_vol_id)

Full error message:

2016-05-15 13:28:39.810 3271 ERROR oslo.messaging.rpc.dispatcher [req-aaaaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb ccccccccccccccccccccc - - -] Exception during message handling: Instance <VolumeTypes at 0xaaaaaaa> has a NULL identity key.  If this is an auto-generated value, check that the database table allows generation of new primary key values, and that the mapped Column object is configured to expect these generated values.  Ensure also that this flush() is not occurring at an inappropriate time, such aswithin a load() event.
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1042, in migrate_volume_completion
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     ctxt, volume_id, new_volume_id)
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/db/api.py", line 190, in finish_volume_migration
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     return IMPL.finish_volume_migration(context, src_vol_id, dest_vol_id)
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 125, in wrapper
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     return f(*args, **kwargs)
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py", line 1115, in finish_volume_migration
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     setattr(dest_volume_ref, key, value_to_dst)
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 470, in __exit__
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     self.rollback()
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     compat.reraise(exc_type, exc_value, exc_tb)
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 467, in __exit__
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     self.commit()
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 377, in commit
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     self._prepare_impl()
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 357, in _prepare_impl
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     self.session.flush()
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/cinder/openstack/common/db/sqlalchemy/session.py", line 488, in _wrap
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher     raise exception.DBError(e)
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher DBError: Instance <VolumeTypes at 0xaaaaaaa> has a NULL identity key.  If this is an auto-generated value, check that the database table allows generation of new primary key values, and that the mapped Column object is configured to expect these generated values.  Ensure also that this flush() is not occurring at an inappropriate time, such aswithin a load() event.
2016-05-15 13:28:39.810 3271 TRACE oslo.messaging.rpc.dispatcher 
2016-05-15 13:28:39.811 3271 ERROR oslo.messaging._drivers.common [req-aaaaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb ccccccccccccccccccccc - - -] Returning exception Instance <VolumeTypes at 0xaaaaaaa> has a NULL identity key.  If this is an auto-generated value, check that the database table allows generation of new primary key values, and that the mapped Column object is configured to expect these generated values.  Ensure also that this flush() is not occurring at an inappropriate time, such aswithin a load() event. to caller

When this issue happens, cinder will be in a state where there is the original volume in migration_status completing and also a temporary volume which is used during the retype process in status attaching. In cinder logs, one can see

016-07-18 14:48:39.716 3271 DEBUG cinder.volume.manager [req-fffffffffffffffffffffffffffffffffffffffffff <user-uuid> <project-uuid> - - -] migrate_volume_completion: completing migration for volume <volume-uuid> (temporary volume <temporary-volume-uuid> migrate_volume_completion /usr/lib/python2.7/site-packages/cinder/volume/manager.py:1008
2016-07-18 14:48:39.808 3271 ERROR cinder.openstack.common.db.sqlalchemy.session [req-fffffffffffffffffffffffffffffffffffffffffff <user-uuid> <project-uuid> - - -] DB exception wrapped.
2016-07-18 14:48:39.808 3271 TRACE cinder.openstack.common.db.sqlalchemy.session Traceback (most recent call last):

How can one reset the original volume to its original state and also prepare it for retype. How can one delete the temporary volume? How to avoid this issue in the future for existing volumes and volumes yet to be created?

Environment

Red Hat Enterprise Linux OpenStack Platform 5.0
Red Hat Enterprise Linux OpenStack Platform 6.0
Red Hat Enterprise Linux OpenStack Platform 7.0

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In