Online Nova DB migration Failed

Solution In Progress - Updated -

Issue

  • We are currently migrating Openstack version OSP 9 to OSP 10.
[root@overcloud-controller-01 ~(keystone_admin)]$ su -s /bin/sh -c "nova-manage db online_data_migrations" nova
Option "rpc_backend" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
Exécution des lots de 50 jusqu'à la fin
Erreur lors de la tentative d'exécution de <function migrate_flavors at 0x7f90ab023c80>
2 lignes correspondent à la requête migrate_instances_add_request_spec, 0 migrées
Erreur lors de la tentative d'exécution de <function migrate_aggregates at 0x7f90ab080758>
Flavor's information related to instance  are not available.
Enable to create new flavors.
  • When patching nova-manage with the following line https://github.com/openstack/nova/blob/stable/newton/nova/cmd/manage.py#L897:
import sys, traceback
traceback.print_exc(file=sys.stdout)

the following errors are displayed:

$ su -s /bin/sh -c "nova-manage -d -v db online_data_migrations" nova
Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
Option "rpc_backend" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
Exécution des lots de 50 jusqu'à la fin
Erreur lors de la tentative d'exécution de <function migrate_flavors at 0x7f41cd898b90>
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nova/cmd/manage.py", line 893, in _run_migration
    found, done = migration_meth(ctxt, count)
  File "/usr/lib/python2.7/site-packages/nova/objects/flavor.py", line 716, in migrate_flavors
    flavor._flavor_create(ctxt, flavor_values)
  File "/usr/lib/python2.7/site-packages/nova/objects/flavor.py", line 462, in _flavor_create
    return _flavor_create(context, updates)
  File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 821, in wrapper
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/nova/objects/flavor.py", line 165, in _flavor_create
    raise db_exc.DBError(e)
DBError: (_mysql_exceptions.OperationalError) (1292, "Incorrect datetime value: '2018-06-27 12:43:20+00:00' for column 'created_at' at row 1") [SQL: u'INSERT INTO flavors (created_at, updated_at, id, name, memory_mb, vcpus, root_gb, ephemeral_gb, flavorid, swap, rxtx_factor, vcpu_weight, disabled, is_public) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'] [parameters: (datetime.datetime(2018, 6, 27, 12, 43, 20, tzinfo=<iso8601.Utc>), None, 608, 'an-controller', 8192, 8, 120, 0, 'f19fbfca-7913-4f63-9a1f-5b2ab9181270', 0, 1.0, 0, 0, 0)]
Erreur lors de la tentative d'exécution de <function migrate_aggregates at 0x7f41cd8f6668>
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nova/cmd/manage.py", line 893, in _run_migration
    found, done = migration_meth(ctxt, count)
  File "/usr/lib/python2.7/site-packages/nova/objects/aggregate.py", line 586, in migrate_aggregates
    _aggregate_create_in_db(ctxt, values, metadata=aggregate.metadata)
  File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 821, in wrapper
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/nova/objects/aggregate.py", line 174, in _aggregate_create_in_db
    aggregate.save(context.session)
  File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 48, in save
    session.flush()
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2027, in flush
    self._flush(objects)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2145, in _flush
    transaction.rollback(_capture_exception=True)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 2109, in _flush
    flush_context.execute()
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
    rec.execute(self)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 532, in execute
    uow
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 174, in save_obj
    mapper, table, insert)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 767, in _emit_insert_statements
    execute(statement, multiparams)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
    util.raise_from_cause(newraise, exc_info)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
DBDataError: (_mysql_exceptions.OperationalError) (1292, "Incorrect datetime value: '2015-09-10 13:28:07+00:00' for column 'created_at' at row 1") [SQL: u'INSERT INTO aggregates (created_at, updated_at, id, uuid, name) VALUES (%s, %s, %s, %s, %s)'] [parameters: (datetime.datetime(2015, 9, 10, 13, 28, 7, tzinfo=<iso8601.Utc>), datetime.datetime(2016, 11, 10, 13, 0, 23, tzinfo=<iso8601.Utc>), 2, '0583567a-061f-4e4d-929f-337080dbe910', 'fontvieille')]

Environment

  • Red Hat OpenStack Platform 10.0 (RHOSP)
  • Red Hat OpenStack Platform 9.0 (RHOSP)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In