openstack commands fails with "DBError: (pymysql.err.InternalError) (1205, u'Lock wait timeout exceeded; try restarting transaction') "
Issue
ceilometer-expirer
command fails withDBError: (pymysql.err.InternalError) (1205, u'Lock wait timeout exceeded; try restarting transaction')
- Different openstack componant is failing with
InternalError: (1205, u'Lock wait timeout exceeded; try restarting transaction')
- Getting error below error in logs :
2017-11-17 11:06:24.201 8802 WARNING oslo_reports.guru_meditation_report [-] Guru mediation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (pymysql.err.InternalError) (1205, u'Lock wait timeout exceeded; try restarting transaction') [SQL: u'DELETE FROM sample WHERE sample.timestamp < %s'] [parameters: (Decimal('1510801584.364885'),)]
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters context)
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters)
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 146, in execute
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters result = self._query(query)
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 296, in _query
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters conn.query(q)
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 781, in query
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 942, in _read_query_result
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters result.read()
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1138, in read
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet()
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 906, in _read_packet
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters packet.check_error()
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 367, in check_error
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters err.raise_mysql_exception(self._data)
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 120, in raise_mysql_exception
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters _check_mysql_exception(errinfo)
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 115, in _check_mysql_exception
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters raise InternalError(errno, errorvalue)
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters InternalError: (1205, u'Lock wait timeout exceeded; try restarting transaction')
2017-11-17 11:07:15.552 8802 ERROR oslo_db.sqlalchemy.exc_filters
2017-11-17 11:07:15.557 8802 CRITICAL ceilometer [-] DBError: (pymysql.err.InternalError) (1205, u'Lock wait timeout exceeded; try restarting transaction') [SQL: u'DELETE FROM sample WHERE sample.timestamp < %s'] [parameters: (Decimal('1510801584.364885'),)]
2017-11-17 11:07:15.557 8802 ERROR ceilometer Traceback (most recent call last):
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/bin/ceilometer-expirer", line 10, in <module>
2017-11-17 11:07:15.557 8802 ERROR ceilometer sys.exit(expirer())
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/ceilometer/cmd/storage.py", line 42, in expirer
2017-11-17 11:07:15.557 8802 ERROR ceilometer cfg.CONF.database.metering_time_to_live)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/ceilometer/storage/impl_sqlalchemy.py", line 384, in clear_expired_metering_data
2017-11-17 11:07:15.557 8802 ERROR ceilometer rows = sample_q.delete()
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 3048, in delete
2017-11-17 11:07:15.557 8802 ERROR ceilometer delete_op.exec_()
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1127, in exec_
2017-11-17 11:07:15.557 8802 ERROR ceilometer self._do_exec()
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1311, in _do_exec
2017-11-17 11:07:15.557 8802 ERROR ceilometer mapper=self.mapper)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1046, in execute
2017-11-17 11:07:15.557 8802 ERROR ceilometer bind, close_with_result=True).execute(clause, params or {})
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
2017-11-17 11:07:15.557 8802 ERROR ceilometer return meth(self, multiparams, params)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
2017-11-17 11:07:15.557 8802 ERROR ceilometer return connection._execute_clauseelement(self, multiparams, params)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
2017-11-17 11:07:15.557 8802 ERROR ceilometer compiled_sql, distilled_params
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
2017-11-17 11:07:15.557 8802 ERROR ceilometer context)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
2017-11-17 11:07:15.557 8802 ERROR ceilometer util.raise_from_cause(newraise, exc_info)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
2017-11-17 11:07:15.557 8802 ERROR ceilometer reraise(type(exception), exception, tb=exc_tb)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2017-11-17 11:07:15.557 8802 ERROR ceilometer context)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2017-11-17 11:07:15.557 8802 ERROR ceilometer cursor.execute(statement, parameters)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 146, in execute
2017-11-17 11:07:15.557 8802 ERROR ceilometer result = self._query(query)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 296, in _query
2017-11-17 11:07:15.557 8802 ERROR ceilometer conn.query(q)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 781, in query
2017-11-17 11:07:15.557 8802 ERROR ceilometer self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 942, in _read_query_result
2017-11-17 11:07:15.557 8802 ERROR ceilometer result.read()
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1138, in read
2017-11-17 11:07:15.557 8802 ERROR ceilometer first_packet = self.connection._read_packet()
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 906, in _read_packet
2017-11-17 11:07:15.557 8802 ERROR ceilometer packet.check_error()
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 367, in check_error
2017-11-17 11:07:15.557 8802 ERROR ceilometer err.raise_mysql_exception(self._data)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 120, in raise_mysql_exception
2017-11-17 11:07:15.557 8802 ERROR ceilometer _check_mysql_exception(errinfo)
2017-11-17 11:07:15.557 8802 ERROR ceilometer File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 115, in _check_mysql_exception
2017-11-17 11:07:15.557 8802 ERROR ceilometer raise InternalError(errno, errorvalue)
2017-11-17 11:07:15.557 8802 ERROR ceilometer DBError: (pymysql.err.InternalError) (1205, u'Lock wait timeout exceeded; try restarting transaction') [SQL: u'DELETE FROM sample WHERE sample.timestamp < %s'] [parameters: (Decimal('1510801584.364885'),)]
2017-11-17 11:07:15.557 8802 ERROR ceilometer
Environment
- Openstack Platform 8
- Openstack Platform 9
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.