The RHEV-M portal fails to load giving errors after the engine-setup command failed during RHEVM upgrade.
Environment
- Red Hat Enterprise Virtualization 3.5
- Self- hosted RHEV
Issue
ovirt-enginethrowing java errors after attempting to upgrade RHEVM versions from 3.5.0.0 to 3.5.7.0 with error asERROR: function get_all_commands() does not existRHEV-Mupgrade failed with below messages in the logs:
2016-01-15 16:09:28 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine_reports.jasper.deploy plugin.execute:937 execute-output: ('./js-export.sh', '--output-dir', '/tmp/tmpW1r2Ka/users', '--users', '--roles') stderr:
2016-01-15 16:09:28 DEBUG otopi.context context._executeMethod:152 method exception
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/otopi/context.py", line 142, in _executeMethod
method['method']()
File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-reports/jasper/deploy.py", line 638, in _export
'--roles',
File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/reports/reportsutil.py", line 129, in jsexport
oreportscons.FileLocations.
File "/usr/lib/python2.6/site-packages/otopi/plugin.py", line 942, in execute
command=args[0],
RuntimeError: Command './js-export.sh' failed to execute
2016-01-15 16:09:28 ERROR otopi.context context._executeMethod:161 Failed to execute stage 'Misc configuration': Command './js-export.sh' failed to execute
Resolution
1] Put to RHEVM VM to global maintenance mode.
2] Then roll back the update with the below command:
# yum history undo <ID-of-Update-from-yum-history-cmd>
3] Stop the below services before proceeding with the rhev db restore:
# service ovirt-engine stop
# service ovirt-engine-dwhd stop
# service ovirt-engine-reportsd stop
# service ovirt-fence-kdump-listener stop
4] Empty the engine database :
# su postgres -c psql
postgres=# DROP DATABASE engine;
postgres=# CREATE DATABASE engine OWNER engine template template0 encoding 'UTF8' lc_collate 'en_US.UTF-8' lc_ctype 'en_US.UTF-8';
postgres-# \q
5] Restore db using below command:
# su - postgres
- Then run the below command, to restore the db:
# pg_restore -d engine /tmp/engine-20160115160530.gjv2JT.dump
- This restored the db successfully.
6] Further, restart the ovirt-engine service and check.
Root Cause
- The upgrade did not complete because the
js-export.shfailed to complete successfully, when something fails during updateengine-setupwill attempt to roll back, butengine-setupstops before it can successfully roll back.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
