An aws-vpc-move-ip resource fails to start/stop and logs python errors in a Pacemaker cluster

Solution Verified - Updated -

Issue

  • An ocf:heartbeat:aws-vpc-move-ip resource fails to start/stop, logging errors like those shown below.
Oct 22 11:31:34 node1 aws-vpc-move-ip(vip_ers11)[4165]: INFO: EC2: Moving IP address 10.28.255.250 to this host by adjusting routing table rtb-34208b4b
Oct 22 11:31:34 node1 aws-vpc-move-ip(vip_ers11)[4165]: INFO: monitor: check routing table (API call) - rtb-34208b4b
Oct 22 11:31:34 node1 aws-vpc-move-ip(vip_ers11)[4165]: WARNING: not routed to this instance (i-022b172bb454eb810) but to instance <unknown> on rtb-34208b4b
Oct 22 11:31:35 node1 aws-vpc-move-ip(vip_ers11)[4165]: INFO: EC2: Adjusting routing table and locally configuring IP address
Oct 22 11:31:35 node1 aws-vpc-move-ip(vip_ers11)[4165]: WARNING: command failed, rc: 255
Oct 22 11:31:36 node1 aws-vpc-move-ip(vip_ers11)[4165]: ERROR: Received 1 from 'aws'
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [ [4393] Failed to execute script pyiboot01_bootstrap ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [ Traceback (most recent call last): ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "PyInstaller/loader/pyiboot01_bootstrap.py", line 127, in <module> ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "<frozen importlib._bootstrap>", line 983, in _find_and_load ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "/codebuild/output/src367604692/src/repos/awscli/.tox/exe/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "ctypes/__init__.py", line 543, in <module> ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "ctypes/__init__.py", line 273, in _reset_cache ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [ MemoryError ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [ [4597] Failed to execute script pyiboot01_bootstrap ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [ Traceback (most recent call last): ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "PyInstaller/loader/pyiboot01_bootstrap.py", line 127, in <module> ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "<frozen importlib._bootstrap>", line 983, in _find_and_load ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "/codebuild/output/src367604692/src/repos/awscli/.tox/exe/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "ctypes/__init__.py", line 543, in <module> ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [   File "ctypes/__init__.py", line 273, in _reset_cache ]
Oct 22 11:31:36 node1 pacemaker-execd[1581]: notice: vip_ers11_start_0:4165:stderr [ MemoryError ]
Oct 22 11:31:36 node1 pacemaker-controld[1587]: notice: Result of start operation for vip_ers11 on node1: 1 (error)

or

Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [ Traceback (most recent call last): ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [   File "/usr/bin/aws", line 27, in <module> ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [     sys.exit(main()) ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [   File "/usr/bin/aws", line 23, in main ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [     return awscli.clidriver.main() ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [   File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 70, in main ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [     driver = create_clidriver() ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [   File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 80, in create_clidriver ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [     event_hooks=session.get_component('event_emitter')) ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [   File "/usr/lib/python2.7/site-packages/awscli/plugin.py", line 46, in load_plugins ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [     modules = _import_plugins(plugin_mapping) ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [   File "/usr/lib/python2.7/site-packages/awscli/plugin.py", line 63, in _import_plugins ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [     module = __import__(path, fromlist=[module]) ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [   File "/usr/lib/python2.7/site-packages/awscli/handlers.py", line 20, in <module> ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [     from awscli.paramfile import register_uri_param_handler ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [   File "/usr/lib/python2.7/site-packages/awscli/paramfile.py", line 20, in <module> ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [     from botocore.httpsession import URLLib3Session ]
Jul 19 21:26:32 nodea lrmd[15659]:  notice: s4_vip_ads_stop_0:16317:stderr [ ImportError: No module named httpsession ]

Environment

  • Red Hat Enterprise Linux 7, 8 High Availability or Resilient Storage clusters
  • AWS resource class aws-vpc-move-ip deployed on the cluster

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content