How can I diagnose fence_ilo failures when connecting to an iLO version 1 or 2 device in RHEL 5, 6?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 5 or 6 with the High Availability Add On
  • HP servers with iLO version 1 or 2

Issue

  • Are there steps that can be followed to diagnose what is causing fence_ilo to fail?
  • How to use 'fence_ilo' to troubleshoot fence issue of iLO fence agent?

Resolution

See the Diagnostic Steps below for ways to troubleshoot failed fence_ilo operations. Some common items to check are:

  • Ensure you are using the appropriate fence agent: fence_ilo for iLO 1 or 2, fence_ipmilan for iLO 3 or 4
  • Ensure that telnet functionality is enabled in the iLO management settings
  • If SSL connections are required, make sure to configure the fencedevice with ssl="on" in /etc/cluster/cluster.conf, or -z on the commandline
  • If using ssl="on" or -z, make sure SSL connections are enabled in the iLO settings
  • Ensure no firewalls are blocking access to port 23 (telnet) or port 443 (telnet_ssl)
  • Ensure the power_wait setting is not too low
  • Ensure the fence device delay is not causing failures
  • A power regulator in some hardware could interfere with iLO power off commands. HP BL495c gen5 has been validated as a potentially affected model.

Diagnostic Steps

  • Try running the command manually from the command line to check the status (-z is optional, depending on whether SSL functionality is enabled in the iLO. If it is, communication will go over port 443 for telnet_ssl)
# fence_ilo -a <iLO IP/Hostname> -l <username> -p <password> -o status -v [-z]
# fence_ilo -a node1-ilo.example.com -l admin -p mypassword -o status -v -z
  • Try manually telnetting to the iLO to see if a connection can be made:
# telnet <iLO IP/Hostname> 23
# telnet node1-ilo.example.com 23

If a connection cannot be made, check the firewall settings, IP/hostname, and network connectivity. If telnet connects successfully, try pasting the following line into the connection and see if XML data is returned:

<?xml version=\"1.0\"?>\r\n
  • If SSL is enabled, try telneting to the iLO using telnet_ssl
# /usr/lib/fence/telnet_ssl <iLO IP/hostname> 443
# /usr/lib/fence/telnet_ssl node1-ilo.example.com 443

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments