iLo-Setup: "Test Failed, Host Status is: unknown"

Latest response

Hi there,

 

Our situation:

We put the iLo-Interfaces into a separate, private network (172.32.32.0/24). Our RHEV-Manager has the IP 172.32.32.1 on his second interface to communicate with the iLos. We've tested it with an HP ProLiant DL380 G7 (using iLo3) and HP ProLiant DL380 G5 (using iLo2).

 

Problem:

While trying to configure Power-Management in the Manager-GUI, we're confronted with following error message (from rhevm.log):

 

2012-06-13 15:49:06,371 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (http-0.0.0.0-8443-9) FINISH, FenceVdsVDSCommand, return: Test Failed, Host Status is: unknown. The fence-agent script reported the following error: Getting status of IPMI:172.32.32.12...Chassis power = Unknown

 

Our troubleshooting yet:

  • ping is ok, iLo is reachable from the manager
  • iptables are ok as well
  • tcpdump didn't gave us any usable results
  • (In the Manager-GUI) Used an IP which isn't assigned. But this also resulted in the same error

 

Did we miss something or what could be the source of this error?

 

Greetings and thanks in advance

rh
Log in to join the conversation

Responses

DY Guru 5340 points
13 June 2012 5:52 PM

RHEV-M does not itself fence hosts, it delegates a fence request to cluster nodes. So if node X needs fencing, RHEv-M finds another available node, Y, and tells Y to issue a fence_xxxx command with X as the target.

 

This means that if you have only one node in the setup - fencing will not work.

 

rh Active Contributor 234 points
14 June 2012 9:30 AM

Hi Dan

 

Thanks for your explanation. We were wondering if it could be a rhevm (RHEV-M) deployement issue: Our manager runs on a machine without any iLO port. We just configured an additional network interface in the manager for the private power management network which connects the iLO-Ports of the hypervisors.

 

Does the RHEV-M to run on a machine using a iLO port to get the Power Management test successful?

How does a typical iLO network look like using RHEV-M?

 

Thanks in advance and kind regards

DY Guru 5340 points
14 June 2012 9:47 AM

As I explained, RHEV-M does not contact the fence devices at all. What it does is contact one of the hypervisors, and tell it to run a fence command against another hypervisor. This is done over the management network. 

rh Active Contributor 234 points
18 June 2012 10:34 AM

Hi Dan,

 

Thanks for your answer. Sadly it still doesn't work.

 

So we equiped two server (hv0; hv1) with an additional NIC (let's call them eth6), to communicate with the private network 172.30.30.0/24 and reinstalled the RHEV-H software.

 

After creating a pseudo-logical network called pm_network, we configured the NICs as following:

hv0:

  • iLo2: 172.30.30.10
  • eth6: 172.30.30.100

hv1:

  • iLo2: 172.30.30.11
  • eth6: 172.30.30.110

In the Power-Management tab for each host (hv0; hv1) we configured as following:

Address: 172.30.30.100 / 172.30.30.110

Username: admin

Password:

Type: ipmilan

Options: power_wait=4,lanplus=yes

 

But the test still fails. 

 

What are we doing wrong? We're wondering how a typical iLO network could look like using RHEV-M?

 

Thanks in advance and kind regards

DY Guru 5340 points
18 June 2012 11:35 AM

According to what you posted, you're using the NIC IP for fencing instead of the ILO IP, so this will not work of course.

 

Typically, the fencing devices are placed on the rhevm network, since it's the management network and the hosts are on this network anyway, so no need for the extra NIC, extra logical network or additional IPs.

rh Active Contributor 234 points
18 June 2012 12:36 PM

Hi Dan,

 

Thanks for your answer. I'm sorry, there was a typo in our previous post.

We've configured the Power-Management with the address of the ILO, not with the address of the extra NIC (as written in previous post).

 

We are going to try it with the rhevm-network and update this post, if it works.

 

Thanks for your investigation and kind regards

DY Guru 5340 points
18 June 2012 12:44 PM

No problem. By the way, the iLOs have to be enabled for lanplus access iirc.

 

Also, if you are sure the hosts are able to contact the iLOs, and fencing still fails, the next step would be to take a look in vdsm.log on the host that is attempting to fence the other host, the output of the fence_xxxx command should be in there for diagnostics

rh Active Contributor 234 points
18 June 2012 2:14 PM

[solved]

 

Finally, we configured a dedicated network (for power management), where all the iLO interfaces and every HV with a dedicated interface are attached. For our HP Proliant DL380 GL5 (ilo) and G7 (ilo3) Hypervisors we configured the power management with the option "lanplus=yes".

Now, everything works (start, stop, restart) as well as the power managment tests.

If we extend our solution (more HVs) we probably reconfigure the iLO interfaces to be connected to the rhevm network.

 

Thx for all the helpful information.

 

Kind regards

rh Active Contributor 135 points
6 August 2012 7:13 AM

Hi everybody

 

I work with Daniel Balsiger on the power management within our RHEV environment. Currently, our power managment works meaning, we can e.g. stop hypervisors which are in maintenance mode from the admin portal.

But we're not sure if the RHEVM really powers down/up the hypervisors according to the power saving policies applied for the cluster containing all our hypervisors (3 hosts). The policy works since depending on the thresholds, VMs are migrated to/from hosts.

Is it correct that hosts without load should be shutdown? That's the behavior we expect from the power management, according to the Technical Reference section 5.1 "Power Management".

 

Thx in advance for your help

 

Kind regards,

Sebastian

DY Guru 5340 points
6 August 2012 7:22 AM

Hi Sebastian,

 

RHEV 3.0 doesn't actually power hosts down, it only prepares them for shutdown, by removing the VMs away. With no load, most modern servers will idle and save power even without restarting.

 

 

5.1. Power Management

The Red Hat Enterprise Virtualization Manager is capable of rebooting hosts that have entered a non-operational or non-responsive state, as well as preparing to power off under-utilized hosts to save power. 
This is not the same as actually shutting hosts down. There is an RFE to shut hosts down completely, in the works, but if this is very important for you, please open a support case and request this functionality - this way the existing feature request will receive more weight when it is considered for inclusion.
 
 
Thanks,
Dan