Storage : Extending an existing FC based storage domain

Latest response

Hi

I'm finding adding LUN's to an existing FC based storage domain a little erratic. Wondered if anyone else was experiencing similar.

 

So, we zone up some new luns to each of the hypervisor nodes in the a RHEV cluster. I then Edit the storage domain and my new luns appear as "Free".

I select one or more of the luns and hit OK.  I expect to see the storage domain extend in size but instead I get an error.

Error: Cannot extend Storage Domain. Storage device in unreachable from ${hostname}.

 

I suspect that one or more of the hypervisors in the cluster has not successfully "scanned" for the new LUN's. Sure enough, an inspection of multipath output shows that one (or more) of the hypervisors is not reporting on the newly added LUNs.

A manual rescan of the HBA's or a reboot of those hypervisors is required for them to successfully report the new LUNs.

We are then able to extend the storage domain.

 

Thanks

Responses

Yes, we've seen similar issues. Some times it fails to add, other times we've seen more serious issues with hypervisors getting into bad state and SPM moving around. I believe the advice we got from Red Hat was that it's not safe to modify existing SDs without taking them offline first.. This is why we ended up using one LUN per SD. If we need more storage, we just add another LUN/SD.

I have experienced the same issue and essentially had to execute the same steps to resolve.  We use RHEL 6.3 as our Hypervisors, RHEVM 3.1 and Hitachi FC storage.  What I find odd is that RHEVM sees the device, but essentially cannot work with it.  Our current procedure is this:

  1. Allocate storage to Hypervisors
  2. run rescan-scsi-bus.sh and review new LUN from Admin Portal
  3. Review new LUN from host (ls -lart /dev && multipath -ll -v2)
  4. fdisk LUN to remove partition if one is there, (partprobe /dev/sdf && partprobe /dev/sdl)
  5. Put the SPM node into maintenance and reboot.  Repeat through all the Hypervisors.
  6. Add new LUN via Admin Portal

Now - I have only done this once, and this doesn't indicate a pattern (yet ;-)  I am hopeful that next time things may go a little more smoothly.

 

This is a known issue for which we have bugs opened already. When you use a new lun to extend a stoage domain, hosts except the one in "Use Host" field do not scan automatically for the new lun which causes failure and moving the other hypervisors to non-operational status.

The easiest work around for this is below.

- Select the storage domain and click Edit.

- In the "Use Host" field select all hypervisors one by one which will trigger rescanning for new luns on all of them. Just select hosts one by one till it shows available luns. (Don't click OK)

- One finished selecting all hypervisors, extend the sd by selecting the lun and clicking OK.

Thanks for reporting this.

I wanted to test it but when I select the SD and click edit the drop down menu for "Use Host" is greyed out so I can not select any host.

Is it possible that this bug was fixed with the latest updates and that is the reason why it is greyed out? I am using 3.1.0-50.el6ev...

Balazs is correct.  It is indeed greyed out and unselectable in 3.1.0-43 and 3.1.0-50

It is selectable in 3.0.7

So, unfortunately Sadique's work around is not possible in 3.1.

If the advise received by Jan-Frode is correct, then we have a serious issue as we will be expected to extend our SD's on a regular basis and downtime on an entire SD is not viable.

We run with the black box RHEV-H and so I'm a little reluctant to start punting commands through the command line and anyway, we shouldn't have to.

Thanks very much for all your input. I will be referencing this thread in the support case and will be chasing for some sort of resolution. Even if it means re instating the manual scan ala 3.0.7