Oracle RAC setup verification script 'runcluvfy.sh' fails with error 'Could not find the storage'
Environment
- Red Hat Enterprise Linux 6
- Oracle RAC
Issue
-
We have assigned shared SAN devices to two RHEL 6 systems and have created ASM disks as per the steps in solution - How to create Oracle ASM disks using DM Multipath devices in Red Hat Enterprise Linux 6?.
-
The ASM disks created using steps in above article are accessible from both nodes and correct permissions are set on it. But when Oracle RAC cluster setup verification script
runcluvfy.shis executed, it shows following errors:$ ./runcluvfy.sh comp ssa -n node1,node2 -s /dev/oracleasm/asm01 [...] ERROR: /dev/oracleasm/asm01 Could not find the storage [...] PRVF-5150 : Path /dev/oracleasm/asm01 is not a valid path on all nodes
Resolution
-
Please use the steps in diagnostic section to verify if the ASM disks
/dev/oracleasm/asmXXare created properly on all RAC nodes and all nodes are able to read/write to same ASM disk without any issues. -
If the ASM disks are created properly and all nodes, then please try to check with Oracle support to verify if there is any issue present from the script
./runcluvfy.shdue to which it reports above error while the ASM disks are properly shared, and accessible?
Root Cause
-
The ASM disks were created using multipath devices on RAC nodes using steps in solution 272153 and both nodes were able to access the same ASM disks without any issues. After checking with Oracle support it was found that there was some issue present with
./runcluvfy.shdue to which it was incorrectly reporting following errors.$ ./runcluvfy.sh comp ssa -n node1,node2 -s /dev/oracleasm/asm01 [...] ERROR: /dev/oracleasm/asm01 Could not find the storage [...] PRVF-5150 : Path /dev/oracleasm/asm01 is not a valid path on all nodes
Diagnostic Steps
Verify if the ASM disks are properly created on both nodes:
-
Node 1:
Check the underlying multipath device pointed by symlink/dev/oracleasm/asmXX.
e.g. on node1, the is a symlink/dev/oracleasm/asm01pointed to multipath devicempathf(dm-14):[root@node1 tmp]# ls -ltra /dev/oracleasm/* lrwxrwxrwx 1 root root 8 Sep 16 18:01 /dev/oracleasm/asm01 -> ../dm-14Verify that appropriate permissions are set on
/dev/dm-XXdevice pointed by/dev/oracleasm/asmXXsymlinks:[root@node2 tmp]# ll /dev/dm-4 brw-rw---- 1 oracle asmadmin 253, 4 Sep 16 18:01 /dev/dm-14 mpathf (360060160f5a23400e0ab1a34ed371212) dm-14 DGC,RAID 5 size=1.1G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw |-+- policy='round-robin 0' prio=1 status=active | |- 1:0:1:4 sdx 65:112 active ready running | `- 2:0:1:4 sdbh 67:176 active ready running `-+- policy='round-robin 0' prio=0 status=enabled |- 1:0:0:4 sdf 8:80 active ready running `- 2:0:0:4 sdap 66:144 active ready running -
Node 2:
Check the same ASM disk, multipath device on second node:
The ASM disk/dev/oracleasm/asm01on second node is a symlink to multipath devicempathf(dm-4):[root@node2 tmp]# ll /dev/oracleasm/* lrwxrwxrwx 1 root root 7 Sep 16 18:01 /dev/oracleasm/asm01 -> ../dm-4Verify that appropriate permissions are set on
/dev/dm-XXdevice pointed by/dev/oracleasm/asmXXsymlink[root@node2 tmp]# ll /dev/dm-4 brw-rw---- 1 oracle asmadmin 253, 4 Sep 16 18:01 /dev/dm-4 mpathf (360060160f5a23400e0ab1a34ed371212) dm-4 DGC,RAID 5 size=1.1G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw |-+- policy='round-robin 0' prio=1 status=active | |- 1:0:1:4 sdx 65:112 active ready running | `- 2:0:1:4 sdbh 67:176 active ready running `-+- policy='round-robin 0' prio=0 status=enabled |- 1:0:0:4 sdf 8:80 active ready running `- 2:0:0:4 sdap 66:144 active ready running -
As per above snip, the symlink
/dev/oracleasm/asm01on both nodes is pointing to same underlying multipath device (WWID: 360060160f5a23400e0ab1a34ed371212), so the symlink is created properly on both nodes.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
