Server reboot causes incorrect major and minor numbers in Oracle ASM disks

Issue

• After a reboot, one of our database servers came up with a mismatch in major and minor numbers for Oracle ASM SAN-attached storage. This resulted in Oracle not being able to mount the disks.

• The second reboot however, had fixed the problem with all numbers looking as they should have.

• This is what the configuration looked like after the first reboot:

• We notice that on one of our server there is mismatch in device-mapper and ASM disks major number,

# /etc/init.d/oracleasm listdisks
DATAVOL0
DATAVOL1
DATAVOL2
DATAVOL3
DATAVOL4
DATAVOL5
DATAVOL6
REDO


# ls -al /dev/mapper/asm*

brw-rw-rw- 1 oracle oinstall 253, 24 Oct 23 20:45 /dev/mapper/asm-100g1

brw-rw-rw- 1 oracle oinstall 253, 10 Oct 23 20:45 /dev/mapper/asm-100g2

brw-rw-rw- 1 oracle oinstall 253, 11 Oct 23 20:45 /dev/mapper/asm-200g

brw-rw-rw- 1 oracle oinstall 253, 12 Oct 23 20:45 /dev/mapper/asm-200g1

brw-rw-rw- 1 oracle oinstall 253, 15 Oct 23 20:45 /dev/mapper/asm-200g2

brw-rw-rw- 1 oracle oinstall 253, 13 Oct 23 20:45 /dev/mapper/asm-200g3

brw-rw-rw- 1 oracle oinstall 253, 22 Oct 23 20:45 /dev/mapper/asm-250g

brw-rw-rw- 1 oracle oinstall 253, 25 Oct 23 20:45 /dev/mapper/asm-250gp1


# /etc/init.d/oracleasm querydisk -d datavol0

Disk "DATAVOL0" is a valid ASM disk on device [8, 224]

# /etc/init.d/oracleasm querydisk -d datavol1

Disk "DATAVOL1" is a valid ASM disk on device [8, 160]

# /etc/init.d/oracleasm querydisk -d datavol2

Disk "DATAVOL2" is a valid ASM disk on device [8, 176]

# /etc/init.d/oracleasm querydisk -d datavol3

Disk "DATAVOL3" is a valid ASM disk on device [8, 208]

# /etc/init.d/oracleasm querydisk -d datavol4

Disk "DATAVOL4" is a valid ASM disk on device [8, 16]

# /etc/init.d/oracleasm querydisk -d datavol5

Disk "DATAVOL5" is a valid ASM disk on device [8, 129]

# /etc/init.d/oracleasm querydisk -d datavol6

Disk "DATAVOL6" is a valid ASM disk on device [8, 192]

# /etc/init.d/oracleasm querydisk -d redo

Disk "REDO" is a valid ASM disk on device [8, 112]

• At this stage, Oracle was not able to mount the disks and the system was rebooted for the second time.

• This is what disks looked like after the second reboot:

# /etc/init.d/oracleasm listdisks
DATAVOL0
DATAVOL1
DATAVOL2
DATAVOL3
DATAVOL4
DATAVOL5
DATAVOL6
REDO


# ls -l /dev/mapper/*asm*
brw-rw-rw- 1 oracle oinstall 253, 22 Oct 24 10:36 /dev/mapper/asm-100g1
brw-rw-rw- 1 oracle oinstall 253, 23 Oct 24 10:36 /dev/mapper/asm-100g2
brw-rw-rw- 1 oracle oinstall 253, 24 Oct 24 10:36 /dev/mapper/asm-200g
brw-rw-rw- 1 oracle oinstall 253, 10 Oct 24 10:36 /dev/mapper/asm-200g1
brw-rw-rw- 1 oracle oinstall 253, 13 Oct 24 10:36 /dev/mapper/asm-200g2
brw-rw-rw- 1 oracle oinstall 253, 11 Oct 24 10:36 /dev/mapper/asm-200g3
brw-rw-rw- 1 oracle oinstall 253, 20 Oct 24 10:36 /dev/mapper/asm-250g
brw-rw-rw- 1 oracle oinstall 253, 25 Oct 24 10:36 /dev/mapper/asm-250gp1



# /etc/init.d/oracleasm querydisk -d datavol0
Disk "DATAVOL0" is a valid ASM disk on device [253, 11]
# /etc/init.d/oracleasm querydisk -d datavol1
Disk "DATAVOL1" is a valid ASM disk on device [253, 22]
# /etc/init.d/oracleasm querydisk -d datavol2
Disk "DATAVOL2" is a valid ASM disk on device [253, 23]
# /etc/init.d/oracleasm querydisk -d datavol3
Disk "DATAVOL3" is a valid ASM disk on device [253, 10]
# /etc/init.d/oracleasm querydisk -d datavol4
Disk "DATAVOL4" is a valid ASM disk on device [253, 13]
# /etc/init.d/oracleasm querydisk -d datavol5
Disk "DATAVOL5" is a valid ASM disk on device [253, 25]
# /etc/init.d/oracleasm querydisk -d datavol6
Disk "DATAVOL6" is a valid ASM disk on device [253, 24]
# /etc/init.d/oracleasm querydisk -d redo
Disk "REDO" is a valid ASM disk on device [253, 19]